Strange failures

[strengejacke]

I'm not sure why this happens now, but it looks like snapshot tests are now "failing" in general? Or is there something I have missed?

https://github.com/easystats/modelbased/actions/runs/19927599627/job/57131184167?pr=586

@easystats/core-team

[IndrajeetPatil]

This means that we haven't committed snapshots to version control. It's a change that was introduced in testthat 3.3.0:

[strengejacke]

Ah, I see. But are snapshot tests now located in a different directory?
I see these failures: https://github.com/easystats/insight/actions/runs/19955715151/job/57224326692?pr=1154
although snapshot files are on GitHub:
https://github.com/easystats/insight/tree/main/tests/testthat/_snaps

[IndrajeetPatil]

No, they are not. Just search for string Adding new file snapshot in the CI logs, and you will see which snapshots are missing:

E.g.

── Failure ('test-residualize_over_grid.R:56:3'): plots show_resisuals ─────────
Adding new file snapshot: 'tests/testthat/_snaps/residualize_over_grid/plot-show-residuals-3.svg'

If you are creating OS-specific snapshots, you also need to check if OS-specific snapshots are present.

[strengejacke]

Yes, for modelbased. But my other example was referring to insight

[IndrajeetPatil]

There are so many other issues in insight that I don't know what's causing what issue:

• Many warnings:

══ Warnings ════════════════════════════════════════════════════════════════════
── Warning ('test-clean_parameters.R:157:3'): clean_parameters brms, sigma3 ────
namespace 'cmdstanr' is not available and has been replaced
by .GlobalEnv when processing object 'brms_sigma_3'

── Warning ('test-get_predicted.R:262:3'): get_predicted - lmerMod ─────────────
the 'findbars' function has moved to the reformulas package. Please update your imports, or ask an upstream package maintainter to do so.
This warning is displayed once per session.

Warning: Undocumented arguments in Rd file 'is_converged.Rd'
  ‘verbose’

• New snapshots

Some snapshots are missing:

── Failure ('test-display.R:10:3'): display, matrix ────────────────────────────
Adding new snapshot:
Code
  as.character(display(mdat))
Output
  [1] "|     | C.1| C.2| C.3|" "|:----|---:|---:|---:|" "|row1 |   1|   2|   3|"
  [4] "|row2 |  11|  12|  13|"

It's adding snapshots for windows:

── Failure ('test-export_table.R:147:3'): export_table, table_width ────────────
Adding new snapshot for variant 'windows':

[strengejacke]

Yeah, but that snaps are already on GitHub:
https://github.com/easystats/insight/tree/main/tests/testthat/_snaps/windows

That's why I don't understand why it's failing.

And the display-snapshot is also available?
https://github.com/easystats/insight/tree/main/tests/testthat/_snaps

[strengejacke]

@easystats/core-team Anyone an idea how to fix snapshot tests? CI checks are pretty useless at the moment because of these failures - GitHub CI complains that snapshots are missing, and thus all snapshot tests fail (e.g. https://github.com/easystats/modelbased/actions/runs/22053455021/job/63716114514), however, they're definitely there and there are no failures for local tests.

[etiennebacher]

I can take a look later

[strengejacke]

The only apparent different I saw between the "longer" workflows that include old-rel and the shortened workflows I use was:
20cf0ed

Will re-run checks on modelbased now and see if the issues still persist (running on main https://github.com/easystats/modelbased and a PR easystats/modelbased#596)

[etiennebacher]

I think I managed to do a reprex here: https://github.com/etiennebacher/test.snapshot.ci

CI fails because it generates a snapshot: https://github.com/etiennebacher/test.snapshot.ci/actions/runs/22963378747/job/66660090631

But the snapshot is already there: https://github.com/etiennebacher/test.snapshot.ci/blob/main/tests/testthat/_snaps/windows/print.md

I'll check later if this is due to an update in testthat and I'll open an issue in testthat.

[etiennebacher]

r-lib/testthat#2329

[strengejacke]

Thanks for investigating!