[alxndrkalinin]

Summary

Third of three stacked PRs that replace #494. This one adds the Spotlight v2 segmentation-aware-loss experiments. It is based on #534 (DynaCell v1.1), which is based on #533 (shared packages).

Spotlight is not part of the DynaCell paper, so its experiments get their own PR. The shared Spotlight losses (SegAuxDice, SAUNA/clDice maps, SegConsistencyLoss, BackgroundLowPass) are already in #533. The default-off engine hooks and mask_conditioning.py are in #534. Everything stays under applications/dynacell: every Spotlight runtime piece is dynacell.* (engine classes, eval pipeline, paths registry, launcher), and an applications/spotlight that imports dynacell would break the repo's no-sideways-imports rule. That move waits until the DynaCell LightningModules live in a package.

Contents (725 new files + 4 reverted trims)

• Configs, all new (e9ad5c71, 714 files):
  • 479 arm leaves in 97 model dirs. Arms: segaux, segauxself, SAUNA/clDice, segcons + control, Track H bglp/bgflat, Stage 1b ccond/cjoint, the half/double weight sweep, cpout. Also the fresh v2 baselines, seed replicates and the small controls.
  • the four *_spotlight_fit* model overlays;
  • the 231 spotlight_v2_* grouped eval buckets.
• Tools and tests, all new (d77a21d0, 13 files; DynaCell v1.1 application #534 later took over the two seg_consistency tests, so 11 remain new here):
  • the generators generate_spotlight_v2_leaves.py and generate_spotlight_v2_eval_configs.py, plus the eval roster;
  • the mask tools write_otsu_thresholds.py and write_classical_fg_masks.py;
  • cellpose_dense.py (the differentiable Cellpose behind the P1 segcons loss);
  • their tests and the seg_consistency focus/training tests.
• The trims DynaCell v1.1 application #534 made, reverted to the dynacell-models text (46e30fb9, a0997f9d, 1d346d34, e2fc238a):
  • the Spotlight registrations and _SPOTLIGHT_ARM_MODELS skip in the grouped generator;
  • the Spotlight entries of the config guards and test_paths;
  • the generated headers of the 40 2D seed1 leaves (generate_spotlight_v2_leaves_test pins them byte for byte);
  • the generator lists in evaluation/README.md and test_evaluate_compose.py.
• Kept from DynaCell v1.1 application #534: the shared skip rule (skip_reason) in the walk and the instance-AP audit, and its tests. This includes DynaCell v1.1 application #534's live-tree fix (FOLLOWUPS Migration from microDL #1). MEASURED on the live tree at 3b07b0d9:
  • walk_predictions() completes with 652 parsed stores. The 2D seed1 replicates are skipped through _SPOTLIGHT_ARM_MODELS, as on dynacell-models.
  • The audit returns 0 errors (dynacell-models: 102).
  • A default regeneration emits 0 wmatch blocks.
  • The canonical pool (596 stores) is identical, store by store, to dynacell-models' own tolerant parse.

Tree equivalence with dynacell-models

At this branch's head, git diff --stat origin/dynacell-models HEAD lists 24 files (+544/−119). Every other path, including engine.py, evaluation/paths.py, the root pyproject.toml and CLAUDE.md, is byte-identical to dynacell-models 8e74a7e. The full list, with a reason per file, is in the split's EQUIVALENCE record. In summary:

File(s)	Why it differs
.github/workflows/lint.yml, test.yml	no dynacell-models PR target; new test-utils job; the dynacell job no longer repeats the viscy-utils tests (#533)
tools/generate_grouped_eval_configs{,_test}.py, tools/generate_instance_ap_eval_configs{,_test}.py	#534's shared skip rule for the walk and the instance-AP audit, the single model-dir acceptance rule, the live-tree fix (wmatch parsed; listed checkpoint re-predicts and named P1 segcons replicates skipped; registry guards), and their tests
11 packages/viscy-utils files, uv.lock	#533's review fixes: torchvision declared, lazy viscy_utils.callbacks, CLI-injector cleanup, guard for fits without callbacks, step-cadence last.ckpt, save_last taken from every monitored checkpoint, zero-range CLAHE guard, check_finite removed, docstrings
6 VCP tutorial files (.py + .ipynb)	install viscy-data from GitHub, since PyPI 0.1.0 lacks what the writer reads (#533)

So the three PRs together carry all 2,626 paths of #494, and nothing else changes.

Tests

Local run: pytest -n 8 on an A40 node, in this branch's own venv (uv sync --all-packages --all-extras --frozen).

Suite	Result
applications/dynacell -k 'not test_combined_ddp'	5058 passed, 197 skipped, 0 failed (at 09a32857)

test_benchmark_config_composition.py collects 2,478 tests here (1,773 on #534). 705 of them are Spotlight leaves.

CI: pending retarget. This stacked PR gets no checks until it retargets to main, and it must be green then, before any merge.

Review guide

660 of the 714 config files carry a generator header: 427 leaves from generate_spotlight_v2_leaves.py, whose test pins every committed leaf to its output, and 233 from the eval generator. Review the generators and their tests rather than those files. The 54 hand-written ones are:

• the eight v1 {fnet2d,fnet3d,pix2pix2d_unetvit,pix2pix3d_unetvit}_spotlight dirs (nucleus, membrane);
• the ER/mito fnet3d_vscyto3daug arm templates;
• the four *_spotlight_fit* overlays.

The only lines that differ from dynacell-models are in the table above.

A /code-review of the re-added hunks confirmed:

• the 23-file equivalence (24 with the audit test DynaCell v1.1 application #534 added since);
• generate_spotlight_v2_leaves.py --check: 467 leaves, 0 stale;
• no Spotlight condition in any canonical bucket.

Its other findings concern code that is identical on dynacell-models and are left as follow-ups:

• the _SPOTLIGHT_ARM_MODELS comment ignores the 2D seed1 replicates the 2D appendix scores;
• the long-wall leaf list vs its comment, and the CLAUDE.md "measured only" rule;
• arm tokens triplicated by hand in the grouped generator;
• the seed1 headers cite a personal plan path.

🤖 Generated with Claude Code

[Copilot]

🔵 Needs a closer look

The changes require final human review because they are too complex or risky for automated approval.

No open issues 🎉

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

The changes require final human review because they are too complex or risky for automated approval.

No open issues 🎉

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

The changes require final human review because they are too complex or risky for automated approval.

No open issues 🎉

_{Review effort: Balanced}