[alxndrkalinin]

Summary

Second of three stacked PRs that replace #494. This one ports applications/dynacell from dynacell-models (8e74a7e) without the Spotlight v2 experiments. #533 (shared packages) is merged (squash 4a4ac9ff); this PR now targets main, with main merged in at d074562f.

• PR-A: shared packages, root, CI, lock.
• PR-B (this PR): DynaCell v1.1: benchmark configs, evaluation pipeline, launcher, tools, lite benchmark, 2D track, HF demo, docs. 1,825 files, +128k/−8.9k against PR-A.
• PR-C: Spotlight v2 arms, eval buckets and tools. It is based on this branch.

After all three merge, main is the DynaCell dev line and dynacell-models stays frozen as the paper-cited snapshot.

Review guide

• Most of the size is generated YAML. configs/ is 1,522 files (+73k). The leaves come from the generators (tools/generate_*), and guard tests pin them: test_benchmark_config_composition, test_evaluate_compose, test_paths, and the generators' own *_test.py. Review the generators and the guard tests, not each leaf.

• Code (src/, tools/, tests/) is 274 files, +53k/−4k, of which src/ is +28k. It is a verbatim copy of dynacell-models except the trim commits listed below.

• Commit layout:

  1. 81a74c92 configs, verbatim;
  2. cdbfcfc6 code, verbatim;
  3. lock, marker, CI and CLAUDE.md;
  4. the four trim commits (e0be4c68, d4277d86, 898e2e14, 9cee1166) and the /code-review fixes (6cb538e6, d86c682f, 76e307f8, 769cb04c, eb9dc300, f21ed1e8, b96ae3f7). These are the only hand edits to DynaCell files;
  5. merges of Port shared VisCy package changes from DynaCell v1.1 #533's review fixes (50f9fe1f, ec725171, d3dc0ce8, 0b07f26c, a04b3eb5);
  6. the live-tree fix for the grouped walk and audit (97c30a1e..8ebca438, see below).
  7. d7986fde deletes seven finished one-shot tools and their tests (path-migration trio, deconv re-tokenizer, NaN-MicroMS3IM scan, instance-AP codemod, joint-leaf smoke), raised in review by edyoshikun. Full suite after it: 5251 passed, 0 failed.

  Review them as a diff against dynacell-models: git diff origin/dynacell-models HEAD -- <file>.

Engine hooks (default off)

engine.py, celldiff_wrapper.py and mask_conditioning.py are exactly as on dynacell-models. The three LightningModules keep the Spotlight training hooks (seg_aux, seg_consistency, mask_mode, target_bg_lowpass):

• every hook defaults off, and bit-identity tests in test_training_integration.py guard that (e.g. test_unet_seg_aux_none_is_bit_identical_to_the_base_path, test_flow_matching_bg_lowpass_off_is_bit_identical_with_fg_mask_in_the_batch);
• only Spotlight leaves (PR-C) turn them on;
• a hook-free engine never existed at any commit, and two DynaCell-paper 2D checkpoints (pix2pix2d_unetvit_wmatch, fcmae_vscyto2d_scratch_samplematched) store seg_aux_weight in their hparams, so they could not resume against a hook-free class.

Behavior change for Spotlight v1 users: fg_mask is re-binarized > 0.5 before it reaches SpotlightLoss.

Trim edits (the only differences from dynacell-models besides the missing Spotlight files)

git diff --stat origin/dynacell-models HEAD lists 794 files (the live-tree fix changed lines inside the generator files listed below; no new file):

• 725 Spotlight files that PR-C adds;
• 40 2D seed1 leaf headers;
• the 9 files below;
• from PR-A: lint.yml, test.yml, uv.lock, the 11 packages/viscy-utils files of Port shared VisCy package changes from DynaCell v1.1 #533's review fixes, and the 6 VCP tutorial files.

1. tools/generate_grouped_eval_configs.py (e0be4c68, refined in 6cb538e6, 769cb04c, eb9dc300, f21ed1e8):
   • drops the ARMS import from the Spotlight leaf generator and the Spotlight names from _CODE_TO_PAPER / _DETERMINISTIC_MODELS / _CELLDIFF_MODELS. It keeps the four 2D *_seed1 replicates the 2D appendix scores.
   • The canonical prediction tree is shared and still holds the arms' stores. One predicate, is_foreign_model_dir, picks out a dir whose model paths.PAPER_KEY knows while this generator neither parses it nor lists it in _CODE_TO_PAPER. walk_predictions and the instance-AP coverage audit both skip those dirs before parsing, and the walk prints the count. Without this, parse_zarr_name raises on the first arm (unknown CellDiff variant 'segaux_iterative').
   • Everything else still raises (walk) or is flagged (audit), as on dynacell-models: an unknown dir, or a _CODE_TO_PAPER model that does not parse (the wmatch dirs).
   • MEASURED on the live tree, the audit's error set is identical to dynacell-models' (102 entries, compared path by path).
   • _match_model_variant is now the single model-dir acceptance rule; _split_model_variant wraps it with the same errors.
   • Tests cover the skip, both raises and walk/audit agreement. PR-C keeps the filter.
2. tests/test_benchmark_config_composition.py, tests/test_paths.py (d4277d86): drop the v1 *_spotlight leaf entries, the Spotlight members of the wide-GPU and long-wall sets, and the two test_paths tests that import the Spotlight generator or need the grouped generator to register the arms.
3. 40 {nucleus,membrane}/{fnet2d,celldiff_2d,fcmae_vscyto2d_scratch,pix2pix2d_unetvit}_seed1 leaves (898e2e14): the header pointed at generate_spotlight_v2_leaves.py, which is not in this PR. It now says what the leaves are for. The change is comments only. PR-C restores the generated header, because that generator's test pins each leaf byte for byte.
4. evaluation/README.md, tests/test_evaluate_compose.py (9cee1166): no longer list the Spotlight eval generator.
5. tools/generate_instance_ap_eval_configs{,_test}.py (eb9dc300): the audit applies the same is_foreign_model_dir skip, with a test that has one registered, one foreign and one unknown store.
6. evaluation/paths.py, tests/test_paths.py (b96ae3f7): comments only. They no longer cite tools/generate_spotlight_v2_leaves.py, which this branch lacks, and the Spotlight model names stay. PR-C restores the original text.

Moved from PR-C to here (76e307f8, verbatim):

• tests/test_seg_consistency_{training,focus}.py. They test this PR's engine hook and focus rule with a stand-in segmenter.
• src/dynacell/mask_conditioning.py, which engine.py and celldiff_wrapper.py import.

Kept on purpose: the Spotlight model names in evaluation/paths.py (PAPER_KEY etc.) and the test_paths tests of those names. canonical_model_name resolves by longest prefix, so without these names an arm's run dir (fnet2d_segaux) would silently resolve onto its baseline (fnet2d).

Root changes on top of PR-A

• uv.lock: dynacell-models' lock verbatim, which this tree's pyprojects match (uv lock --check exit 0). Against PR-A it moves cubic 0.7.0 → 0.9.0a4 and adds dinov3 (pinned commit), mlcroissant, rdflib, etils, submitit and others. A fresh uv lock would float dinov3 to its current HEAD.
• pyproject.toml: drops the requires_data marker PR-A registered, since its only user (main's grouped-generator test) is replaced.
• pyproject.toml: re-adds the applications/dynacell/examples/hf_demo pytest path, for build_demo_data_test.py.
• test.yml: the dynacell job gets back test_submit_benchmark_batch and the eight test files PR-A left out. Like PR-A, it no longer repeats the four viscy-utils test files that the new test-utils job runs.
• CLAUDE.md gets the hung-job / stall-watchdog / wall-limit runbook.

Live-tree fix for the grouped eval walk (FOLLOWUPS #1)

On dynacell-models, walk_predictions() and the instance-AP coverage audit fail on the live prediction tree.

Before the fix (MEASURED):

• the walk raised unknown train_set token 'ipsc_final';
• the audit returned 102 errors.

This PR now handles every known non-canonical dir explicitly. Anything else still raises.

• Parsed, sibling-only: pix2pix2d_unetvit_wmatch{,_seed1} (56 stores). _CODE_TO_PAPER already declared them a parse-only registration, but _DETERMINISTIC_MODELS lacked them. The committed *__2d_focus2d_pix2pix2d_wmatch buckets score them. _SIBLING_ONLY_MODELS keeps them out of the canonical grouped buckets and the instance-AP leaves; the walk still returns them, so sibling buckets can be built from it.
• Skipped, with a count:
  • _CHECKPOINT_REPREDICTS: 12 (organelle, model, train set) triples, 34 stores. These are checkpoint variants from the voxel-matched FNet-2D and sample-matched UNeXt2-2D ablations (ipsc_step8400, ipsc_step11k, ipsc_step200k, ipsc_final, ipsc_repredict). Hand-written experiment buckets score them, and no committed config references them.
  • _FOREIGN_MODEL_DIRS: the three P1 segcons replicate dirs (12 stores). Neither registry nor PAPER_KEY knows them.
• The walk and the audit share one rule (skip_reason). Guard tests check that every _CODE_TO_PAPER model parses and that the skip lists stay disjoint from the registries.

After the fix (MEASURED on the live tree, 1,148 stores):

• the walk completes: 684 parsed, 378 foreign-skipped, 34 re-predicts skipped, 25 R1 CellDiff skipped, 27 on out-of-scope test sets;
• the audit returns 0 errors.

Regression proof (MEASURED, both generators run the default way, without --out-root or a filter):

• 0 wmatch blocks in any canonical {org}_{train_set} bucket or instance_ap leaf.
• The canonical pool main() buckets (628 stores) is identical, store by store, to what the pre-fix rules parse. Bucketing and emission code is unchanged, so a default regeneration writes exactly what the pre-fix generator would have written if its walk had not crashed.
• The committed leaves are still not byte-identical after a default regeneration, for a pre-existing reason. They were frozen under DON'T-FOLD before the 2D, ablation and brightfield stores existed, so the regeneration adds 492 blocks from stores the pre-fix rules also parse.
• Every committed condition block is reproduced verbatim. Filtered to the committed conditions, all 16 canonical leaves are byte-identical.

Known limits

• The SegConsistencyLoss docstring names dynacell.cellpose_dense, which arrives with PR-C.

Tests

Local runs: pytest -n 8 on an A40 node, in this branch's own venv (uv sync --all-packages --all-extras --frozen: cubic 0.9.0a4). The root addopts deselect slow.

Suite	Result
applications/dynacell -k 'not test_combined_ddp'	4100 passed, 134 skipped, 0 failed (at 8ebca438; one earlier run at that head failed test_ssim_real_cubic_2d_and_3d_identical_scores_one, which depends on test order; it is pre-existing and unchanged from dynacell-models)
packages/	1219 passed, 1 xfailed, 0 failed

Composition guard runs 0 Spotlight leaves. test_benchmark_config_composition.py collects 1,773 tests here, and 0 of their ids match a Spotlight arm token. The same regex matches 705 of 2,478 ids on PR-C, so the check can tell the two apart. The only fnet3d_vscyto3daug ids are nucleus/fnet3d_vscyto3daug, the DynaCell patch-vs-augmentation ablation.

Lint: the pre-commit hooks pass on every commit.

CI: green on d074562f after the retarget (29 checks pass, 1 skipped).

Merging main (d074562f). #533 landed as a squash while this branch carried its commits only up to 0b1cb979, so four files conflicted against the pre-#533 merge base. test.yml, root pyproject.toml and uv.lock keep this branch's side (main's copy equals #533 at 0b1cb979, which this branch only adds to; uv lock --check passes); applications/dynaclr/tests/conftest.py takes main's side (#533's pandas infer_string fix; this branch never touched it). Every merged file equals this branch's copy where it changed the file and main's otherwise. Full repo pytest -n 11 at d074562f: 5269 passed, 0 failed, 138 skipped, 1 xfailed.

Review triage and follow-ups are recorded with the split (/code-review triage per finding, and a FOLLOWUPS list of the issues that are pre-existing on dynacell-models).

🤖 Generated with Claude Code

[Copilot]

🔵 Needs a closer look

The changes require final human review because they are too complex or risky for automated approval.

No open issues 🎉

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

The changes require final human review because they are too complex or risky for automated approval.

No open issues 🎉

_{Review effort: Balanced}

[edyoshikun]

this is a pretty solid and huge PR with most of the changes being the addition of the all the config files + tests.

Not sure how much you plan to reuse things but the /tools/*.py has a bunch of stuff.

Otherwise LGTM!

[alxndrkalinin]

@edyoshikun thanks for pointing out the leftovers in tools, it cleaned it up