Skip to content

feat(dynaclr): read per-FOV tracks from in-plate tracks.geff - #525

Open
ieivanov wants to merge 2 commits into
mainfrom
feat/dynaclr-geff-tracks
Open

ieivanov wants to merge 2 commits into
mainfrom
feat/dynaclr-geff-tracks

Conversation

@ieivanov

@ieivanov ieivanov commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Summary

biahub track now writes tracks inside the image plate FOV: <plate>/<row>/<col>/<fov>/tracks.geff (a GEFF graph, zarr v3, geff >= 1.3) and, for now, also tracks_<row>_<col>_<fov>.csv. This PR lets DynaCLR read them, with no separate tracking.zarr.

Pairs with Biohub/biahub#413 (merged; closes Biohub/biahub#360 / #361). Updated to its merged layout: the FOV also holds the label images (labels/<model> from biahub segment, labels/tracks_<model> from biahub track, (T, 1, Z, Y, X) uint32), the GEFF's seg_id is the track_id painted into the tracked one, and a track run that names its tracked label (output_label) writes tracks_<name>.geff / tracks_<name>_<fov>.csv instead of tracks.geff / tracks_<fov>.csv.

Changes

  • viscy_data.tracks.read_fov_tracks(tracks_root, fov_path) (new): reads <root>/<fov>/tracks.geff when it exists and returns the same DataFrame as the ultrack CSV: track_id, t, [z], y, x, id, parent_track_id, parent_id.
    • Node ids become id.
    • parent_id is the source node of the incoming edge (-1 for roots).
    • parent_track_id is read from the node property if stored. Otherwise it is rebuilt as in ultrack's to_tracks_layer: the track of the parent of the track's first node, constant over the track, -1 for root tracks.
    • Only the needed node props are loaded (via geff.core_io.read_to_memory, no networkx). The track property name is taken from track_node_props["tracklet"] when present.
    • Without tracks.geff, exactly one tracks_<name>.geff is read (several raise); without any GEFF it falls back to the existing exactly-one-*.csv behavior, with the same error messages.
    • It replaces the three duplicated readers in cell_index._build_experiment_tracks, dynaclr.data.index._load_experiment_fovs and TripletDataModule._align_tracks_tables_with_positions.
  • ExperimentEntry.tracks_path is now optional and defaults to data_path. Empty and null values also default to data_path. The parquet schema still has the tracks_path column, filled with the resolved path.
  • geff is an optional extra: viscy-data[tracks], lazily imported, with a clear ImportError. It is also included in [all] and the test group. dynaclr depends on viscy-data[tracks,triplet], because its pyproject says required runtime deps should not sit behind extras.
  • airtable prepare:
    • If the NFS plate has in-plate tracks (*/*/*/tracks*.geff), 01_concatenate.sh copies each FOV's GEFF(s), tracks_*.csv and labels/ (the segmentation and tracked label images the GEFF links to) into the same FOV of the VAST plate (biahub concatenate only copies image arrays). No tracking.zarr is created in that case.
    • Datasets with a separate tracking zarr keep the existing rsync step.
    • prepare status reports tracking: in-plate.
    • tracks_path is dropped from the parquet-readiness required fields.
  • Docs: the build-cell-index.md recipe gains a "Where tracks are read from" section. The airtable README is updated.

Note on uv.lock

The lock on main was already out of date (uv lock --check fails on main). It still had orphaned entries such as napari, ultrack and an editable napari-geff path. Re-locking removes those and bumps tracksdata from a git rev to rc11; the same happens when re-locking main with no changes. The only change caused by this PR is geff 1.2.0 -> 1.3.1.

Test plan

  • New packages/viscy-data/tests/test_tracks.py:
    • A synthetic GEFF written with geff write_arrays(zarr_format=3), with a division and shuffled node order. It is checked against the equivalent ultrack CSV in 2D and 3D, with parent_track_id both derived and stored.
    • A missing track_id raises.
    • The GEFF is preferred over a CSV in the same FOV.
    • CSV fallback, plus the no-CSV and multiple-CSV errors.
    • tracks_path defaults to data_path.
    • End-to-end build_timelapse_cell_index from an in-plate GEFF without tracks_path, including the lineage across the division.
  • New applications/airtable/tests/test_prepare.py (concatenate script variants, in-plate detection incl. tracks_<name>.geff, the labels/ include). The generated in-plate copy step was also run with bash on a synthetic NFS/VAST layout with a named GEFF and two label images: GEFF, CSV and labels/ arrive, the image array does not.
  • pytest packages/viscy-data/tests: all pass (2 skips: pycocotools, CUDA).
  • pytest applications/airtable/tests: 85 passed.
  • pytest applications/dynaclr/tests/{test_index,test_experiment,test_flat_parquet_e2e,test_dataset}.py: 104 passed.
  • pre-commit (ruff, pyproject-fmt) clean.
  • Checked against a real biahub track output (2 FOVs of 2026_08_11_A549_SEC61B_DENV_debug/v8, tracked with feat(segment,track)!: segment and track write into the plate; labels and tracks.geff (#360, #361) biahub#413): read_fov_tracks on tracks.geff equals the CSV for both FOVs.

🤖 Generated with Claude Code

biahub track now writes tracks inside the image plate FOV as
<plate>/<row>/<col>/<fov>/tracks.geff (GEFF, zarr v3) plus a tracks CSV.

- viscy_data.tracks.read_fov_tracks: read <root>/<fov>/tracks.geff when
  present and return the ultrack CSV layout (track_id, t, [z], y, x, id,
  parent_track_id, parent_id; parent_* rebuilt from edges when not stored),
  otherwise fall back to exactly one *.csv as before. Replaces the three
  duplicated CSV readers in cell_index, dynaclr index and triplet.
- ExperimentEntry.tracks_path is optional and defaults to data_path.
- geff is an optional viscy-data extra ([tracks]), lazily imported;
  dynaclr depends on viscy-data[tracks].
- airtable prepare: copy each FOV's tracks.geff + tracks CSV into the VAST
  plate for in-plate datasets instead of rsyncing a separate tracking.zarr
  (unchanged for older datasets); tracks_path no longer required for the
  parquet readiness report.
- Docs: build-cell-index recipe and airtable README.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…plate labels

`biahub track` as merged (Biohub/biahub#413) still writes `tracks.geff`
by default, but a run that names its tracked label (`output_label`) writes
`tracks_<name>.geff` / `tracks_<name>_<fov>.csv`, and the FOV now also holds
the label images: `labels/<model>` from `biahub segment` and
`labels/tracks_<model>` from track, which the GEFF's `seg_id` indexes.

- `read_fov_tracks` reads exactly one `tracks_<name>.geff` when there is no
  `tracks.geff` (several raise), before the CSV fallback.
- airtable `prepare`: in-plate detection matches `tracks*.geff`; the copy step
  carries the GEFF(s), the CSV and `labels/` into the VAST FOV, since
  `biahub concatenate` copies image arrays only.
- Docs name the layout; tests cover the named GEFF and the labels include.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@ieivanov
ieivanov marked this pull request as ready for review October 24, 2026 01:01
@ieivanov

Copy link
Copy Markdown
Contributor Author

@edyoshikun and team, Biohub/biahub#413 merged so new reconstructions will have tracking in geff format. We've still kept the .csv files as legacy format. I suggest merging this PR which will allow you to read the new label arrays and geff tracks.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DCA: track writes segmentations as OME-NGFF labels directly into the assembled store

1 participant