[tayllatheodoro]

Summary

biahub track can now choose its tracker. Besides ultrack (still the default), a track config can set:

• tracker: tracksdata: tracksdata's ILP on the cellpose labels (region nodes, distance + IoU edges).
• tracker: hoct: HOCT's learned edge scores, then an ILP.
• tracker: trackastra: Trackastra's pretrained transformer predicts the links (greedy or ILP).

These three link the instance labels of the label image biahub segment wrote (labels:, the default segmentation_method: labels) as they are, so they need output_mode: 2D on a projected (Z=1) plate; foreground_contour and 3D are refused. Every tracker writes into the plate like ultrack after #413: the tracked labels labels/tracks_<labels> (value = track id; or labels/<output_label>) and tracks_<fov>.csv. The sibling filter (#415) applies to all of them. The tracked labels are the final segmentation and are preferred downstream over labels/<labels>; only ultrack changes the masks (it picks among its own segmentation hypotheses). Unlike ultrack they write no tracks.geff yet (#441), and they remove a stale one left by an earlier ultrack run.

labels: nuclei                   # the label image `biahub segment` wrote
tracker: trackastra              # ultrack (default) | tracksdata | hoct | trackastra
trackastra_config:
  model: general_2d              # pretrained name or a trained model folder
  mode: greedy                   # greedy | greedy_nodiv | ilp
  # image_channel: nuclei_prediction  # default: the channel segment recorded
# tracksdata_config: {distance_threshold_um, n_neighbors, distance_norm_um, node/appearance/disappearance/division weights, border_margin_um, ...}
# hoct_config: {model: general_v1, distance_threshold_um, n_neighbors, max_delta_t, solver}

• Distances in um, converted with the plate's pixel size. Defaults are the A549 benchmark's settings at 0.1133 um/px; tuning comes in a follow-up.
• HOCT and Trackastra read the image the labels were segmented from: the first channel biahub segment recorded in the label image's biahub-segment provenance, or hoct_config.image_channel / trackastra_config.image_channel when set (to override it, or for a label image segment did not write). --init checks that the channel exists. HOCT runs in pixel space (as its models were trained); HOCT 0.3 promotes 2D movies to (T, 1, Y, X) internally, and the conversion squeezes it back.
• GPU: when biahub track submits the jobs, hoct and trackastra get a gpu node; ultrack and tracksdata stay on the CPU preempted partition.
• Model weights (HOCT, Trackastra) are downloaded once by --init.
• One-child links: HOCT links across skipped frames, and converting its solution (or Trackastra's CTC table) can leave a parent with one child, which the sibling filter refuses. Such a child is merged into its parent, in the tracks and the labels.
• A new skill, track-with-biahub, runs and compares trackers on a segmented projected plate: one A549 config template per tracker (each with its own output_label, so runs sit side by side in the plate), a run script, and compare_tracks.py (counts, and division precision/recall against TrackEdit annotations).

Dependencies

tracksdata (git pin: no stable release yet), hoct>=0.3 and trackastra[ilp] join the existing track extra. The lock only adds packages, except gurobipy 13.0.3 -> 12.0.3 (HOCT requires <13); Gurobi licences work with older versions. Gurobi is optional: without a licence nothing fails. ultrack falls back to CBC; tracksdata, HOCT and Trackastra's ilp mode (via ilpy) use SCIP; Trackastra's default greedy mode needs no ILP solver. Biohub HPC has a Gurobi token-server licence, but each user must be added to it; then ~/gurobi.lic with TOKENSERVER=license01.czbiohub.org enables it for ultrack. SCIP can be slow on larger problems (cuOpt is an alternative, not wired in). ilpy does not: it takes LicenseID == 0 to mean gurobipy's size-limited pip licence, and a token-server licence also reports 0, so the ilpy trackers use SCIP even with it. That is the Solver failed with Gurobi, trying Scip warning in the runs below (not a GPU-node limitation).

Real data

The A549 track config of #415 (60-plane focus window, cellpose cpsam_v2, min_sibling_length: 3), changing only tracker:; untuned defaults for the new trackers.

2026_04_28 (4 TrackEdit FOVs, 78 divisions):

tracker	division F1	precision	recall	mock F1	DENV F1	tracks (GT 287)
ultrack	0.68	0.60	0.79	0.70	0.63	376
trackastra	0.63	0.71	0.57	0.65	0.60	355
hoct	0.53	0.68	0.43	0.59	0.47	311
tracksdata	0.52	0.47	0.57	0.48	0.55	331

2026_04_21 (5 DENV FOVs, 38 divisions from mitosis labels):

tracker	mitotic cells detected	divisions found	predicted divisions	tracks
ultrack	0.95	19	80	382
trackastra	0.95	14	45	419
hoct	0.96	7	21	334
tracksdata	0.94	17	76	344

ultrack stays the best default. Trackastra and HOCT are more conservative (higher precision, fewer divisions); tracksdata needs tuning. Each new tracker took ~13-16 min for 4-5 FOVs on one GPU node, cellpose included, ~11 GB peak.

Tests

Written failing first (21 new tests, counting parametrized cases): settings (tracker choice, 2D required, um defaults, unknown keys), each tracker on a synthetic movie with one division, um -> pixel conversion and the image passed to HOCT, an end-to-end biahub track --cluster debug run per tracker, --init warming the weights, one-child merging (gaps, chains, divisions kept), and the skill templates loading on the current schema. HOCT's and Trackastra's models are stubbed in CI; with their weights cached, real-model tests also run (both find the synthetic division). Full suite: 637 passed, 13 skipped; ruff clean.

Not in this PR

• GEFF export for the new trackers: track: write tracks.geff for every tracker, not only ultrack #441, a tracker-independent writer from the final tracks table so the sibling filter's edits are included.
• Tuning the new trackers' defaults on the annotated datasets.
• 3D tracking with tracksdata, HOCT or Trackastra.

Updated from main (merge 16fcd4e)

main gained #424 (biahub project), #413 (biahub segment writes the labels; track reads labels: and writes into the plate) and #430 (pyramids) after this PR was opened, so the trackers were reworked on top of them as described above. The settings, outputs, tests and the track-with-biahub skill all follow the new flow, and the real-data tables above predate the merge (in-track cellpose).

• Tests: full suite 735 passed, 13 skipped; ruff clean.
• Real data after the merge: v11 2026_08_11_A549_SEC61B_DENV_1_proj.zarr C/2/000000, using the same labels/nuclei from biahub segment for every tracker, with min_sibling_length: 3 and untuned defaults. Each ran in-process on one GPU node.

tracker	tracks	divisions	detections kept (of 1461)	mean track length	time
ultrack (the v11 pipeline run)	200	22	94.3%	6.9	—
tracksdata	226	54	90.7%	5.9	82 s
hoct	87	15	98.9%	16.6	86 s
trackastra	197	13	95.3%	7.1	23 s

For every tracker, each frame's label ids equal the CSV's track ids for that frame. HOCT and Trackastra read nuclei_prediction, which they resolved from segment's provenance.

[Copilot]

🟡 Changes recommended

Runtime compatibility, configuration validation, dependency installation, and comparison scoring have unresolved defects.

5 open issues

• Use natural sorting to select the latest annotation version · New
• Compute zero F1 for unmatched or missing predicted divisions · New
• Reject missing Cellpose configuration for Cellpose trackers · New
• Use a cross-platform fallback when CPU affinity is unavailable · New
• Install Cellpose and validate it for the track extra · New

What changed in this PR

Adds selectable tracking backends to biahub track, retaining ultrack as the default and preserving existing outputs and sibling filtering.

Changes:

• Adds tracksdata, HOCT, and Trackastra configuration and execution.
• Adds backend tests and dependency updates.
• Provides tracker-comparison guidance, templates, and scoring tools.

File	Description
uv.lock	Locks new tracker dependencies.
tests/​test_track_backends.py	Tests configuration, tracking, and lineage conversion.
tests/​test_example_settings.py	Checks tracker templates load.
settings/​example_track_settings.yml	Documents backend settings.
pyproject.toml	Adds tracker dependencies.
biahub/​track.py	Implements backend selection and output conversion.
biahub/​settings.py	Adds tracker schemas and validation.
.claude/​skills/​track-with-biahub/​templates/​track-ultrack.yml	Provides ultrack baseline configuration.
.claude/​skills/​track-with-biahub/​templates/​track-tracksdata.yml	Provides tracksdata configuration.
.claude/​skills/​track-with-biahub/​templates/​track-trackastra.yml	Provides Trackastra configuration.
.claude/​skills/​track-with-biahub/​templates/​track-hoct.yml	Provides HOCT configuration.
.claude/​skills/​track-with-biahub/​templates/​run_track.sh	Supplies GPU job template.
.claude/​skills/​track-with-biahub/​templates/​compare_tracks.py	Compares counts and division scores.
.claude/​skills/​track-with-biahub/​SKILL.md	Describes tracker-comparison workflow.
.claude/​skills/​track-with-biahub/​references/​trackers.md	Explains settings and benchmark results.
.claude/​skills/​track-with-biahub/​references/​evaluation.md	Documents evaluation and visual checks.

_{Review effort: Balanced}

[Copilot]

🟡 Changes recommended

Python 3.14 installation compatibility, tracking without candidate links, and empty-frame comparison statistics remain unresolved.

2 open issues

• Include empty frames when calculating mean detections · New
• Handle detection graphs with no candidate edges · New

5 resolved since last review

• Use a cross-platform fallback when CPU affinity is unavailable
• Reject missing Cellpose configuration for Cellpose trackers
• Compute zero F1 for unmatched or missing predicted divisions
• Use natural sorting to select the latest annotation version
• Install Cellpose and validate it for the track extra

_{Review effort: Balanced}

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[Copilot]

🟡 Changes recommended

Empty tracking results and comparisons containing unannotated runs can still abort instead of producing valid outputs.

2 open issues

• Index F1 by run to preserve unscored runs · New
• Fix COMMANDS example to use a dictionary · New

2 previously missed issues

In code that hasn't changed since last review

Handle empty detections and empty solver results

biahub/​track.py:551

Handle both empty detections and empty solver results. With an all-background movie, the pinned RegionPropsNodes never registers y/x, so the border-coordinate lookup in _solve_tracksdata fails before the no-edge guard. Even with detections and candidate edges, the solver can select nothing: under the defaults, a stationary interior cell in two frames costs 5, versus 0 for an empty solution. This exporter then raises ValueError("Graph is empty"). Return zero labels and an empty tracks table with the normal typed columns for both cases, and bypass sibling filtering for empty tracks so those results can reach the CSV and label writers. Add regression coverage with the sibling filter enabled.

Correct reversed division_weight tuning guidance

.claude/​skills/​track-with-biahub/​templates/​track-tracksdata.yml:47

This tuning instruction is reversed: tracksdata minimizes the objective, and division_weight is a division cost. Increasing it discourages divisions rather than allowing more, as references/trackers.md:23 correctly states. Correct the comment so users tuning missed divisions do not move the parameter in the wrong direction.

_{Review effort: Balanced}

[tayllatheodoro]

Re the two previously-missed findings: empty detections and empty solutions now give zero labels and an empty tracks table (the sibling filter is skipped for empty tracks), with tests for an empty movie, a solution that selects nothing, and an end-to-end run per tracker with the filter on. The tracksdata template's division_weight comment is corrected (a cost: lower it to allow more divisions).

[Copilot]

🟡 Changes recommended

HOCT can still abort on temporally separated sparse links because the fallback checks total graph edges rather than individual inference windows.

1 open issue

• Handle sparse inference windows before concatenating predictions · New

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

The three new backend integrations and solver compatibility changes warrant final human validation of real-model execution beyond the primarily stubbed CI coverage.

No open issues 🎉

1 resolved since last review

• Handle sparse inference windows before concatenating predictions

_{Review effort: Balanced}

[tayllatheodoro]

Real-model validation, since CI stubs the HOCT and Trackastra models:

• Local run with the weights cached (HOCT general_v1, Trackastra general_2d), at 43ea1d8: the 5 real-model tests pass (Trackastra and HOCT each find a synthetic division; HOCT's solver path with an override; a single candidate link; temporally separated links).
• Real data, end to end through biahub track --cluster debug on GPU nodes (SCIP, as Gurobi is not licensed there): all three trackers on the 4 TrackEdit FOVs of 2026_04_28 and 5 FOVs of 2026_04_21; scores in the description (e.g. 04_28 division F1: ultrack 0.68, Trackastra 0.63, HOCT 0.53, tracksdata 0.52). The HOCT run is what surfaced the one-child-link fix (77fc383).

[Copilot]

🔵 Needs a closer look

Despite the reported tests and benchmarks, the model, solver, and lineage-conversion integrations warrant final human review.

No open issues 🎉

_{Review effort: Balanced}

[ieivanov]

@tayllatheodoro heads-up: I merged main into this branch (16fcd4e, a merge commit, no force-push). #424, #413 and #430 landed after this PR opened, and they changed what the trackers build on, so this is more than a conflict fix:

• Input: tracksdata, hoct and trackastra now link the label image biahub segment writes (labels: <model>) instead of running cellpose in track. segmentation_method: cellpose, cellpose_config and z_slicing are refused in track on main.
• Image for hoct and trackastra: the first channel segment recorded for that label image, or a new image_channel in their configs, which overrides it or covers a label image segment didn't write.
• Outputs: they write into the plate like ultrack (labels/tracks_<labels> + the CSV) but no tracks.geff yet. That's track: write tracks.geff for every tracker, not only ultrack #441.
• Tests and skill: both are ported to the new flow. The full suite passes, and a real-data run on one v11 FOV is in the updated description.

I edited the description to match. Please check the API changes, and feel free to rework anything that doesn't fit what you had in mind.

[ieivanov]

@tayllatheodoro how is gurobi licensing handled? Would Sookie or Allyson be able to run the pipeline (in general) without a gurobi license? Would they be able to run tracking without a license? Is it just slower or does it fail? I don't think I have a gurobi license and for me tracking runs within a minute - is gurobi absolutely necessary or can it be an optional add-on?

[tayllatheodoro]

@ieivanov Gurobi is optional: tracking never fails without a licence, it falls back to an open-source solver. Biohub users can get the Gurobi licence through the Biohub licence server: a ~/gurobi.lic containing TOKENSERVER=license01.czbiohub.org.

I checked with and without a licence (licence file pointed to a nonexistent path, on the same ~5000-variable test problem):

	licence (TOKENSERVER=license01.czbiohub.org)	no licence
ultrack (python-mip)	Gurobi	warns "Gurobi environment could not be loaded", then uses CBC and solves
tracksdata, hoct, trackastra ilp (ilpy)	SCIP (see below)	SCIP, solves
trackastra greedy (default)	no ILP solver	no ILP solver

So Sookie or Allyson can run the pipeline, tracking included, without a licence. The difference is speed (and solution quality within the time limit) on larger problems, which is likely why your v11 FOV ran within a minute without one. Even with the token server, ilpy reported "Gurobi license is not available" and used SCIP. That is what the "Solver failed with Gurobi, trying Scip" warnings in the tracksdata/HOCT runs were, not a GPU-node issue as the description says; I'll correct it.

@JoOkuma could you add context on how much Gurobi vs CBC/SCIP matters in practice for ultrack and the ILP trackers (speed, and solution quality within the time limit on large movies)? And is ilpy expected to pick up a token-server licence?

[tayllatheodoro]

@ieivanov thanks for porting this onto #424/#413. I went through the merge, and it looks right to me:

• Input and image: HOCT and Trackastra resolve the image channel from segment's provenance (or image_channel). They see the same raw projected image as before the merge and normalize it themselves (HOCT normalize_image, Trackastra normalize_imgs=True), so the benchmark numbers in the description should carry over.
• Frames and channels: blank frames are filled in the image as in the labels, so they stay frame-aligned. A missing channel is caught at --init.
• Settings checks: a config with no labels, foreground_contour or 3D is refused for these trackers, and old cellpose_config / z_slicing configs get a clear error.
• Outputs and jobs:
  • A stale tracks.geff is removed.
  • The full settings, tracker included, are recorded on the tracked label image by --init.
  • The hoct/trackastra jobs go to gpu.
• Tests: the tracker tests pass locally (129: backends, skill, example settings, track CLI).

Small things:

1. Gurobi wording (fixed in e65d803, docs and comments only): the skill blamed the GPU nodes for the SCIP fallback. The real cause is ilpy's licence check (LicenseID == 0, which token-server licences also report). The PR description now says the same.
2. Image channel checked on one position: --init checks the channel on the first position only, so a plate whose other positions lack it would fail inside the jobs. Minor, fine to leave.

A question on segmentation: ultrack does not keep the segment labels as they are. It builds a hierarchy from their contours and picks the segments that track best, which corrects some cases (e.g. merging an over-split nucleus, or splitting two touching ones). tracksdata, HOCT and Trackastra link the cellpose labels unchanged. So:

• the tracked label image is a different segmentation depending on the tracker;
• comparing trackers mixes segmentation correction with linking (on your v11 FOV, the share of the 1461 detections each tracker kept ranged from 90.7% to 98.9%, ultrack 94.3%);
• downstream steps that read labels/tracks_<labels> get ultrack-corrected or uncorrected masks depending on the config.

How would you like to handle this? Options I see:

• (a) treat the tracked labels as the final segmentation, and document that only ultrack corrects it;
• (b) let the label-linking trackers optionally take ultrack's corrected labels as input;
• (c) keep the segment labels authoritative, and record per tracker whether the tracked labels can differ from them.

[JoOkuma]

@JoOkuma could you add context on how much Gurobi vs CBC/SCIP matters in practice for ultrack and the ILP trackers (speed, and solution quality within the time limit on large movies)?

Gurobi is faster and uses less memory than CBC or SCIP.
CBC is only used in ultrack as a fallback.
SCIP is the new fallback because it's easier to share, but it's really bad and can easily take too long to solve moderately large problems, which might or might not be your case, so you need to test.
The OPS project has been using cuOPT backend to avoid using gurobi, see royerlab/hoct#9, it might be a good alternative.

All ILPs, if solved to the same GAP, should yield similar results, though some might take much longer to achieve this.

And is ilpy expected to pick up a token-server licence?

HPC has a token server license where each user needs to be added manually.

[ieivanov]

Thanks Taylla! I think we should "treat the tracked labels as the final segmentation, and document that only ultrack corrects it". Segmentation and tracking a coupled problems. Within biahub all trackers output a labels layer, so any tracker could have the ability to edit segmentation masks. In the presence of tracking, labels/tracks_<labels> should be preferred over labels/<labels>.

[ieivanov]

@tayllatheodoro when you're happy with this PR, please take it out of draft and I'll approve it

[tayllatheodoro]

@ieivanov thanks, it's ready for review now. ultrack stays the default tracker (in biahub and in the A549 track.yml); the other three are opt-in with untuned defaults.

Plan after this merges

1. Choose the A549 default tracker and its parameters from the tuning we're running on the annotated data, and set them in a follow-up PR (biahub defaults + nextflow/configs/a549/track.yml). The tuning so far, with settings chosen on 04_28 + 07_24 and checked on held-out 08_26 and the mitosis-labelled FOVs:
   • HOCT and Trackastra need the input downsampled, since they were trained on much smaller nuclei. HOCT general_v1 shrunk 6x: held-out division F1 0.58 vs ultrack's 0.55, and mitosis recall 0.72 vs 0.64.
   • Fine-tuning: Trackastra fine-tuned on our TrackEdit data (4x): held-out F1 0.57, mitosis recall 0.71. Fine-tuning HOCT's edge head didn't help.
   • tracksdata overfits: its best tuning settings drop to 0.46 on held-out data.
   • Swaps are rare for every tracker on these FOVs (0-2 per tracker).
   • Caveat: the held-out set is small (34 divisions). Each tracker is now also being tested with the division classifier on top.
2. Keep testing the default on the current annotations and on new ones as they come in.
3. Then extend the track-with-biahub skill so users can run and evaluate tracking on new kinds of data (other cell lines, markers, magnifications) and set up their own pipeline.

Your guidance on decisions

• What should decide the default? Division F1 on held-out data, fragmentation and ID switches, or also cost: HOCT and Trackastra need a GPU and model weights, while ultrack runs on CPU. And what margin over ultrack would justify switching, given how small the held-out set is?
• Downsampling HOCT/Trackastra: OK to add as a downsample setting in their configs, with tracking done at the coarser scale and the result mapped back to the full-resolution labels?
• Three-way divisions happen in our cells, but ultrack's sibling filter only accepts 0 or 2 children; HOCT produced one and crashed the filter. Options: the filter skips tracks with 3+ children, or it treats each daughter separately. Do you have a preference?
• The division classifier (a random forest that removes false divisions; on A549 division F1 goes 0.68 -> 0.82 on 04_28, 0.47 -> 0.68 and 0.41 -> 0.73 on the two v1 datasets) is trained on only 4 annotated FOVs, so it comes in a later PR, after more annotation. For when we get there: would you put it inside biahub track or in a separate command? And should biahub project write the phase-texture image it uses (Z std of Phase3D over the focus window)?

[tayllatheodoro]

@ieivanov one more question on scope: should we merge this PR now (ultrack as default, the other trackers opt-in) and do the plan above in follow-up PRs (tuned defaults + downsample + A549 track.yml, then the classifier after more annotation)? Or would you rather keep everything in this PR until the default is settled? We're happy either way; merging now keeps each PR smaller to review.

[tayllatheodoro]

@ieivanov could you approve and merge this when you have a moment? ultrack stays the default.

The tuning results so far don't justify changing the default yet. On data no method was tuned on (08_26, 34 divisions; 04_21 mitosis labels), HOCT general_v1 shrunk 6x and a fine-tuned Trackastra are slightly ahead of ultrack, but within noise:

• 08_26 division F1: ultrack 0.55, HOCT 0.58 (recall 0.88), Trackastra 0.57.
• 04_21 divisions found: 21 / 25 / 21 of 38.

We'll recheck them on new annotations and bring any default change, with the downsample setting, in a follow-up PR.

[ieivanov]

Thanks @tayllatheodoro! Let's merge this now and discuss follow-up on an issue.

[tayllatheodoro]

Thanks @ieivanov! Follow-up discussion is in #444 (default tracker, downsample for HOCT/Trackastra, three-way divisions, division classifier). cpdino as a segmentation option is in #445.