Skip to content

feat: read OME-Zarr 0.5 by its metadata (plate/well/image, dataset path, axes by type); fail loud - #42

Merged
hongquanli merged 11 commits into
mainfrom
feat/ome-zarr-05-reader
Nov 7, 2026
Merged

hongquanli merged 11 commits into
mainfrom
feat/ome-zarr-05-reader

Conversation

@hongquanli

Copy link
Copy Markdown
Collaborator

Summary

ndviewer_light now reads OME-Zarr 0.5 by its metadata instead of by Squid's folder names, fails loudly instead of showing zeros, and labels incomplete arrays.

Reading by the spec (new Qt-free module ndviewer_light/ome_zarr.py):

  • An image group is read from attributes.ome.multiscales: axes mapped by type (time/channel/space; spatial axes by name when they are x/y/z, otherwise by declared order), the array at datasets[0].path (level 0, whatever the path is called — 0, s0, scale0/image), the scale composed from the dataset-level and optional multiscale-level transforms, length units converted to µm (angstrom … kilometer; anything else is refused by name), omero optional (Ch0… otherwise), _squid optional (spec values win, it only fills gaps), ome.version other than 0.5 warns and proceeds.
  • Discovery walks plate.wells → well.images → image groups, labelling regions by row + column name and fields by their numeric path. A plate, a well, an image group, or a Squid experiment folder (plate.ome.zarr, zarr/<region>/fov_<n>.ome.zarr, zarr/<region>/acquisition.zarr) can be opened. A listed field whose directory does not exist is skipped with one warning (Squid lists planned fields; an aborted run never writes some); a directory that exists but cannot be parsed fails the load with its path. Pre-0.5 Squid layouts still go through the old directory walk.

Loaders (core.py): one _arrange_axes for both the per-FOV and the 6D paths; a FOV that cannot be opened raises (the da.zeros placeholder is gone, and so is the catch-all that turned every error into "Failed to load dataset"); the status label says Loaded: <name> or Loaded: <name> (incomplete acquisition) when an array's _squid block says its planned planes were not all written; 6D regions are concatenated along the FOV axis with region:i labels; FOV labels are set in pull mode (so get_fov_list() works for Zarr) without affecting push-mode detection (is_push_mode_active() now has its own flag); refresh() reports loader errors in the status label instead of raising; labels are replaced only when a load succeeds.

Unchanged: the push API (start_acquisition, start_zarr_acquisition*, notify_zarr_frame, end_acquisition, go_to_well_fov), the exported names Squid imports (detect_format, discover_zarr_v3_fovs, parse_zarr_v3_metadata, open_zarr_tensorstore, FPATTERN_OME), OME-TIFF and single-TIFF reading. No resolution pyramids (level 0 only), no Zarr v2 / NGFF 0.4.

Tests

  • tests/test_ome_zarr_05.py: metadata parsing and discovery on real tensorstore arrays written by tests/ome_zarr_fixtures.py (dataset path, axes subsets and arbitrary spatial names, composed transforms, units, no omero, no _squid, unknown version, broken metadata, absent vs broken fields, numeric field ids, recognised-but-empty plates, pre-0.5 layouts).
  • tests/test_ome_zarr_05_viewer.py: the headless viewer loading plates, image groups, 6D stores (one and several regions, shape mismatch), the legacy layout, broken FOVs, the incomplete label, dimension_names mismatch, pull vs push mode, failed refresh, Zarr-then-TIFF.
  • tests/fixtures/external_05.ome.zarr: a 20 KB image written by ngff-zarr 0.50.0 (dataset path scale0/image), loaded with the right dims, scale and pixels — the non-Squid case.
  • Full suite: 277 passed (QT_QPA_PLATFORM=offscreen python -m pytest tests/), black clean.

Two deliberate choices worth knowing: a well group opened directly is labelled row + column ("B3", from its parent and own directory names) rather than the bare directory name, so labels match the plate case and go_to_well_fov's well-id format; and the loader is exception-safe through load_dataset(), refresh() and the _maybe_refresh slot (the live-refresh timer itself is never started today).

Follow-ups noted during review, none blocking: a corrupt plate zarr.json is treated as unrecognised (falls to the legacy walk silently); field ids use str.isdigit() (Unicode digits, "01" vs "1"); omero channel-count mismatch replaces names without a warning; a non-string dataset path fails late; the external-fixture generator's header lacks --no-project.

Also verified: Squid's own OME-Zarr 0.5 output (tests/ngff_readers/write_samples.py: plate, per-FOV, sharded + tiled) loads headlessly with correct dims, channel names, pixel sizes and pixel values.

🤖 Generated with Claude Code

hongquanli and others added 10 commits November 4, 2026 11:53
…taset path, composed scale)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…eld ids, absent vs broken

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ZarrError

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…incomplete arrays; FOV labels in pull mode

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…uccessful load; no incomplete label on legacy stores; refresh() reports loader errors

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ell_fov only in push mode; labels replaced only with accepted data

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…rrors name the image; docs: zarr viewer architecture

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… stores without axes infer dims from rank; refresh updates the incomplete label

Review of PR #42: a corrupt acquisition.zarr or plate zarr.json inside a recognised layout was
swallowed by discovery and the region silently dropped; pre-0.5 stores without axes metadata were
forced to five dims (3D/4D/6D stores that main loaded now failed); a refresh updated the incomplete
flag but not the status label.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@hongquanli

Copy link
Copy Markdown
Collaborator Author

Addressed the three review findings in 4f751c2: an unreadable region or plate metadata file inside a recognised layout now raises an error naming it (no silent drop); pre-0.5 stores without axes metadata infer dims from the array rank again (2D–6D); a refresh updates the status label with the incomplete flag. 8 tests added; suite 285 passed.

…gion store without metadata is an error

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@hongquanli

Copy link
Copy Markdown
Collaborator Author

Both follow-up findings fixed in d08510f: axes-less legacy 6D stores infer (t, fov, c, z, y, x) as main did (test expectation corrected), and an existing acquisition.zarr directory without zarr.json now raises an error naming it instead of being skipped. Suite 286 passed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant