Skip to content

Port to ndv 0.5: public APIs for channel names, navigation, in-place swap and 3D scale - #44

Merged
hongquanli merged 13 commits into
mainfrom
feat/ndv-0-5
Nov 8, 2026
Merged

hongquanli merged 13 commits into
mainfrom
feat/ndv-0-5

Conversation

@hongquanli

@hongquanli hongquanli commented Nov 8, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Moves ndviewer_light to ndv>=0.5,<0.6 and talks to ndv through its public API wherever one exists, so the next ndv release is a pin bump, not a port. The Squid-facing surface (LightweightViewer, the push APIs, go_to_well_fov, set_current_index, get_fov_list, has_fov_dimension) is unchanged.

ndv 0.5.0 is what an unpinned install gets today. On it, with the <0.5 pin lifted and nothing else changed, the plate loads and renders but:

  • LUT rows read "0" / "1" instead of the channel names;
  • set_current_index is a silent no-op (Squid's plate double-click stops navigating);
  • every push-mode frame rebuilds the ArrayViewer (flicker, zoom lost);
  • the 3D voxel-scale patch is inert;
  • any test that constructs a viewer under QT_QPA_PLATFORM=offscreen segfaults (rc 139).

Verified causes (each checked by running against ndv 0.5.0)

Symptom Cause on ndv 0.5 Fix
Channel names lost ndv labels a LUT row from the public LUTModel.name (ndv #238) and falls back to the channel coordinate; our coords are range(n) and the old _lut_controllers retry loop wrote a field 0.5 no longer reads build_luts() -> luts={i: {"cmap", "name"}} at all six LUT sites; retry machinery deleted
Navigation no-op display_model.current_index stays empty until a slider moves (ndv #233 made index resolution pure), so the dim in current_index guard always failed validate against data_wrapper.dims / sizes(), write the key unconditionally; unknown dim and out-of-range return False with the model untouched
Rebuild per push frame viewer._data_model and _request_data() are gone public data_wrapper + its documented dims_changed / data_changed signals; dims_changed only on a shape change, then always data_changed (a shape change at an unchanged index does not refetch otherwise)
3D scale inert the vispy VolumeVisual monkeypatch never took effect (frozen visual, swallowed AttributeError) and the private VispyArrayCanvas.add_volume camera patch is reset by ndv's own set_range scales={z dim: dz / pixel_size} to ArrayViewer (public, ndv #238); patches and the private import deleted
Oversized 3D textures Downsampling3DXarrayWrapper never queried the GPU (tested an attribute on the wrong object, always the 2048 fallback) and distorted the extent; ndv 0.5 downsamples itself with the scales compensated wrapper and MAX_3D_TEXTURE_SIZE deleted (nothing in Squid reads them). ndv's strided result is a view that vispy keeps alive, so _ndv_compat.install_compact_downsample() makes it own its memory (review finding, reproduced: 256 KiB retained for an 8 KiB texture; now 8 KiB, as on main)
Offscreen segfault ndv 0.5 queries GL_MAX_TEXTURE_SIZE on the first add_image; no GL context under the offscreen platform tests/conftest.py pins the limits to (4096, 2048); NDVL_REAL_GL=1 keeps the real query

Changes

  • build: ndv>=0.5,<0.6 in pyproject.toml and setup.cfg (kept in sync); superqt[iconify]>=0.8.1 (ndv 0.5's Qt view imports QIconifyIcon; the floor is the one ndv's own Qt extra declares); pyopengl declared (3D textures; ndv[vispy] is deliberately not used because it demands vispy>=0.16 while napari pins <0.15); environment.yml installs ndv via pip with the same pin; scipy dropped from all three lists (its only user was the deleted wrapper); stale "Squid runs under PyQt6" comments fixed (its default is PyQt5).
  • core.py: build_luts; set_current_index rewrite; installs _ndv_compat.install_compact_downsample() (new module ndviewer_light/_ndv_compat.py, the one private ndv shim, with its removal condition); _try_inplace_ndv_update on data_wrapper; _set_ndv_data passes scales; deleted the retry machinery, the dims fallback, the vispy patch block, the private ndv.views._vispy import, Downsampling3DXarrayWrapper and the scipy import only it used. Net -589 lines in core.py.
  • Tests: test_channel_names.py (public-model assertions, no _lut_controllers in core.py), new test_set_current_index.py, test_inplace_update.py rewritten to the data_wrapper surface, test_memory_leak_reproduction.py counts data_changed, new test_3d_scales.py, test_downsampling_wrapper.py deleted, new test_ndv_compat.py (the displayed volume owns its memory under a forced 16-voxel limit), new test_window_smokes.py (S1-S3, @pytest.mark.window) with tests/_ndv_private.py holding every private read used by tests; there, "ndv has not created this object yet" (poll on) and "ndv renamed the path" (skip) are distinct outcomes, pinned by test_ndv_private_helper.py.
  • CI: two runs of the suite, offscreen with the pinned GL limits and xcb under Xvfb + Mesa with NDVL_REAL_GL=1 (the window smokes execute there); the real-GL step reads its JUnit report and fails if a smoke was skipped or not collected, so a GL or Xvfb problem cannot pass as skips.
  • Docs: README.md, docs/DESIGN.md (new "ndv Integration" section with the remaining private points), docs/zarr_viewer_architecture.md, tests/README.md.

Probes (numbers)

Public viewer.data = new vs the in-place swap (ndv 0.5.0, vispy 0.17, PyQt5, cocoa, real LightweightViewer in push mode, 200 frames of (3, 2, 256, 256) uint16, z_level=2 shown, camera zoomed in):

wrapper._data + signals viewer.data = new
texture updated every frame yes yes
latency per frame (mean / max) 4.1 / 46 ms 84.8 / 136 ms
add_image calls over 199 frames 0 398
vispy GL objects created 0 3353
frames with no image handle after apply 0 200
camera rect preserved yes no (reset each frame)
live handle / texture / wrapper delta after gc 0 / 0 / 0 0 / -1 / 0
tracemalloc delta +1.8 MB +17.8 MB
RSS delta +12 MB +43 MB

Neither path leaks handles (ndv #253 holds on 0.5), but the public setter clears the canvas and re-adds every image on the next response: a rebuild in all but name, so the one private write wrapper._data = data stays (pinned by test_data_wrapper_private_data_slot and smoke S1).

3D through scales= (cocoa, NDVL_REAL_GL=1, (3, 2100, 2100) uint16 volume, dz 1.5 um / px 0.5 um; identical on vispy 0.17.0 and 0.14.3): resolved visible scales (3, 1, 1), STTransform.scale (2, 2, 3) over a (3, 1050, 1050) texture (ndv's strided downsample), camera x/y range (-21, 2121), z range (-0.09, 9.09), 0 draw errors.

Headless: without the conftest pin the ndv 0.5 suite segfaults (rc 139) at test_ome_zarr_05_viewer.py::test_plate_loads_with_labels_and_pixels, the first test that constructs a viewer; with it the full suite passes offscreen.

Test runs on this branch

  • ndv 0.5.0 / vispy 0.17.0 / PyQt5 5.15.14 / Python 3.11, offscreen: 292 passed, 3 skipped (the window smokes skip offscreen)
  • ndv 0.5.0 / PyQt6 6.12.0, offscreen: 292 passed, 3 skipped
  • ndv 0.5.0 / PyQt5, QT_QPA_PLATFORM=cocoa NDVL_REAL_GL=1: 295 passed (S1-S3 ran)
  • black --check .: clean

The real-GL run under Xvfb + Mesa on ubuntu-latest is new in this PR's CI and is verified by this PR's first CI run. The apt set (libxcb-* incl. libxcb-cursor0, which Qt >= 6.5's xcb plugin requires, plus Mesa GL) is a subset of what ndv's CI installs through pyvista/setup-headless-display-action; a step before the real-GL run loads the xcb plugin and creates a GL context on its own so a plugin or GL failure is reported separately from the tests.

Private points that remain (each pinned by a test)

  • DataWrapper._data write in _try_inplace_ndv_update (no public setter; see the probe).
  • display_model.current_index[dim] = value (public model, no setter method on ArrayViewer; ndv's keybindings write the same mapping).
  • ndv.views._vispy._array_canvas.downsample_data wrapped by _ndv_compat.install_compact_downsample(): ndv 0.5.0 strides oversized textures to a view and vispy keeps it as _last_data, so the full (z, y, x) block of each displayed channel stayed in RAM in 3D (900 MB for a 50 x 3000 x 3000 uint16 stack where 225 MB is displayed). The shim copies the reduced volume into contiguous memory; core.py imports only the shim module. Pinned by tests/test_ndv_compat.py; removed once ndv copies itself.
  • superqt QLabeledSlider.setRange patch (superqt-private, not ndv; untouched here).
  • Test-only: tests/_ndv_private.py (_lut_controllers[k].handles[0].data(), _view._qwidget.dims_sliders._sliders[dim], _is_idle()), tests/conftest.py (ndv.views._vispy._array_canvas.get_max_texture_sizes). core.py imports neither.

For consumers

  • ndv 0.5 depends on ihist, which requires numpy >= 2; psygnal>=0.12; superqt[iconify] (pyconify).
  • vispy stays unpinned: 2D and the 3D check above pass on vispy 0.14.3 (napari's range) and 0.17.
  • Qt through qtpy; PyQt5 and PyQt6 both exercised.

Squid follow-up (separate PR)

Submodule bump to this PR's merge commit; installer and open_in_viewer.py specs move to "ndv>=0.5,<0.6" plus pyconify (replacing the interim ndv<0.5 pin from Squid #797); simulation check of the NDViewer tab (labels, plate double-click navigation, live push without rebuild, 3D). It is blocked on one decision: Squid's numpy<2 pin (installer and CI, from #501) has to be lifted for ndv 0.5's ihist, which needs re-testing on Ubuntu.

Upstream (to file with ndv)

  • get_max_texture_sizes() should return (None, None) when no GL context is current instead of segfaulting (maxd is None already means "skip downsampling").
  • downsample_data should return a contiguous copy when it strides: vispy retains the array it is given (VolumeVisual._last_data), so a strided view keeps the whole source block alive.
  • A public setter for DataWrapper's array and an ArrayViewer.set_current_index would remove the last two private points here.

🤖 Generated with Claude Code

hongquanli and others added 13 commits November 7, 2026 17:22
pyproject.toml / setup.cfg: ndv>=0.5,<0.6; superqt[iconify] (ndv 0.5's Qt view
imports QIconifyIcon, which needs pyconify); pyopengl (3D volume textures;
declared here because ndv[vispy] is not used - that extra demands vispy>=0.16
while napari pins vispy<0.15). setup.cfg's install_requires synced with
pyproject (qtpy instead of pyqt5). Fix the stale comments claiming Squid runs
the viewer under PyQt6: Squid's default binding is PyQt5.

environment.yml: ndv moved from conda-forge to the pip list with the same pin.

tests/conftest.py: ndv 0.5 queries GL_MAX_TEXTURE_SIZE on the first
add_image/add_volume (ndv.views._vispy._array_canvas._downcast_and_downsample
-> ndv.views._vispy._util.get_max_texture_sizes). Under QT_QPA_PLATFORM=offscreen
there is no GL context and the raw glGetIntegerv segfaults (rc 139 at
tests/test_ome_zarr_05_viewer.py::test_plate_loads_with_labels_and_pixels, the
first test that constructs a viewer). Patch get_max_texture_sizes in _util and
its import-time binding in _array_canvas to (4096, 2048) before any viewer
exists; NDVL_REAL_GL=1 keeps the real query for xvfb/cocoa runs. Drop the
pre-qtpy PYTEST_QT_API=pyqt5 default (CI is PyQt6-only; qapp is a local fixture).

Verified on ndv 0.5.0 / vispy 0.17.0 / PyQt5 5.15.11: offscreen 286 passed
(rc 139 without the patch); NDVL_REAL_GL=1 under cocoa 286 passed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…retry loop

ndv 0.5 labels each LUT row from the public LUTModel.name field (ndv PR #238)
and falls back to the channel coordinate when it is empty. ndviewer_light keeps
integer channel coordinates and set neither, so on 0.5 the rows read "0"/"1":
the retry loop found _lut_controllers, wrote controller.key, logged success,
and changed nothing (0.5's synchronize() no longer pushes the key as the name).

core.py: new module-level build_luts(channel_names, channel_colors=None) ->
{position: {"cmap", "name"}} used at all six LUT-building sites (push TIFF
start, the two zarr-push builders, OME-TIFF, single-TIFF, OME-Zarr). cmap is
always set because ndv applies its default colour cycle to any LUT whose cmap
was left unset. The dict still travels in attrs["luts"] into
ndv.ArrayViewer(luts=...), so names are in the model at construction and no
readiness wait exists. Deleted: CHANNEL_LABEL_UPDATE_MAX_RETRIES /
_RETRY_DELAY_MS, _channel_label_generation / _pending_channel_label_retries,
_initiate/_schedule_channel_label_update, _update_channel_labels, their two
call sites (in-place branch and end of _set_ndv_data), and the comments that
described the hack. The in-place branch needs no rename: data_structure_changed
already forces a rebuild when channel_names or luts differ.

tests/test_channel_names.py: TestChannelLabelRetryMechanism and
TestChannelLabelUpdate (simulations of the deleted private path) removed;
TestChannelNamesReachNdv added: build_luts pairing, push-start and OME-Zarr
plate names in display_model.luts[i].name right after construction, LUT
checkbox text (polls processEvents up to 3 s), and no "_lut_controllers" in
core.py. tests/README.md updated accordingly.

Verified on ndv 0.5.0 offscreen (conftest GL patch): 281 passed under PyQt5
and under PyQt6. RED before the change: 5 failed (names "" / [] on 0.5).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… index unconditionally

On ndv 0.5 display_model.current_index stays empty until a user moves a
slider (ndv PR #233 made index resolution pure), so the old
`dim in current_index` guard turned set_current_index into a silent
no-op: Squid's plate double-click (go_to_fov) never navigated.

Validate dim against the public data_wrapper.dims and value against
data_wrapper.sizes(), then write the key. Unknown dims are rejected
(writing one makes ndv warn on every later write) and out-of-range
values are rejected with the model left untouched; both return False.
Drop the dims.current_step fallback and the hasattr guards.

Tests: tests/test_set_current_index.py pins the empty-model fact and the
three contract cases (model-level, offscreen).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…signals

_try_inplace_ndv_update read viewer._data_model.data_wrapper and called
viewer._request_data(); neither exists on ndv 0.5, so every push-mode frame
and every same-structure refresh fell back to _set_ndv_data: a new
ArrayViewer plus widget swap per frame (flicker, and the GPU-handle leak the
in-place path exists to avoid).

Use the public ArrayViewer.data_wrapper and the wrapper's documented
dims_changed / data_changed signals. The wrapped array has no public setter,
so wrapper._data stays the one private write. Emit dims_changed only when
the shape changed (sliders re-created, view re-resolved) and then always
data_changed: on 0.5 dims_changed alone re-fetches only when the resolved
index changed, so a shape change at an unchanged index would keep the old
texture. The AttributeError-as-expected branch is gone: with public APIs any
failure is logged at WARNING before the rebuild fallback.

Tests: tests/test_inplace_update.py now mocks the data_wrapper surface
(data property, _data, the two signals on one parent mock) and pins the
same-shape (data_changed only) and shape-change (dims_changed then
data_changed) sequences, the no-viewer / no-wrapper / wrapper-None cases and
the emit-exception fallbacks; test_push_register_image_keeps_viewer_instance
drives two push frames through a real viewer offscreen and asserts the
ArrayViewer instance survives with its data being the viewer's current array
(fails on the old code: the instance is replaced).
tests/test_memory_leak_reproduction.py: MockArrayViewer exposes data_wrapper
and counts data_changed emits; the new-code path must emit once per swap.

Verified on ndv 0.5.0 / PyQt5: offscreen 286 passed; the in-place file also
passes under cocoa with NDVL_REAL_GL=1.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…and _ndv_private

Leak probe (ndv 0.5.0, vispy 0.17, PyQt5, cocoa, real LightweightViewer in push
mode, 200 frames of (3, 2, 256, 256) uint16, z_level=2 shown, camera zoomed in):

                                  wrapper._data + signals   viewer.data = new
  texture updated every frame                 yes                  yes
  latency per frame (mean / max)       4.1 / 46 ms         84.8 / 136 ms
  add_image calls over 199 frames                0                  398
  vispy GL objects created                       0                 3353
  frames with no image handle after apply        0                  200
  camera rect preserved                        yes      no (reset each frame)
  live VispyImageHandle / Texture / DataWrapper
    delta after gc                          0 / 0 / 0           0 / -1 / 0
  tracemalloc delta                         +1.8 MB             +17.8 MB
  RSS delta                                  +12 MB               +43 MB
  LUT names, z index kept                      yes                  yes

Neither path leaks handles (ndv#253 holds), but the public setter runs
_fully_synchronize_view -> _apply_changes(EMPTY_STATE, ...) -> _clear_canvas and
re-adds the images on the next response (add_image -> set_range): an empty canvas
between frames, the zoom lost, 20x the latency. It is a rebuild in all but name,
so the private write stays. test_inplace_update.py records the verdict and pins
the slot with test_data_wrapper_private_data_slot.

Window smokes (tests/test_window_smokes.py, @pytest.mark.window, skipped when the
live Qt platform is offscreen via the conftest window_qapp fixture; the marker is
registered in pyproject): S1 push frame N+1 reaches its channel's texture with the
ArrayViewer instance kept; S2 a shape change through _try_inplace_ndv_update
updates the z slider range and the texture (fails when only dims_changed is
emitted, checked); S3 set_current_index("z_level", 2) shows plane 2. They poll
processEvents() and ndv's idle state, never sleep. tests/_ndv_private.py isolates
the private reads (displayed_texture, dims_slider, is_idle) and skips on rename.

Runs: ndv 0.5 offscreen PyQt5 287 passed / 3 skipped; PyQt6 287 passed / 3
skipped; cocoa NDVL_REAL_GL=1 window smokes 3 passed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…patches and the wrapper

ndv 0.5 has a public per-axis display scale (ArrayViewer(..., scales={dim: f}),
ndv PR #238): _set_ndv_data now passes {z dim: dz_um / pixel_size_um} for the
"z" / "z_level" dim present, and nothing when either size is missing. ndv
applies it as an STTransform on the volume and frames the camera to match.

Deleted, all inert or redundant on ndv 0.5:
- the VolumeVisual __init__ / _create_vertex_data monkeypatch (never took
  effect: the visual is frozen, the AttributeError was swallowed),
- the private VispyArrayCanvas.add_volume camera patch (reset by ndv's own
  set_scales -> set_range right after it), with the ndv.views._vispy import,
- the _current_voxel_scale module global,
- Downsampling3DXarrayWrapper and MAX_3D_TEXTURE_SIZE (nothing in Squid reads
  them): ndv 0.5 downsamples oversized textures itself with a strided view and
  compensates the scales, so a 2100-px volume keeps its 2100-unit extent
  instead of the wrapper's 2048; the wrapper's GL query was dead (it tested an
  attribute on the wrong object, always the 2048 fallback). The scipy import
  only the wrapper used goes with it.

Tests: tests/test_downsampling_wrapper.py deleted (14 tests); new
tests/test_3d_scales.py (scale passed for "z" and for "z_level", none without
both sizes, no ndv.views._vispy import left in core.py).

Window check (cocoa, NDVL_REAL_GL=1, real LightweightViewer, (3, 2100, 2100)
uint16 volume, dz 1.5 um / px 0.5 um), identical on vispy 0.17.0 and 0.14.3:
ndv's 3D button picks (z, y, x) past the channel axis, resolved
visible_scales (3.0, 1.0, 1.0), STTransform scale (2, 2, 3) over a
(3, 1050, 1050) texture (strides (1, 2, 2)), camera xlim/ylim (-21, 2121),
zlim (-0.09, 9.09), 0 draw errors.

Runs: ndv 0.5.0 offscreen PyQt5 277 passed / 3 skipped; PyQt6 277 passed /
3 skipped (287 before: -14 wrapper tests, +4 new).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ument the ndv 0.5 integration

.github/workflows/ci.yml: the test job runs the suite twice. First under the Qt
offscreen platform (QT_QPA_PLATFORM=offscreen set explicitly instead of relying
on the test modules' setdefault) with ndv's GL texture-size query pinned by
tests/conftest.py: the headless path, window smokes skipped. Then under xcb on
Xvfb (+extension GLX) with NDVL_REAL_GL=1, where the real query runs and the
@pytest.mark.window texture smokes execute; it runs even when the first step
failed so both results are visible. The Mesa packages (libgl1, libglx-mesa0,
libgl1-mesa-dri, libegl-mesa0) give Xvfb a software GL context; ndv's own CI
installs the same set. This real-GL step is new and verified by the first CI
run of the PR.

docs/DESIGN.md: new "ndv Integration (ndv 0.5)" section (the public APIs used
per need, the two-signal in-place swap, the private points that remain and the
tests that pin them, headless testing); set_current_index contract under
Navigation; the 3D section now describes scales= and ndv's own downsampling
instead of the deleted wrapper and vispy patch; the in-place row of the
performance table carries the probe numbers. docs/zarr_viewer_architecture.md:
how attrs["luts"] and the pixel sizes reach ndv's constructor; related files.
README.md: requirements (ndv>=0.5,<0.6, numpy>=2 via ihist, qtpy binding) and a
Development section with the offscreen / real-GL test commands. tests/README.md:
the offscreen default comes from the test modules, not the viewer modules.

Runs on this branch: ndv 0.5.0 / vispy 0.17.0 offscreen, PyQt5 5.15.14: 277
passed / 3 skipped; PyQt6 6.12.0: 277 passed / 3 skipped; cocoa NDVL_REAL_GL=1
(PyQt5): 280 passed (S1-S3 ran); black --check clean.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… and a GL context before the real-GL run

The real-GL step is the first time this CI loads PyQt6's xcb platform plugin,
and from Qt 6.5 that plugin refuses to load without libxcb-cursor0
(environment.yml installs PyQt6 unpinned, 6.12.0 today). The ubuntu-24.04
runner image does not ship it, so the step would have aborted before pytest
collected anything. Add libxcb-cursor0, plus libxcb-shape0, libxcb-xinput0 and
libopengl0, and describe the list as what it is: a subset of the set ndv's CI
installs through pyvista/setup-headless-display-action with qt: true.

A step before the real-GL run now loads the xcb plugin under Xvfb and creates
a GL context through qtpy on its own, so a plugin-load or no-GL failure is
reported with its own message instead of as a pytest failure.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ks the smokes ran

The window smokes (S1-S3) are the only texture-level coverage of the port, and
their skip path fired on a timing race, not only on an ndv rename: _walk()
called pytest.skip on any KeyError/IndexError, but ndv 0.5 creates
_lut_controllers[key] and its image handle in _on_data_response_ready, a
main-thread callback queued after the async fetch, while _is_idle() is already
True once the worker's future is done. A smoke polling in that window was
SKIPPED with a message blaming an ndv rename (reproduced on ndv 0.5.0: idle
True, controllers {}, predicate -> Skipped "KeyError: 0"), and a real GL or ndv
failure in CI's real-GL step would have surfaced as skips too.

- tests/_ndv_private.py: skip only on AttributeError/TypeError (the path itself
  changed); a missing key or index returns a sentinel. displayed_texture() is
  None until the handle exists; dims_slider() fails loudly for a dim ndv never
  built; is_idle() documents the ordering.
- tests/test_window_smokes.py: texture_mean_is() returns False (a real bool)
  while the texture is absent, so wait_until keeps polling and a timeout is a
  failure, never a skip; S1's final read is None-safe.
- tests/test_ndv_private_helper.py (new): pins the three outcomes on stand-in
  objects (before the change: 5 skipped, 1 failed on np.True_).
- ci.yml real-GL step: set -eo pipefail, -rs --junitxml, then a check that the
  window smokes were collected and none skipped (checked locally: a cocoa
  report passes; an all-skipped offscreen report and a no-smokes report fail).
- README.md, tests/README.md, docs/DESIGN.md: describe the two outcomes.

ndv 0.5 (uv, offscreen): 289 passed, 3 skipped (the smokes) under PyQt5 and
under PyQt6; cocoa smokes with NDVL_REAL_GL=1: 3 passed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…_is into the test helper

scipy's only user was scipy.ndimage.zoom inside Downsampling3DXarrayWrapper,
deleted in 67040e9; nothing under ndviewer_light/ imports it now, so it leaves
pyproject.toml, setup.cfg and environment.yml. superqt[iconify] gets the
>=0.8.1 floor ndv 0.5's own Qt extra declares, so an install into an env that
already holds an older superqt (napari needs only >=0.6.7) upgrades it instead
of failing at ndv's QIconifyIcon import.

texture_mean_is is composed only of is_idle and displayed_texture, so it lives
in tests/_ndv_private.py with them; test_ndv_private_helper.py no longer
imports a window-marked test module to get at it, and its docstring says what
it is (stand-in viewer objects, no window) instead of claiming "no Qt".

Final-panel minors from the ndv 0.5 port review. ndv 0.5.0 offscreen: 289
passed / 3 skipped (PyQt5 and PyQt6); cocoa NDVL_REAL_GL=1: full suite green.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…l block is released

Review finding on #44 (reproduced): ndv 0.5.0's downsample_data strides an
oversized texture to a view, and vispy's VolumeVisual keeps whatever it is
given as _last_data for CPU clim rescaling. So while a volume was shown, the
full-resolution (z, y, x) block of every displayed channel stayed in RAM: a
lazy (32, 1, 64, 64) stack under a forced 16-voxel limit retained 256 KiB for
an 8 KiB texture, where main (the old wrapper's zoomed copy) retained 8 KiB.
For a 50 x 3000 x 3000 uint16 stack that is 900 MB kept per channel where
225 MB is displayed.

ndviewer_light/_ndv_compat.py: install_compact_downsample() wraps
ndv.views._vispy._array_canvas.downsample_data (the name the vispy canvas
binds at import) to np.ascontiguousarray the result when any factor > 1 and
to return the same object otherwise; idempotent; warns and returns if ndv's
private path is gone. core.py calls it from its ndv import block and still
imports nothing under ndv.views (test_core_imports_no_private_vispy_canvas
holds). The shim is the one private ndv patch on this branch; DESIGN.md lists
it under "Private points that remain" with its removal condition (ndv's
downsample_data returning a copy).

tests/test_ndv_compat.py pins it: the wrapped function owns memory only when
strided, install is idempotent, and a displayed lazy volume's texture has no
base array. The reviewer's probe now reports 0 materialised arrays alive.

ndv 0.5.0 offscreen: 292 passed / 3 skipped (PyQt5 and PyQt6); cocoa
NDVL_REAL_GL=1: 295 passed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
ndv 0.5 re-resolves the display state on every slider move, on the main
thread, and walks every coordinate in Python: tuple(coords[d]) in
_resolve._norm_data_coords and [float(v) for v in coord] in
DataWrapper.axis_scales. ndv's stock xarray wrapper returns xarray's own
coords, and iterating one of those builds a new DataArray per element:
~8,000 per move for a 2048 x 2048 image, 100-450 ms of frozen UI per step
(ndv 0.4 did not do this, so it is a regression of the port).

_set_ndv_data now passes ndv a DataWrapper subclass (public ndv API) whose
coords are the underlying numpy arrays. Its supports() is False so
DataWrapper.create never picks it for other xarray objects, and the
in-place push-mode swap keeps working on it.

2048 x 2048 x 3-channel TIFF dataset: slider step 200-490 ms -> ~30 ms
(main, ndv 0.4: ~65 ms).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ndv 0.5 tags each data request with a generation and draws a response
only if it is from the latest one. Every slider move starts a new
request, so while a slider is dragged faster than a frame loads, every
response arrives superseded and the image stays frozen until the slider
stops. ndv 0.4 drew those frames, so this is a regression of the port.

_ndv_compat.install_draw_superseded_frames() draws a superseded response
when only the index changed since it was requested and nothing newer is
on screen; responses requested before a data, shape or layout change are
still discarded. Same logic as the fix proposed to ndv
(ArrayViewer._is_drawable); the shim stands down once ndv has it. It
wraps five private ArrayViewer methods and is pinned by
tests/test_draw_superseded.py (the drag test fails without it: only the
final plane is drawn).

Fast drag (a move every 8 ms, cached planes): 1/233 positions drawn ->
104/105.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@hongquanli

Copy link
Copy Markdown
Collaborator Author

Two follow-up commits from testing this branch on Windows against real data. Each fixes a regression that the ndv 0.5 port introduces relative to main.

994d8d7 perf: hand ndv numpy coordinates so slider moves don't freeze the UI
ndv 0.5 re-resolves the display state on every slider move, on the main thread, and iterates every coordinate in Python (_norm_data_coords, DataWrapper.axis_scales). With ndv's stock xarray wrapper each element becomes a new DataArray, about 8 000 per move for a 2048 × 2048 image. _set_ndv_data now passes a DataWrapper subclass (public ndv API, supports() is False) whose coords are the underlying numpy arrays.

13873b7 fix: keep drawing frames while a slider is dragged (ndv 0.5 workaround)
Since pyapp-kit/ndv#233, ndv draws a response only if it belongs to the latest request, so during a drag faster than one fetch the image freezes until the slider stops (0.4 drew those frames). _ndv_compat.install_draw_superseded_frames() draws a superseded response when only the index changed and nothing newer is on screen. Responses from before a data, shape or layout change are still dropped. It wraps five private ArrayViewer methods, is pinned by tests/test_draw_superseded.py, and stands down once ndv has the upstream fix: upstream PR to pyapp-kit/ndv to follow.

Numbers (Windows, PyQt5, real window, 2048 × 2048 × 3-channel TIFF dataset):

main (ndv 0.4) this PR before after
slider step ~65 ms 200–490 ms ~30 ms
positions drawn, drag with a move every 8 ms 38 / 39 1 / 233 104 / 105

Separately, OME-Zarr plates were slow on main as well, which is not caused by this PR: #45.

Test runs: offscreen 300 passed / 3 skipped; real GL (QT_QPA_PLATFORM=windows NDVL_REAL_GL=1) 303 passed. In both, the only failures are the two Windows-only path tests that also fail on main. On Windows two of this PR's tests also need PYTHONUTF8=1, because test_3d_scales.py and test_channel_names.py call read_text() without encoding="utf-8". CI runs on Linux, so it won't hit this.

🤖 Generated with Claude Code

@hongquanli

Copy link
Copy Markdown
Collaborator Author

Upstream fix for the drag issue is now open: pyapp-kit/ndv#290. Once a released ndv includes it and the pin here moves past that release, _ndv_compat.install_draw_superseded_frames() stands down on its own and can be deleted along with tests/test_draw_superseded.py.

@hongquanli
hongquanli merged commit 2366671 into main Nov 8, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant