Skip to content

feat(rai_core): numpy-2-safe image bridge; drop ROS 2 Humble and Python 3.10 - #895

Merged
maciejmajek merged 9 commits into
mainfrom
fix/numpy2-without-cv-bridge
Nov 11, 2026
Merged

maciejmajek merged 9 commits into
mainfrom
fix/numpy2-without-cv-bridge

Conversation

@adamdbrw

@adamdbrw adamdbrw commented Nov 11, 2026 •

Copy link
Copy Markdown
Member

Move rai_core to numpy 2, drop ROS 2 Humble / Python 3.10, and drop the NoMaD integration.

Why

ROS 2 Jazzy's apt cv_bridge is compiled against Ubuntu 24.04's numpy 1.26. Under numpy 2, import cv_bridge hides the import failure, and imgmsg_to_cv2 then segfaults the process. That is why rai_core capped numpy below 2, and that cap is what kept OpenCV at 4.11 for every downstream user.

Changes

  • rai.communication.ros2.image_bridge.CvBridge replaces cv_bridge for the methods RAI uses. It is pure numpy + OpenCV and works under numpy 1 and 2. Every import in rai_core, rai_perception, rai_semap and the tests now uses it.
    • Parity with the real cv_bridge: 1595 + 22 special-value cases, all identical, under numpy 1.26 / OpenCV 4.11 and numpy 2.5 / OpenCV 4.14. The harness lives outside the repo.
    • One deliberate difference: row padding that isn't a whole number of pixels is read correctly (cv_bridge scrambles those images), and a step shorter than a row raises an error instead of cropping silently.
  • rai_core requires numpy >=2.0,<3 and opencv-python(-headless) >=4.10.0.84,<6 (4.10.0.84 is the first OpenCV build that imports under numpy 2). Only numpy 2 is supported, so there is one configuration to test. requires-python is >=3.12,<3.13 for the workspace and its packages; rai_finetune is unchanged because it is not a workspace member.
  • rai_semap requires rai_perception>=0.5.0, since 0.4.0 still imports the apt cv_bridge.
  • Humble is removed from the CI matrix, download_demo.sh, a service-call branch, the test notes and the docs.
  • NoMaD is removed: the nomad dependency group, the diffusion-policy override, rai_nomad, and its docs. Its visualnav-transformer required numpy<2 and was the only thing holding the lock on numpy 1.26.
  • uv.lock is on numpy 2.5.4 / OpenCV 5.0.0 (re-locked with uv lock -P numpy -P opencv-python -P opencv-python-headless). 84 nomad-only packages are removed and nothing else changes.
  • CI: the pre-commit job runs on the plain runner instead of a ROS container, and the license check runs the same checker in a prebuilt Deno image instead of building an Ubuntu image with apt on every run. Both had been failing on runner DNS outages.
  • Versions: rai_core 2.15.0, rai_perception 0.5.0, rai_semap 0.2.0, rai_s2s 1.2.0, rai_bench 0.5.0, rai_sim 0.1.0, rai_whoami 0.3.0.

Release notes

  • Dropped: Python 3.10, ROS 2 Humble, Ubuntu 22.04, numpy 1, and the NoMaD integration (rai_nomad, --group nomad). Projects that need numpy<2 stay on rai_core 2.14.
  • Code that still imports cv_bridge in the same process segfaults on the first imgmsg_to_cv2 once numpy 2 is installed. Migrate to from rai.communication.ros2.image_bridge import CvBridge.
  • With ros2 run / ros2 launch (system Python with the venv on PYTHONPATH), venv numpy 2 shadows the apt numpy, so apt-only scipy and matplotlib fail to import. Install them into the venv.
  • Conversion errors are rai.communication.ros2.image_bridge.CvBridgeError (a TypeError), not cv_bridge.CvBridgeError. Some failures that cv_bridge reported as KeyError or cv2.error now raise it too.

Tested

  • Jazzy, CI-style venv, numpy 2.5.4 + OpenCV 5.0.0 (locked): 954 passed, 0 failed.
  • Other Jazzy Python packages under numpy 2: only the apt cv_bridge uses numpy's C API. Message serialization, rclpy pub/sub, tf2, rosbag2, point_cloud2, image_geometry and laser_geometry all work under numpy 2.0 and 2.3.

🤖 Generated with Claude Code

adamdbrw and others added 2 commits November 11, 2026 16:48
… cv_bridge

ROS 2 Jazzy's apt cv_bridge cannot run under numpy 2: its compiled helper
(cv_bridge_boost) is built against Ubuntu 24.04's numpy 1.26, `import
cv_bridge` swallows the import failure, and the first imgmsg_to_cv2 call
then segfaults the interpreter. Big-endian images also hit
ndarray.newbyteorder(), which numpy 2 removed. This is why rai_core had to
cap numpy below 2, which in turn held opencv-python at 4.11.

rai.communication.ros2.image_bridge.CvBridge is a pure numpy + OpenCV
drop-in for the cv_bridge methods RAI uses (imgmsg_to_cv2, cv2_to_imgmsg,
compressed_imgmsg_to_cv2, cv2_to_compressed_imgmsg). It follows
cv_bridge's encodings, colour conversions, 8<->16 bit scaling and
refusals, and OpenCV's saturate_cast for float->integer depth changes
(NaN, +-inf and values beyond int32 become INT_MIN before saturation, so
invalid depth pixels become 0, not the maximum). A parity harness against
the real cv_bridge (1595 cases: every source x target encoding, padding,
big-endian, compressed, errors; plus 22 special-value cases) matches 100%
under numpy 1.26/OpenCV 4.11 and numpy 2.5/OpenCV 4.14.

One deliberate difference: rows are read with `step` as the exact row
stride. cv_bridge assumes padding is a whole number of pixels and returns
a scrambled image otherwise; a step shorter than a row now raises instead
of silently cropping.

All cv_bridge imports in rai_core, rai_perception, rai_semap and the tests
now use the new module.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ROS 2 Humble (Python 3.10) is no longer supported; ROS 2 Jazzy on
Python 3.12 is.

- requires-python ">=3.12,<3.13" for the workspace and rai_core,
  rai_perception, rai_semap, rai_s2s, rai_sim, rai_bench, rai_whoami.
  rai_finetune is not part of the uv workspace and is unchanged.
- rai_core: numpy ">=1.26.0,<3.0.0" (was "<2.0"), now that images no
  longer go through cv_bridge; opencv-python(-headless) "<6" (was "<5").
  The Python-3.10-only pillow and streamlit variants are gone; rai_s2s
  likewise drops its 3.10 onnxruntime variants.
- rai_perception and rai_semap require rai_core>=2.15.0 (image_bridge).
- Versions (minor: Python 3.10/Humble users stay on the previous
  releases through requires-python): rai_core 2.15.0, rai_perception 0.5.0, rai_semap 0.2.0,
  rai_s2s 1.2.0, rai_bench 0.5.0, rai_sim 0.1.0, rai_whoami 0.3.0.
- uv.lock re-locked; only the Python 3.10 forks are removed. The
  workspace lock itself stays on numpy 1.26/OpenCV 4.11 because the
  nomad group's visualnav-transformer requires numpy<2.
- ROS2ServiceAPI always passes timeout_sec (the Humble-only branch is
  removed); Humble removed from CI (test matrix, pre-commit container),
  download_demo.sh, test notes and the docs. The pre-commit job now runs in
  the Jazzy container, whose Python is externally managed (PEP 668), so it
  installs pre-commit into a venv.

The test suite passes on Jazzy both as locked (numpy 1.26, OpenCV 4.11)
and with numpy 2.5 + OpenCV 5.0 installed: 930 passed, 0 failed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.13514% with 9 lines in your changes missing coverage. Please review.
✅ Project coverage is 77.28%. Comparing base (30ea963) to head (423e337).
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
...rc/rai_core/rai/communication/ros2/image_bridge.py 95.02% 9 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #895      +/-   ##
==========================================
+ Coverage   76.40%   77.28%   +0.88%     
==========================================
  Files          83       84       +1     
  Lines        3670     3848     +178     
==========================================
+ Hits         2804     2974     +170     
- Misses        866      874       +8     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

@maciejmajek maciejmajek left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work, the core of this holds up well. I checked a few things independently.

Bridge parity. I compared image_bridge.CvBridge with the real cv_bridge from ROS 2 Lyrical (numpy 2.3.5). These all came out identical:

  • imgmsg_to_cv2 on 812 source x target encoding pairs (passthrough, generic nUCm, bare depths, Bayer, YUV, NaN/inf, signed/unsigned 8/16 bit)
  • cv2_to_imgmsg on sliced, Fortran-ordered, big-endian and odd-shaped arrays
  • compression in jpg/jpeg/png/tiff/bmp/webp, and decoding of grayscale, 16-bit and alpha PNGs
  • which methods return a view of msg.data and which return a copy

Big-endian input works in the new bridge, while Lyrical's cv_bridge raises AttributeError on it under numpy 2.

numpy 2 with the rest of Jazzy. In osrf/ros:jazzy-desktop-full, I scanned all 464 Python extension modules. cv_bridge is the only ROS package that uses the numpy C API. Message bindings, rclpy and rosbag2_py include numpy headers but never call import_array. Under numpy 2.0.2 and 2.3.5, all 156 ROS Python packages import the same way as under 1.26. These all work: fixed-array message serialization (CameraInfo, Imu, Odometry), a real rclpy pub/sub round trip, point_cloud2, tf2_ros + tf2_geometry_msgs, image_geometry, laser_geometry, rosbag2_py MCAP write/read, message_filters and nav2_simple_commander. Only the apt cv_bridge.imgmsg_to_cv2 segfaults, as the PR says.

Test suite. I re-ran the suite in osrf/ros:jazzy-desktop-full following the CI steps (vcs import, rosdep, colcon build, CPU torch, jackd). As locked (numpy 1.26.4, OpenCV 4.11): 955 passed, 4 skipped, 11 xfailed, 0 failed. With numpy 2.5.4 and OpenCV 5.0.0 installed over the locked set: 955 passed, 4 skipped, 11 xfailed, 0 failed.

Semver. Minor bumps look right to me under CONTRIBUTING.md. Nothing in the public API, ROS interfaces or config schemas breaks, and requires-python keeps Python 3.10 / Humble users on the previous releases. Removing rai_nomad (see inline) doesn't change that: it is a colcon extension, not part of the published rai_core API.

CONTRIBUTING.md asks for breaking changes to be documented in the release notes, so please add these when releasing rai_core 2.15.0:

  • Dropped: Python 3.10, ROS 2 Humble, Ubuntu 22.04, and the NoMaD integration (rai_nomad, --group nomad).
  • Any code that still imports cv_bridge (your own nodes, other libraries in the same process) will segfault on the first imgmsg_to_cv2 once numpy 2 is installed, and the import gives no warning. Migrate to from rai.communication.ros2.image_bridge import CvBridge. That class covers the four conversion methods. It does not have cv_bridge's helpers cvtype2_to_dtype_with_channels, encoding_to_cvtype2, numpy_type_to_cvtype, cvdepth_to_numpy_depth or cvtype_to_name.
  • With the ros2 run / ros2 launch setup from the docs (system Python with the venv on PYTHONPATH), venv numpy 2 shadows the apt numpy. Apt-only modules then fail to import: scipy ("numpy.dtype size changed") and matplotlib ("numpy.core.multiarray failed to import"). Install them into the venv.
  • Error behaviour changes (details inline):
    • Conversion errors are now rai.communication.ros2.image_bridge.CvBridgeError. It still subclasses TypeError, but except cv_bridge.CvBridgeError no longer catches it.
    • A step shorter than one row now raises instead of silently cropping.
    • An undecodable compressed image with a non-passthrough encoding now raises instead of returning None.

Small nit outside the diff: src/rai_finetune/README.md:59 and src/rai_finetune/pyproject.toml:8 still say Python 3.10 was chosen "for compatibility with the rest of RAI components". After this PR, rai_finetune (>=3.10,<3.12) and the rest (==3.12) share no Python version, so that sentence is now wrong. Either update the wording or move rai_finetune to 3.12.

Blocking for me: dropping nomad together with the lock move, CI coverage for the numpy version the lock doesn't use, and the rai_perception pin in rai_semap (all inline).

Comment thread pyproject.toml Resolved
Comment thread src/rai_semap/pyproject.toml Resolved
Comment thread src/rai_core/pyproject.toml Resolved
Comment thread .github/workflows/pre-commit.yml Resolved
maciejmajek and others added 7 commits November 11, 2026 17:32
The nomad dependency group (visualnav-transformer, which requires
numpy<2) was the only thing keeping the workspace lock on numpy 1.26.
The integration is low value and the model is outdated, so it goes:
the nomad group, the diffusion-policy override, rai_nomad, its docs
page, and every reference in the docs, READMEs, NOTICE, the
configurator and the rai_semap design notes.

uv.lock is re-locked with `uv lock -P numpy -P opencv-python
-P opencv-python-headless`: 84 nomad-only packages are removed, numpy
moves 1.26.4 -> 2.5.4 and opencv-python(-headless) 4.11.0.86 ->
5.0.0.93, and nothing else changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
rai_perception 0.4.0 still imports the apt cv_bridge, which segfaults
under numpy 2. Pin the release that uses rai's image bridge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
uv.lock is now on numpy 2, but rai_core still allows numpy 1.26, which
is what ROS 2 Jazzy ships through apt. Add a matrix leg that installs
numpy 1.26.4 with opencv-python(-headless) <4.12 (4.12+ needs numpy 2)
on top of the locked set, so both ends of the range are tested.
Coverage is uploaded from the locked leg only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
rai_finetune said Python 3.10 was chosen for compatibility with the
rest of RAI, which now requires Python 3.12. Say that it needs its own
virtual environment instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…matrix

None of the pre-commit hooks need ROS, so the job no longer runs in the
jazzy-desktop-full container: no image pull, no apt step inside the
container, and no PEP 668 venv workaround. Python comes from
setup-python, and shellcheck (already on the runner image) is installed
only if missing. The `vcs import` step is dropped: pre-commit
--all-files only checks tracked files, so the imported repos were never
linted.

build-and-test sets fail-fast: false so a failure in one numpy leg does
not cancel the other.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kt3k/license_checker builds an ubuntu:latest image and apt-installs
curl, git and unzip on every run, so the check fails whenever the
runner cannot reach the Ubuntu mirrors (it did on both runs of this
PR). Run the same checker (license_checker v3.1.2, same .licenserc.json)
in denoland/deno:alpine-1.46.3, pinned by digest. Locally it checks the
same 423 files and still fails on a file without the header.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Support only numpy 2 instead of both 1.26 and 2, so there is one
configuration to test and maintain. rai_core now requires
numpy>=2.0.0,<3 and opencv-python(-headless)>=4.10.0.84: 4.10.0.84 is
the first opencv-python build that imports under numpy 2 (4.9.0.80 and
4.10.0.82 fail with "numpy.core.multiarray failed to import").

The numpy 1.26 CI leg is no longer needed, so build-and-test goes back
to a single job.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@maciejmajek
maciejmajek merged commit 7b75a3d into main Nov 11, 2026
10 of 12 checks passed
@maciejmajek
maciejmajek deleted the fix/numpy2-without-cv-bridge branch November 11, 2026 17:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants