Repository navigation
feat(pixi-build-ros): RoboStack's rosdep mappings per distribution, one lookup shared with RoboStack - #7533
feat(pixi-build-ros): RoboStack's rosdep mappings per distribution, one lookup shared with RoboStack#7533Tobias-Fischer wants to merge 3 commits into
Conversation
|
Converted to draft until there's a summary |
This comment was marked as outdated.
This comment was marked as outdated.
conda_index files are loaded and resolved with pixi_build_ros::package_map (the lookup prefix-dev/pixi uses for `pixi build` of ROS packages), instead of vinca-rs's own types: subdir keys before OS keys, robostack:/conda:/ros: entries, unreadable or invalid files are errors. A ros: mapping's packages get the checks of any ROS dependency (in the distribution, not excluded, else unsatisfied). Pinned to the pixi branch (prefix-dev/pixi#7533) until it is merged. The generated recipes are byte-identical (44950 recipes, 4 distributions x 5 platforms). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Rosdep keys resolved by pixi-build-ros's lookup, shared with pixi build conda_index files are loaded and resolved with pixi_build_ros::package_map (the lookup prefix-dev/pixi uses for `pixi build` of ROS packages), instead of vinca-rs's own types: subdir keys before OS keys, robostack:/conda:/ros: entries, unreadable or invalid files are errors. Pinned to the pixi branch until it is merged. The generated recipes are byte-identical (44950 recipes, 4 distributions x 5 platforms). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Rosdep keys resolved by pixi-build-ros's lookup, shared with pixi build conda_index files are loaded and resolved with pixi_build_ros::package_map (the lookup prefix-dev/pixi uses for `pixi build` of ROS packages), instead of vinca-rs's own types: subdir keys before OS keys, robostack:/conda:/ros: entries, unreadable or invalid files are errors. A ros: mapping's packages get the checks of any ROS dependency (in the distribution, not excluded, else unsatisfied). Pinned to the pixi branch (prefix-dev/pixi#7533) until it is merged. The generated recipes are byte-identical (44950 recipes, 4 distributions x 5 platforms). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…s-Fischer#3) into add-agents-md-typos-tombi Conflicts: Cargo.toml keeps main's pixi-build-ros pin (the shared rosdep lookup, until prefix-dev/pixi#7533 is merged); README's development commands are the union. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Hofer-Julian
left a comment
There was a problem hiding this comment.
I let the clanker review the PR. It fixed a few things and found this.
I think that's not a blocker though? What do you say?
I think we're missing Kilted's Qt5/PyQt5 overrides. bundled_package_map("kilted") falls back to the shared mappings, so qtbase5-dev now resolves to qt6-main and python3-qt5-bindings to pyqt6.
Kilted's existing RoboStack mappings still use Qt5/PyQt5, and Kilted's rviz_common requires Qt5. Qt6 won't satisfy that dependency.
Can you add the Kilted overrides to the RoboStack source and resync the bundled mappings before merging? I'd keep the correction upstream so the next sync doesn't overwrite it.
|
Thanks, good catch: kilted fell back to the shared Qt 6 mappings. Fixed at the source as you suggested. RoboStack/robostack#217 now has a mapping-only — Claude (on behalf of @Tobias-Fischer) |
# Conflicts: # Cargo.lock # crates/pixi_build_ros/src/package_map.rs
# Conflicts: # Cargo.lock # crates/pixi_build_ros/src/package_map.rs
fb217de to
4f592e7
Compare
# Conflicts: # Cargo.lock # crates/pixi_build_ros/src/package_map.rs
# Conflicts: # Cargo.lock # crates/pixi_build_ros/src/package_map.rs
Hofer-Julian
left a comment
There was a problem hiding this comment.
I let an agent review this and it found a couple of things.
I feel like I am still not deep enough in the robostack ecosystem to take decisions here.
Please think what to do with them, rather than asking Claude directly to just fix them.
| A missing platform entry, `[]`, or YAML `null` means no dependency for that platform; it does not trigger the ROS-name fallback. | ||
| Package templates such as `cuda-nvcc-dev_${{ target_platform }}` are rendered for the requested target before validating the conda spec. | ||
| For a conda mapping with several packages, a `package.xml` version constraint applies only to the first package. A pre-existing constraint on that package remains an error. | ||
| The old `REQUIRE_GL` and `REQUIRE_OPENGL` markers are not supported; use explicit platform-specific conda packages instead. |
There was a problem hiding this comment.
Can we keep REQUIRE_GL and REQUIRE_OPENGL working in user mappings? I tested extra-package-mappings = [{ custom_gl = { conda = ["REQUIRE_OPENGL"] } }] through the backend protocol. On Linux it emits a literal require_opengl dependency instead of the GL/X11 libraries. This breaks an existing config rather than just changing the bundled mappings.
There was a problem hiding this comment.
Agreed, dropping them broke existing user mappings. They work again (5d5c223): REQUIRE_GL → libgl-devel on Linux; REQUIRE_OPENGL → libgl-devel + libopengl-devel on Linux, xorg-libx11 + xorg-libxext on Linux/macOS. A package.xml version constraint still goes to the mapped package, not to what a marker adds. Test added, docs updated.
🤖 Claude (AI assistant), on behalf of @Tobias-Fischer
| } | ||
| })?; | ||
| map.extend(distros.remove(distro).flatten().unwrap_or_default()); | ||
| Ok(map) |
There was a problem hiding this comment.
Keep Qt5 mappings for the older distributions too. With distro = "iron", qtbase5-dev resolves to qt6-main because Iron has no override. Iron is explicitly listed in the config docs and previously received qt-main. Fix the missing overrides in RoboStack and re-sync them here so existing Qt5 consumers keep building.
There was a problem hiding this comment.
RoboStack has never built Iron (there is no robostack-iron channel), so there is no RoboStack mapping to sync, and adding one would invent a distribution. Distributions without bundled overrides get the shared mappings, which follow the newest distributions (Qt6). The docs now say so, point to extra-package-mappings for changing it, and no longer use Iron as an example (5d5c223).
🤖 Claude (AI assistant), on behalf of @Tobias-Fischer
| return Err(PackageMapError::VersionSpecConflict { | ||
| dep_name: dep.name.clone(), | ||
| count: conda_packages.len(), | ||
| existing_spec: first.clone(), | ||
| }); | ||
| } | ||
| first.push_str(spec); | ||
| } | ||
|
|
There was a problem hiding this comment.
The first package is not always the one being constrained. Both pygraphviz mappings contain [graphviz, pygraphviz], so <depend version_gte="1.12">python3-pygraphviz</depend> resolves to graphviz >=1.12 plus unconstrained pygraphviz. I reproduced this through the resolver. Put pygraphviz first in the upstream mappings and re-sync them here.
There was a problem hiding this comment.
Agreed. I checked every multi-package mapping: besides pygraphviz, libudev-dev and libuvc-dev also listed a companion (libusb) first. All are reordered here (5d5c223) and in RoboStack (RoboStack/robostack#223, which this mapping file is synced from), with a test that python3-pygraphviz >=1.12 constrains pygraphviz.
🤖 Claude (AI assistant), on behalf of @Tobias-Fischer
pixi-build-ros applies a package.xml version constraint to a mapping's first package (prefix-dev/pixi#7533 review): python(3)-pygraphviz constrained graphviz, libudev-dev and libuvc-dev constrained libusb. Same packages, reordered; RoboStack's own recipes don't use the order. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ne lookup shared with RoboStack The bundled robostack.yaml was a hand-maintained copy of RoboStack's mappings that had drifted (64 keys missing, 51 mapped differently, the old REQUIRE_GL / REQUIRE_OPENGL markers). RoboStack's own mappings are now the source, and RoboStack's recipe generator (vinca-rs) uses the same lookup: - robostack.yaml (every distribution) and robostack_distros.yaml (each distribution's own mappings over it: humble, jazzy, kilted and noetic keep Qt 5) are copied from RoboStack/robostack, verbatim (dprint leaves them alone). - package_map::lookup() says what a rosdep key maps to on a platform; platform keys can be a subdir (linux-aarch64) or an OS, and the subdir wins. A platform without a value maps to nothing. bundled_package_map(distro) gives a distribution's mappings; mapping paths and package templates are resolved. - A mapping file that can't be read or parsed is an error; a package.xml version constraint applies to a mapping's first package (the others are companions); REQUIRE_GL / REQUIRE_OPENGL handling is removed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n the named package - User mappings with the older REQUIRE_GL / REQUIRE_OPENGL markers resolve to the GL/X11 packages again instead of a literal require_opengl dependency. - Re-sync of RoboStack's reordered mappings (pygraphviz, libudev-dev, libuvc-dev), so a version constraint applies to the named package. - Docs: which distributions get bundled overrides (Iron is not one). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
5d5c223 to
f1d649c
Compare
|
@Hofer-Julian this is ready for re-review :) |
|
I think this adds a subtle breaking change in how relative extra-package-mappings paths are resolved, and wanted to ask about the intended workflow for multi-package ROS workspaces. In my workspace setup we have a multi-repo ROS 2 workspace (managed with
[package.build.config]
distro = "jazzy"
extra-package-mappings = [{ path = "mappings.yaml" }]Previously, relative extra-package-mappings paths resolved against the process CWD (the workspace root), so With this PR, I think relative paths are joined with manifest_root ( So my questions are:
|
… the workspace root too A relative mapping file path is looked up next to the package's manifest first and then at the workspace root, so one workspace-level mapping file keeps serving packages at any depth (before, relative paths followed the backend's working directory). A file found in neither place is an error naming both locations; the resolved file is tracked as a metadata input. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks for the detailed report, you're right: this PR made relative
🤖 Claude (AI assistant), on behalf of @Tobias-Fischer |
Description
Human Tobi description: Follow-up to #7526 and #7529.
The current pixi-build-ros is not feature complete for the new vinca-rs, and also has an outdated robostack.yaml. This PR adds some features that are being re-used in the new vinca-rs, e.g. better platform lookup. The PR also removes deprecated
REQUIRE_GL/REQUIRE_OPENGLhandling. The mapping here had a few keys that RoboStack hadn't had before, and they're added to the robostack monorepo.AI-generated details (Claude Opus 5.5)
pixi-build-ros's bundled `robostack.yaml` was a hand-maintained copy of RoboStack's rosdep mappings, and it had drifted. 64 keys that RoboStack maps were missing and 51 mapped differently (for example `eigen` without `eigen-abi-devel`, and `ignition-gazebo5` → gazebo6). It also still used the `REQUIRE_GL` / `REQUIRE_OPENGL` markers, which RoboStack has since replaced with explicit packages. This PR makes RoboStack's own mappings the source, and gives RoboStack and pixi-build-ros one rosdep lookup:Data.
robostack.yaml(every distribution) and the newrobostack_distros.yaml(each distribution's own mappings, over it) are copied from RoboStack/robostack:shared/robostack.yamlanddistros/<distro>/robostack.yaml. A robostack-bot job will keep them in sync, opening PRs here when the mappings change (RoboStack/robostack PR linked below). The files say not to edit them here, and dprint leaves them alone so that the copies stay verbatim. Distributions with their own mappings: humble, jazzy and noetic (Qt 5; the shared ones are Qt 6).Lookup.
package_map::lookup()says what a rosdep key maps to on a platform (MappedPackages::Ros/Conda), orNone. RoboStack's recipe generator (vinca-rs) now uses it too, and generates byte-identical recipes with it (44,950 recipes, 4 distributions × 5 platforms).linux-aarch64, which RoboStack's mappings use) or an OS (linux,osx,win64,emscripten); the subdir wins. Before, onlylinux/osx/win64worked, and any other platform was an error.bundled_package_map(distro)gives a distribution's mappings.Behaviour changes:
qt-main >=5.15, libopengl-devel, libgl-devel). Before, a mapping with several packages was an error.REQUIRE_GL/REQUIRE_OPENGLhandling is removed.Recipes in the snapshot tests: Qt now comes with
libopengl-devel, libgl-develon Linux, as RoboStack builds it, instead oflibgl-devel, xorg-libx11, xorg-libxext.New tests: subdir keys over OS keys, a distribution's mappings over the shared ones (humble's Qt 5), an unreadable mapping file being an error, and the version constraint on the first package.
Checks run:
cargo test -p pixi-build-ros(73 + 23),cargo clippy -p pixi-build-ros --all-targets -- -D warnings,pixi run typos,dprint check,check-into-diagnostic(one fewerinto_diagnostic(), baseline updated).RoboStack side:
crypto++,lua-dev,python3-tabulate, ...).cartographeris dropped on purpose: it's a ROS package, and RoboStack removed that mapping deliberately.Checklist:
🤖 Generated with Claude Code