Skip to content

feat(pixi-build-ros): RoboStack's rosdep mappings per distribution, one lookup shared with RoboStack - #7533

Open
Tobias-Fischer wants to merge 3 commits into
prefix-dev:mainfrom
Tobias-Fischer:pixi-build-ros-shared-map
Open

Tobias-Fischer wants to merge 3 commits into
prefix-dev:mainfrom
Tobias-Fischer:pixi-build-ros-shared-map

Conversation

@Tobias-Fischer

@Tobias-Fischer Tobias-Fischer commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Description

Human Tobi description: Follow-up to #7526 and #7529.

The current pixi-build-ros is not feature complete for the new vinca-rs, and also has an outdated robostack.yaml. This PR adds some features that are being re-used in the new vinca-rs, e.g. better platform lookup. The PR also removes deprecated REQUIRE_GL / REQUIRE_OPENGL handling. The mapping here had a few keys that RoboStack hadn't had before, and they're added to the robostack monorepo.

AI-generated details (Claude Opus 5.5) pixi-build-ros's bundled `robostack.yaml` was a hand-maintained copy of RoboStack's rosdep mappings, and it had drifted. 64 keys that RoboStack maps were missing and 51 mapped differently (for example `eigen` without `eigen-abi-devel`, and `ignition-gazebo5` → gazebo6). It also still used the `REQUIRE_GL` / `REQUIRE_OPENGL` markers, which RoboStack has since replaced with explicit packages. This PR makes RoboStack's own mappings the source, and gives RoboStack and pixi-build-ros one rosdep lookup:
  • Data. robostack.yaml (every distribution) and the new robostack_distros.yaml (each distribution's own mappings, over it) are copied from RoboStack/robostack: shared/robostack.yaml and distros/<distro>/robostack.yaml. A robostack-bot job will keep them in sync, opening PRs here when the mappings change (RoboStack/robostack PR linked below). The files say not to edit them here, and dprint leaves them alone so that the copies stay verbatim. Distributions with their own mappings: humble, jazzy and noetic (Qt 5; the shared ones are Qt 6).

  • Lookup. package_map::lookup() says what a rosdep key maps to on a platform (MappedPackages::Ros / Conda), or None. RoboStack's recipe generator (vinca-rs) now uses it too, and generates byte-identical recipes with it (44,950 recipes, 4 distributions × 5 platforms).

    • Platform keys can be a subdir (linux-aarch64, which RoboStack's mappings use) or an OS (linux, osx, win64, emscripten); the subdir wins. Before, only linux/osx/win64 worked, and any other platform was an error.
    • bundled_package_map(distro) gives a distribution's mappings.
  • Behaviour changes:

    • A mapping file that can't be read or parsed is an error; before, it was silently ignored.
    • A package.xml version constraint applies to the first package of a mapping; the others are its companions (qt-main >=5.15, libopengl-devel, libgl-devel). Before, a mapping with several packages was an error.
    • REQUIRE_GL / REQUIRE_OPENGL handling is removed.
  • Recipes in the snapshot tests: Qt now comes with libopengl-devel, libgl-devel on Linux, as RoboStack builds it, instead of libgl-devel, xorg-libx11, xorg-libxext.

  • New tests: subdir keys over OS keys, a distribution's mappings over the shared ones (humble's Qt 5), an unreadable mapping file being an error, and the version constraint on the first package.

  • Checks run: cargo test -p pixi-build-ros (73 + 23), cargo clippy -p pixi-build-ros --all-targets -- -D warnings, pixi run typos, dprint check, check-into-diagnostic (one fewer into_diagnostic(), baseline updated).

  • RoboStack side:

    • RoboStack's mappings gained the eight keys only this copy had (crypto++, lua-dev, python3-tabulate, ...).
    • cartographer is dropped on purpose: it's a ROS package, and RoboStack removed that mapping deliberately.

Checklist:

  • I've discussed the proposed changes with a maintainer.
  • I understand the purpose of my changes and how they interact with the greater system.
  • User-facing changes are documented and prose in the documentation has been written/edited/reviewed by hand.

🤖 Generated with Claude Code

@Hofer-Julian

Copy link
Copy Markdown
Member

Converted to draft until there's a summary

@Tobias-Fischer

This comment was marked as outdated.

@Tobias-Fischer
Tobias-Fischer marked this pull request as ready for review November 13, 2026 10:27
Tobias-Fischer added a commit to Tobias-Fischer/vinca-rs that referenced this pull request Nov 13, 2026
conda_index files are loaded and resolved with pixi_build_ros::package_map (the
lookup prefix-dev/pixi uses for `pixi build` of ROS packages), instead of vinca-rs's
own types: subdir keys before OS keys, robostack:/conda:/ros: entries, unreadable
or invalid files are errors. A ros: mapping's packages get the checks of any ROS
dependency (in the distribution, not excluded, else unsatisfied). Pinned to the pixi
branch (prefix-dev/pixi#7533) until it is merged.

The generated recipes are byte-identical (44950 recipes, 4 distributions x 5 platforms).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tobias-Fischer added a commit to Tobias-Fischer/vinca-rs that referenced this pull request Nov 13, 2026
* Rosdep keys resolved by pixi-build-ros's lookup, shared with pixi build

conda_index files are loaded and resolved with pixi_build_ros::package_map (the
lookup prefix-dev/pixi uses for `pixi build` of ROS packages), instead of vinca-rs's
own types: subdir keys before OS keys, robostack:/conda:/ros: entries, unreadable
or invalid files are errors. Pinned to the pixi branch until it is merged.

The generated recipes are byte-identical (44950 recipes, 4 distributions x 5 platforms).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Rosdep keys resolved by pixi-build-ros's lookup, shared with pixi build

conda_index files are loaded and resolved with pixi_build_ros::package_map (the
lookup prefix-dev/pixi uses for `pixi build` of ROS packages), instead of vinca-rs's
own types: subdir keys before OS keys, robostack:/conda:/ros: entries, unreadable
or invalid files are errors. A ros: mapping's packages get the checks of any ROS
dependency (in the distribution, not excluded, else unsatisfied). Pinned to the pixi
branch (prefix-dev/pixi#7533) until it is merged.

The generated recipes are byte-identical (44950 recipes, 4 distributions x 5 platforms).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Tobias-Fischer added a commit to Hofer-Julian/vinca-rs that referenced this pull request Nov 13, 2026
…s-Fischer#3) into add-agents-md-typos-tombi

Conflicts: Cargo.toml keeps main's pixi-build-ros pin (the shared rosdep lookup,
until prefix-dev/pixi#7533 is merged); README's development commands are the
union.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@Hofer-Julian Hofer-Julian left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I let the clanker review the PR. It fixed a few things and found this.
I think that's not a blocker though? What do you say?

I think we're missing Kilted's Qt5/PyQt5 overrides. bundled_package_map("kilted") falls back to the shared mappings, so qtbase5-dev now resolves to qt6-main and python3-qt5-bindings to pyqt6.

Kilted's existing RoboStack mappings still use Qt5/PyQt5, and Kilted's rviz_common requires Qt5. Qt6 won't satisfy that dependency.

Can you add the Kilted overrides to the RoboStack source and resync the bundled mappings before merging? I'd keep the correction upstream so the next sync doesn't overwrite it.

@Tobias-Fischer

Copy link
Copy Markdown
Contributor Author

Thanks, good catch: kilted fell back to the shared Qt 6 mappings. Fixed at the source as you suggested. RoboStack/robostack#217 now has a mapping-only distros/kilted/robostack.yaml (Qt5/PyQt5, plus the xtensor ==0.24.7 kilted was built with, both taken from RoboStack/ros-kilted), so the next sync won't overwrite it. 505cd87 resyncs the bundled maps from that commit and extends test_bundled_distro_mappings_override_shared_ones to check that kilted resolves qtbase5-dev to qt-main and python3-qt5-bindings to pyqt. Your commit is kept underneath.

— Claude (on behalf of @Tobias-Fischer)

@Tobias-Fischer

Copy link
Copy Markdown
Contributor Author

Merge order for the pixi-build-ros RoboStack PRs. GitHub's stacked PRs don't support fork branches, so they're plain PRs with the dependencies below:

  1. feat(pixi-build-ros): RoboStack's rosdep mappings per distribution, one lookup shared with RoboStack #7533 (shared rosdep lookup) and feat(pixi-build-ros): name packages as RoboStack does #7539 (names): independent, either order
  2. feat(pixi-build-ros): build with RoboStack's build scripts #7540 (build scripts): needs feat(pixi-build-ros): name packages as RoboStack does #7539; its first commit is feat(pixi-build-ros): name packages as RoboStack does #7539
  3. feat(pixi-build-ros): make requirements by RoboStack's rules #7541 (requirement rules): needs feat(pixi-build-ros): build with RoboStack's build scripts #7540 and feat(pixi-build-ros): RoboStack's rosdep mappings per distribution, one lookup shared with RoboStack #7533; only its last commit is new

Already merged: #7529 (licenses), #7534 (SPDX fix). Once a PR merges, I'll rebase the next one so its diff shows only its own changes.

— Claude (on behalf of @Tobias-Fischer)

Tobias-Fischer added a commit to Tobias-Fischer/pixi that referenced this pull request Nov 14, 2026
Tobias-Fischer added a commit to Tobias-Fischer/pixi that referenced this pull request Nov 14, 2026
# Conflicts:
#	Cargo.lock
#	crates/pixi_build_ros/src/package_map.rs
Tobias-Fischer added a commit to Tobias-Fischer/pixi that referenced this pull request Nov 14, 2026
Tobias-Fischer added a commit to Tobias-Fischer/pixi that referenced this pull request Nov 14, 2026
@Tobias-Fischer
Tobias-Fischer force-pushed the pixi-build-ros-shared-map branch from fb217de to 4f592e7 Compare November 14, 2026 07:48
Tobias-Fischer added a commit to Tobias-Fischer/pixi that referenced this pull request Nov 14, 2026
# Conflicts:
#	Cargo.lock
#	crates/pixi_build_ros/src/package_map.rs
Tobias-Fischer added a commit to Tobias-Fischer/pixi that referenced this pull request Nov 14, 2026
# Conflicts:
#	Cargo.lock
#	crates/pixi_build_ros/src/package_map.rs
Comment thread docs/build/backends/pixi-build-ros.md Resolved

@Hofer-Julian Hofer-Julian left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I let an agent review this and it found a couple of things.
I feel like I am still not deep enough in the robostack ecosystem to take decisions here.
Please think what to do with them, rather than asking Claude directly to just fix them.

Comment thread docs/build/backends/pixi-build-ros.md Outdated
A missing platform entry, `[]`, or YAML `null` means no dependency for that platform; it does not trigger the ROS-name fallback.
Package templates such as `cuda-nvcc-dev_${{ target_platform }}` are rendered for the requested target before validating the conda spec.
For a conda mapping with several packages, a `package.xml` version constraint applies only to the first package. A pre-existing constraint on that package remains an error.
The old `REQUIRE_GL` and `REQUIRE_OPENGL` markers are not supported; use explicit platform-specific conda packages instead.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we keep REQUIRE_GL and REQUIRE_OPENGL working in user mappings? I tested extra-package-mappings = [{ custom_gl = { conda = ["REQUIRE_OPENGL"] } }] through the backend protocol. On Linux it emits a literal require_opengl dependency instead of the GL/X11 libraries. This breaks an existing config rather than just changing the bundled mappings.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, dropping them broke existing user mappings. They work again (5d5c223): REQUIRE_GL → libgl-devel on Linux; REQUIRE_OPENGL → libgl-devel + libopengl-devel on Linux, xorg-libx11 + xorg-libxext on Linux/macOS. A package.xml version constraint still goes to the mapped package, not to what a marker adds. Test added, docs updated.

🤖 Claude (AI assistant), on behalf of @Tobias-Fischer

}
})?;
map.extend(distros.remove(distro).flatten().unwrap_or_default());
Ok(map)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keep Qt5 mappings for the older distributions too. With distro = "iron", qtbase5-dev resolves to qt6-main because Iron has no override. Iron is explicitly listed in the config docs and previously received qt-main. Fix the missing overrides in RoboStack and re-sync them here so existing Qt5 consumers keep building.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RoboStack has never built Iron (there is no robostack-iron channel), so there is no RoboStack mapping to sync, and adding one would invent a distribution. Distributions without bundled overrides get the shared mappings, which follow the newest distributions (Qt6). The docs now say so, point to extra-package-mappings for changing it, and no longer use Iron as an example (5d5c223).

🤖 Claude (AI assistant), on behalf of @Tobias-Fischer

Comment on lines +298 to 305
return Err(PackageMapError::VersionSpecConflict {
dep_name: dep.name.clone(),
count: conda_packages.len(),
existing_spec: first.clone(),
});
}
first.push_str(spec);
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The first package is not always the one being constrained. Both pygraphviz mappings contain [graphviz, pygraphviz], so <depend version_gte="1.12">python3-pygraphviz</depend> resolves to graphviz >=1.12 plus unconstrained pygraphviz. I reproduced this through the resolver. Put pygraphviz first in the upstream mappings and re-sync them here.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. I checked every multi-package mapping: besides pygraphviz, libudev-dev and libuvc-dev also listed a companion (libusb) first. All are reordered here (5d5c223) and in RoboStack (RoboStack/robostack#223, which this mapping file is synced from), with a test that python3-pygraphviz >=1.12 constrains pygraphviz.

🤖 Claude (AI assistant), on behalf of @Tobias-Fischer

Tobias-Fischer added a commit to RoboStack/robostack that referenced this pull request Nov 19, 2026
pixi-build-ros applies a package.xml version constraint to a mapping's first
package (prefix-dev/pixi#7533 review): python(3)-pygraphviz constrained graphviz,
libudev-dev and libuvc-dev constrained libusb. Same packages, reordered;
RoboStack's own recipes don't use the order.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ne lookup shared with RoboStack

The bundled robostack.yaml was a hand-maintained copy of RoboStack's mappings that
had drifted (64 keys missing, 51 mapped differently, the old REQUIRE_GL /
REQUIRE_OPENGL markers). RoboStack's own mappings are now the source, and
RoboStack's recipe generator (vinca-rs) uses the same lookup:
- robostack.yaml (every distribution) and robostack_distros.yaml (each
  distribution's own mappings over it: humble, jazzy, kilted and noetic keep Qt 5)
  are copied from RoboStack/robostack, verbatim (dprint leaves them alone).
- package_map::lookup() says what a rosdep key maps to on a platform; platform
  keys can be a subdir (linux-aarch64) or an OS, and the subdir wins. A platform
  without a value maps to nothing. bundled_package_map(distro) gives a
  distribution's mappings; mapping paths and package templates are resolved.
- A mapping file that can't be read or parsed is an error; a package.xml version
  constraint applies to a mapping's first package (the others are companions);
  REQUIRE_GL / REQUIRE_OPENGL handling is removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n the named package

- User mappings with the older REQUIRE_GL / REQUIRE_OPENGL markers resolve to
  the GL/X11 packages again instead of a literal require_opengl dependency.
- Re-sync of RoboStack's reordered mappings (pygraphviz, libudev-dev,
  libuvc-dev), so a version constraint applies to the named package.
- Docs: which distributions get bundled overrides (Iron is not one).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Tobias-Fischer
Tobias-Fischer force-pushed the pixi-build-ros-shared-map branch from 5d5c223 to f1d649c Compare November 20, 2026 19:35
@Tobias-Fischer

Copy link
Copy Markdown
Contributor Author

@Hofer-Julian this is ready for re-review :)

@benjaminholdennearearth

Copy link
Copy Markdown

I think this adds a subtle breaking change in how relative extra-package-mappings paths are resolved, and wanted to ask about the intended workflow for multi-package ROS workspaces.

In my workspace setup we have a multi-repo ROS 2 workspace (managed with vcstool) with ~100 packages under src/<repo>/<pkg>/ listed in the root pixi.toml's [dev] table, plus a single shared mappings.yaml at the workspace root for custom/internal dependencies:

  workspace/
  ├── pixi.toml          # [dev] references packages under src/
  ├── mappings.yaml      # single shared mapping file for the workspace
  └── src/
      └── <repo>/<pkg>/
          ├── package.xml
          └── pixi.toml  # only present when the package needs mappings.yaml
  • Packages that don't need the custom mappings are in the dev table as { path = "src/<repo>/<pkg>/package.xml" } with no pixi.toml in src/.
  • I have about 28 packages that need entries from mappings.yaml currently have a minimal src/<repo>/<pkg>/pixi.toml containing:
  [package.build.config]
  distro = "jazzy"
  extra-package-mappings = [{ path = "mappings.yaml" }]

Previously, relative extra-package-mappings paths resolved against the process CWD (the workspace root), so { path = "mappings.yaml" } found the single root mappings.yaml regardless of nesting depth.

With this PR, I think relative paths are joined with manifest_root (src/<repo>/<pkg>/mappings.yaml), and unreadable/missing files are now a hard error. That makes sense, but it breaks all of our src/<repo>/<pkg>/pixi.toml files. This means I will have to hardcode ../../../mappings.yaml inside sub-repositories which is brittle because those sub-repos are reused across different workspaces and nesting depths.

So my questions are:

  1. Is resolving relative extra-package-mappings strictly against manifest_root intentional here, or should pixi-build-ros also check/fall back to workspace_directory (which is already passed in for sibling discovery)?
  2. Longer term, what is the intended way to share a single workspace-level mappings.yaml across package.xml targets in [dev] without having to maintain a per-package pixi.toml in src/?

… the workspace root too

A relative mapping file path is looked up next to the package's manifest
first and then at the workspace root, so one workspace-level mapping file
keeps serving packages at any depth (before, relative paths followed the
backend's working directory). A file found in neither place is an error
naming both locations; the resolved file is tracked as a metadata input.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Tobias-Fischer

Copy link
Copy Markdown
Contributor Author

Thanks for the detailed report, you're right: this PR made relative extra-package-mappings paths resolve against the package directory and made a missing file a hard error, which together broke your layout.

  1. Fixed in a3e97f5: a relative path is now looked up next to the package's manifest first and then at the workspace root, so your src/<repo>/<pkg>/pixi.toml files with { path = "mappings.yaml" } find the root mappings.yaml again at any nesting depth, without ../../../. A package-local file of the same name still wins. A file found in neither place stays an error, but the message now names both locations it looked in. The resolved file is also tracked as a metadata input, so editing the root mappings.yaml invalidates the cached metadata of those packages. (Before this PR the path followed the backend process's working directory, which happened to be the workspace root in your setup.)
  2. Sharing one workspace-level mapping across package.xml entries in [dev] without a per-package pixi.toml would need build config that packages inherit from the workspace, which is a broader pixi feature rather than something for this PR. With the fallback above, the minimal per-package pixi.toml keeps working unchanged in the meantime.

🤖 Claude (AI assistant), on behalf of @Tobias-Fischer

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants