Skip to content

feat(pixi-build-ros): make requirements by RoboStack's rules - #7541

Open
Tobias-Fischer wants to merge 8 commits into
prefix-dev:mainfrom
Tobias-Fischer:pixi-build-ros-requirements
Open

Tobias-Fischer wants to merge 8 commits into
prefix-dev:mainfrom
Tobias-Fischer:pixi-build-ros-requirements

Conversation

@Tobias-Fischer

@Tobias-Fischer Tobias-Fischer commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Description

Part 3 of 3, split out of #7535 for easier review. It builds on #7540 (build scripts), #7539 (names) and #7533 (the shared rosdep lookup, merged in): only the last commit, "make requirements by RoboStack's rules", is this PR. (GitHub's stacked pull requests don't support branches from forks, so this is a plain PR against main.)

Tobi human description

Similar to #7540 - the pixi-build-ros way of converting a package.xml into a recipe.yaml differs from the RoboStack way. This PR mirrors the current RoboStack way of the build/host requirements, which supports cross-compilation, uses proper compiler packages, etc.

With this PR (on top of the other ones), pixi-build-ros generates recipes that are identical to the current vinca.

More details in AI summary below.

AI-generated details (Claude Opus 5.5)

pixi-build-ros and RoboStack's recipe generator (vinca-rs) each had their own rules for turning a package.xml into conda requirements, and they had drifted apart. This PR puts those rules in one place (requirements::Rules), used by both:

  • ROS dependencies go to host only, no longer also to build, which is right for cross-compiling.
  • build holds compilers, cmake, ninja, cython (it said cpython), and the cross-compilation tools when cross-compiling.
  • Python, ros_workspace and ros_environment are handled as in RoboStack.

It adds about 440 lines in requirements.rs (plus tests) and removes about 290 from main.rs and package_map.rs.

With all three parts, RoboStack generates its recipes with these functions and gets byte-identical recipes for all four distributions and five platforms: vinca-rs with this stack gives the same 44,950 recipes as before (humble, jazzy, lyrical, rolling × linux-64, linux-aarch64, osx-64, osx-arm64, win-64).

  • Snapshot changes: build is the base list. host and run are sorted and include ros_environment, ros_workspace and python.
  • Library API used by RoboStack's generator: requirements::{Rules, Reqs, Req, DepKind, python_needed, CORE}.
  • Checks run on the stack: cargo test -p pixi-build-ros (79 + 23), clippy with -D warnings, cargo fmt --check, pixi run typos, check-into-diagnostic.

Checklist:

  • I've discussed the proposed changes with a maintainer.
  • I understand the purpose of my changes and how they interact with the greater system.
  • User-facing changes are documented and prose in the documentation has been written/edited/reviewed by hand.

🤖 Generated with Claude Code

@Tobias-Fischer

Copy link
Copy Markdown
Contributor Author

Merge order for the pixi-build-ros RoboStack PRs. GitHub's stacked PRs don't support fork branches, so they're plain PRs with the dependencies below:

  1. feat(pixi-build-ros): RoboStack's rosdep mappings per distribution, one lookup shared with RoboStack #7533 (shared rosdep lookup) and feat(pixi-build-ros): name packages as RoboStack does #7539 (names): independent, either order
  2. feat(pixi-build-ros): build with RoboStack's build scripts #7540 (build scripts): needs feat(pixi-build-ros): name packages as RoboStack does #7539; its first commit is feat(pixi-build-ros): name packages as RoboStack does #7539
  3. feat(pixi-build-ros): make requirements by RoboStack's rules #7541 (requirement rules): needs feat(pixi-build-ros): build with RoboStack's build scripts #7540 and feat(pixi-build-ros): RoboStack's rosdep mappings per distribution, one lookup shared with RoboStack #7533; only its last commit is new

Already merged: #7529 (licenses), #7534 (SPDX fix). Once a PR merges, I'll rebase the next one so its diff shows only its own changes.

— Claude (on behalf of @Tobias-Fischer)

@Tobias-Fischer
Tobias-Fischer force-pushed the pixi-build-ros-requirements branch 4 times, most recently from 39666d1 to f337c57 Compare November 14, 2026 06:02
…ne lookup shared with RoboStack

The bundled robostack.yaml was a hand-maintained copy of RoboStack's mappings that
had drifted (64 keys missing, 51 mapped differently, the old REQUIRE_GL /
REQUIRE_OPENGL markers). RoboStack's own mappings are now the source, and
RoboStack's recipe generator (vinca-rs) uses the same lookup:
- robostack.yaml (every distribution) and robostack_distros.yaml (each
  distribution's own mappings over it: humble, jazzy, kilted and noetic keep Qt 5)
  are copied from RoboStack/robostack, verbatim (dprint leaves them alone).
- package_map::lookup() says what a rosdep key maps to on a platform; platform
  keys can be a subdir (linux-aarch64) or an OS, and the subdir wins. A platform
  without a value maps to nothing. bundled_package_map(distro) gives a
  distribution's mappings; mapping paths and package templates are resolved.
- A mapping file that can't be read or parsed is an error; a package.xml version
  constraint applies to a mapping's first package (the others are companions);
  REQUIRE_GL / REQUIRE_OPENGL handling is removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Tobias-Fischer
Tobias-Fischer force-pushed the pixi-build-ros-requirements branch from f337c57 to cc3554a Compare November 14, 2026 07:51
Tobias-Fischer and others added 5 commits November 14, 2026 22:09
RoboStack's canonical names for ROS 2 are `ros2-<name>`; `ros-<distro>-<name>`
are only compatibility aliases. The backend now uses them for the package it
builds, for unmapped and `ros:`-mapped dependencies, and for workspace siblings
(`Distro::conda_name`). ROS 1 keeps `ros-<distro>-<name>`: robostack-noetic
only publishes those names.

ROS 2 names no longer say which distribution a package belongs to, so the
distro mutex in host and run is pinned to it: `ros2-distro-mutex * <distro>_*`
(`Distro::mutex_spec`; RoboStack builds the mutex with the build string
`<distro>_<build number>`). Merging that with a mutex the user declares (e.g.
`0.5.*`) needs merged specs to keep a build string: `merge_specs` now merges the
versions and keeps the build string of either side (conflicting build strings
are an error).

The docs say how to migrate a `pixi.toml` that names ROS 2 path dependencies
`ros-<distro>-<name>`. The integration tests and the docs' example workspace use
the new names, and the test workspace requires macOS 14: RoboStack's `ros2-*`
builds need `__osx >=14` (the old names still resolved to older builds).

Snapshot changes:
- the jazzy inline snapshots (generate_recipe, condition_evaluation_*) name
  everything `ros2-<name>` instead of `ros-jazzy-<name>`, and the mutex is
  `ros2-distro-mutex * jazzy_*`
- in the noetic .snap files `ros-distro-mutex` is `ros-distro-mutex * noetic_*`
  in host and run, and in generate_recipe_with_mutex_version the user's `0.5.*`
  merges into `ros-distro-mutex 0.5.* noetic_*`

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The backend's templates were an old copy of RoboStack's, which have since
been fixed while building every package of RoboStack's distributions on five
platforms. Adopt those (they live here now, and RoboStack's recipe generator
renders them through this crate): among others the cross-compilation fixups,
the real `ROS_PYTHON_VERSION` (the old script exported `PY_VER=3`), conda's
macOS deployment target instead of a hardcoded 11.0, a `PYTHON_INSTALL_DIR`
that works without GNU realpath, short object paths on Windows
(`CMAKE_INTERMEDIATE_DIR_STRATEGY=SHORT`, so every Windows template builds with
Ninja; RoboStack's ament_cmake one had switched to Visual Studio for MSVC's
path limit).

ROS 2 cmake/catkin packages used the ROS 1 catkin template; they now share the
ament_cmake one, whose copies had drifted apart. The shared template takes
the ament_cmake settings for both (`BUILD_TESTING=OFF`, a `PYTHON_INSTALL_DIR`
relative to the prefix, as ament_cmake_python expects, and on Windows
`ROS_BUILD_SHARED_LIBS`/`NOGDI` for both) plus conda's `${CMAKE_ARGS}`, which
only the catkin-derived one passed. Dropped: `BOOST_ERROR_CODE_HEADER_ONLY` (a
no-op since Boost 1.69), `CMAKE_FIND_FRAMEWORK=LAST` (in conda's
`${CMAKE_ARGS}` on macOS), an always-empty `CMAKE_AUTOMOC_EXECUTABLE` and
`CMAKE_NO_SYSTEM_FROM_IMPORTED`. The catkin templates use conda's macOS
deployment target too.

Special cases by package name are gone where they are no longer needed: the
emscripten source directories (vinca-rs puts an additional recipe's folder in
`source_dir`), eigenpy's single-job Windows build (from 2021's 7 GB runners),
and the activation scripts of ros_workspace / catkin, which are now whatever
activation scripts a recipe ships. ROS 1's catkin cookie, `environment`
profile scripts and Makefile generator for euslisp stay.

What differs between `pixi build` and a RoboStack recipe is a parameter:
- `ScriptVars::source_dir`: the manifest directory here, the checked-out source
  in a recipe (the templates no longer set `SRC_DIR`)
- `BuildMode::Incremental` keeps the build directory (the configure step is
  incremental too, so the `build.ninja` guard is gone) and lists what the
  build installed in `$RATTLER_BUILD_PACKAGE_FILES`, as before;
  `BuildMode::Clean` is RoboStack's fresh build
- `ScriptVars::generator`: the scripts' first line names what wrote them
  (`pixi-build-ros` here)

The catkin templates stay for ROS 1, with `@(package_prefix)-` instead of the
hardcoded `ros-noetic-` names. `BuildType::of` decides the template (no build
type plus an ament_cmake build tool is ament_cmake; catkin/cmake is the cmake
template in ROS 2 and the catkin one in ROS 1).

The templates put the source directory in double quotes, and `pixi build`
escapes the manifest directory for them (`escape_quoted`), so a path with spaces,
`$` or `%` works. Which template a package gets follows the host platform's
operating system (it was the machine generating the recipe, which made the
recipe depend on where it was generated; `pixi build` does not cross-compile
between operating systems). The gcc/g++ links on Linux are forced, so a
rebuild in the same build prefix works. The templates are left out of shfmt:
their `@[...]` line prefixes are not shell.

Tests: `test_every_template_is_filled` renders every template in both modes
for Unix and Windows; `test_build_type` covers the template choice;
`test_escape_quoted` the escaping.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…brush on Windows

The ROS 2 templates were a bash script and a batch file per build type, kept in
sync by hand. Now each build type has one bash script for every platform: on
Windows the recipe runs it with brush (https://github.com/reubeno/brush, a
bash-compatible shell; rattler-build runs the compiler activation with cmd.exe
first), and `brush` is a build requirement there (`script_interpreter`). ROS 1's
catkin template keeps its batch file (`is_batch`).

The Windows differences are branches in the scripts: the `%LIBRARY_PREFIX%`
layout, MSVC (`CC`/`CXX=cl.exe`, `CL` for `ROS_BUILD_SHARED_LIBS`/`NOGDI`), short
object paths, and on Windows the ament_python script installs with `setup.py`
into `%LIBRARY_PREFIX%`. brush has no coreutils builtins, so file operations use
`cmake -E` and bash builtins, and `PYTHON_INSTALL_DIR` is computed with Python's
`os.path.relpath` on every platform. brush 0.4.0 exits a script when an
assignment follows a failed condition under `-e` and `-x` (fixed upstream in
reubeno/brush#1248, not yet released), so the scripts turn off `-x` under brush.

The brush requirement and the interpreter follow the same platform as the
template, the host platform's operating system.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	Cargo.lock
#	crates/pixi_build_ros/src/package_map.rs
`requirements::Rules` holds RoboStack's requirement rules, which this backend
and RoboStack's recipe generator now share:
- `Rules::requirements`: what every package builds with (compilers, the C
  stdlib, cython, the build tools, pkg-config/python/numpy and cross-python only
  when cross-compiling, clang-tools on macOS), Python in host and run, the
  pinned distro mutex, ros_environment/ros_workspace (except for the packages
  they are built from), and the package.xml dependencies: ROS and system
  dependencies in host (a format 1 `run_depend` too), build tools in host except
  git, which goes to build; run dependencies in run. How a dependency becomes
  conda specs is the caller's: here the package map, keeping package.xml's
  versions
- `Rules::finish`: build-time-only Python for a package without Python
  (RoboStack's opt-in conditional Python; off here), the cross-compiling and
  Emscripten rules (cmake, mimick_vendor, rosidl generators, pybind11, qt-main,
  build-machine tools native/cross), host and run sorted
- `Reqs`/`Req`: the requirement lists, with `if: build_platform != ...`
  entries; `package_xml_to_conda_requirements` turns them into the recipe's

Behaviour changes for `pixi build`: ROS and system dependencies are no longer
also in `build` (they are host dependencies; this is what makes cross-compiling
work), `stdlib('c')` is added when the variants have `c_stdlib`, `cpython` was
a typo for `cython`, setuptools only for ament_python, pkg-config/python/numpy
in build only when cross-compiling, and ros_workspace is a run dependency.

Snapshot changes, all from the above:
- build: only the base requirements (compilers first, `cython` instead of
  `cpython`, the cross-only pkg-config/python/numpy); no package.xml dependencies
- host: sorted; python/numpy/pip, `pkg-config` only when native, no
  setuptools, ros2-ros-environment for ROS 2
- run: sorted; python, and ros2-ros-workspace for ROS 2
- the package_xml_to_recipe_config tests and the condition-evaluation and
  workspace-discovery tests look at host instead of build

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Tobias-Fischer
Tobias-Fischer force-pushed the pixi-build-ros-requirements branch from cc3554a to c3e7749 Compare November 14, 2026 12:15
@Tobias-Fischer

Copy link
Copy Markdown
Contributor Author

@Hofer-Julian this is ready for review

@Tobias-Fischer
Tobias-Fischer marked this pull request as ready for review November 17, 2026 07:32
Tobias-Fischer and others added 2 commits November 17, 2026 20:10
…mplates

A Windows checkout with autocrlf embedded them with CRLF, which brush
rejects (syntax error), failing every ROS build on Windows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The build environment no longer has ROS packages to constrain Python, and
cross-python pins the host Python to the build one, so it floated to 3.15
while robostack-humble is built for 3.14.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant