Skip to content

feat(pixi-build-ros): build with RoboStack's build scripts - #7540

Open
Tobias-Fischer wants to merge 4 commits into
prefix-dev:mainfrom
Tobias-Fischer:pixi-build-ros-build-scripts
Open

Tobias-Fischer wants to merge 4 commits into
prefix-dev:mainfrom
Tobias-Fischer:pixi-build-ros-build-scripts

Conversation

@Tobias-Fischer

@Tobias-Fischer Tobias-Fischer commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Description

Part 2 of 3, split out of #7535 for easier review. It builds on #7539 (names; the scripts use its package prefix), so the first commit here is #7539 and the last two are this PR: "build with RoboStack's build scripts" adopts the scripts, and "one bash build script per build type, run with brush on Windows" replaces the Windows batch files. Each can be reviewed on its own. (GitHub's stacked pull requests don't support branches from forks, so this is a plain PR against main.)

Tobi human description

The pixi-build-ros build scripts were taken from a VERY old RoboStack/vinca version. There have been many improvements since in the build scripts, and this PR simply copies over these build scripts so that the new vinca-rs can benefit from it and doesn't need to duplicate build scripts (which is the status quo).

Another cool feature is that we now use brush to unify the build scripts between Linux and Windows, which again streamlines things substantially and will maintenance a lot easier in the future.

Lots of details in the AI summary if needed/wanted.

AI-generated details (Claude Opus 5.5)

The backend's build scripts were an old copy of RoboStack's. RoboStack's have since been fixed while building every package of its distributions on five platforms. This PR adopts them, and RoboStack's recipe generator renders them through this crate, so pixi build and RoboStack's binaries build a package the same way. The old scripts here were missing several of those fixes:

  • they exported PY_VER=3, overriding rattler-build's;
  • they hardcoded ROS_PYTHON_VERSION=3 and macOS 11.0 instead of conda-forge's deployment target;
  • they lacked the cross-compilation fixes;
  • on Windows they had no fix for MSVC's path limit; CMAKE_INTERMEDIATE_DIR_STRATEGY=SHORT (CMake ≥ 4.2) shortens object paths, so every template keeps Ninja.

ROS 2 cmake packages used the ROS 1 catkin script. Now they share the ament_cmake script, whose copies had drifted apart. Both get its settings (BUILD_TESTING=OFF, a PYTHON_INSTALL_DIR relative to the prefix as ament_cmake_python expects, and ROS_BUILD_SHARED_LIBS/NOGDI on Windows) plus conda's ${CMAKE_ARGS}. Settings that did nothing or repeated ${CMAKE_ARGS} are dropped: BOOST_ERROR_CODE_HEADER_ONLY (a no-op since Boost 1.69), CMAKE_FIND_FRAMEWORK=LAST, an always-empty CMAKE_AUTOMOC_EXECUTABLE, and CMAKE_NO_SYSTEM_FROM_IMPORTED. Package-name special cases that are no longer needed are gone: the emscripten source directories, eigenpy's single-job Windows build, and the activation scripts of ros_workspace and catkin, which are now whatever activation scripts a recipe ships. ROS 1 keeps the catkin scripts, which now use conda's macOS deployment target. Built locally on osx-arm64 with the old and the new scripts, ros_workspace, ament_cmake_test, rcutils, builtin_interfaces, fastcdr and ruckig contain the same files and have the same run dependencies.

One script for every platform. The ROS 2 build types have one bash script each, also on Windows: the recipe runs it with brush, a bash-compatible shell, which rattler-build supports as interpreter: brush after setting up the compiler environment with cmd.exe. brush goes into the Windows build requirements (script_interpreter). The Windows differences are branches in the script: the %LIBRARY_PREFIX% layout, MSVC, CL, short object paths. File operations that also run on Windows use cmake -E. brush 0.4.0 exits after an assignment in an else branch when it runs with both -e and -x, so the scripts turn command tracing off under brush (fixed upstream in reubeno/brush#1248, not released yet). The catkin scripts keep their batch file for ROS 1. Locally on osx-arm64, the six packages above and ament_index_python and demo_nodes_py run under brush give the same packages as under bash.

What differs between pixi build and a RoboStack recipe is a parameter:

  • ScriptVars holds the source directory, the package prefix, the distro, extra CMake arguments, and the name of the tool that wrote the script, which goes on its first line.
  • BuildMode::Incremental is what pixi build uses: the build directory is kept and the installed files are recorded in $RATTLER_BUILD_PACKAGE_FILES, as before. CMake now reconfigures on every build, which is fast on a kept directory. BuildMode::Clean is RoboStack's fresh build.

Kept for RoboStack and inert for pixi build: the opt-in VINCA_USE_SCCACHE and VINCA_CUSTOM_CMAKE_BUILD_DIR environment variables, which RoboStack's CI sets.

Also fixed along the way:

  • The templates put the source directory in double quotes, and pixi build escapes the manifest path for them, so paths with spaces, $ or % work.

  • Which script a package gets (and brush) follows the host platform's operating system, not the machine that generates the recipe, so a recipe no longer depends on where it was generated.

  • The gcc/g++ links on Linux are forced, so a rebuild in the kept build prefix works.

  • The templates are excluded from shfmt: their @[...] line prefixes are not shell.

  • Tests: test_every_template_is_filled renders every template in both modes for Unix and Windows and checks that no placeholder is left. test_build_type covers which template is chosen. There are no snapshots of the rendered scripts; they would repeat the templates.

  • ros_workspace's activation scripts, which feat(pixi-build-ros): RoboStack's build scripts, requirement rules and package names #7535 also moved here, stay in RoboStack's generator because pixi build doesn't use them.

Checklist:

  • I've discussed the proposed changes with a maintainer.
  • I understand the purpose of my changes and how they interact with the greater system.
  • User-facing changes are documented and prose in the documentation has been written/edited/reviewed by hand.

🤖 Generated with Claude Code

@Tobias-Fischer

Copy link
Copy Markdown
Contributor Author

Merge order for the pixi-build-ros RoboStack PRs. GitHub's stacked PRs don't support fork branches, so they're plain PRs with the dependencies below:

  1. feat(pixi-build-ros): RoboStack's rosdep mappings per distribution, one lookup shared with RoboStack #7533 (shared rosdep lookup) and feat(pixi-build-ros): name packages as RoboStack does #7539 (names): independent, either order
  2. feat(pixi-build-ros): build with RoboStack's build scripts #7540 (build scripts): needs feat(pixi-build-ros): name packages as RoboStack does #7539; its first commit is feat(pixi-build-ros): name packages as RoboStack does #7539
  3. feat(pixi-build-ros): make requirements by RoboStack's rules #7541 (requirement rules): needs feat(pixi-build-ros): build with RoboStack's build scripts #7540 and feat(pixi-build-ros): RoboStack's rosdep mappings per distribution, one lookup shared with RoboStack #7533; only its last commit is new

Already merged: #7529 (licenses), #7534 (SPDX fix). Once a PR merges, I'll rebase the next one so its diff shows only its own changes.

— Claude (on behalf of @Tobias-Fischer)

@traversaro

Copy link
Copy Markdown
Contributor

on Windows they used Ninja, which fails on long paths; Visual Studio is used instead.

That seems a regression, ninja has many advantages w.r.t. visual studio. To avoid long paths failures we could try https://www.scivision.dev/cmake-short-build-path/ and keep ninja?

@Tobias-Fischer
Tobias-Fischer force-pushed the pixi-build-ros-build-scripts branch 2 times, most recently from f3c9d65 to 48847b4 Compare November 14, 2026 00:34
@Tobias-Fischer

Copy link
Copy Markdown
Contributor Author

That seems a regression, ninja has many advantages w.r.t. visual studio. To avoid long paths failures we could try https://www.scivision.dev/cmake-short-build-path/ and keep ninja?

Good point, thanks. pixi's template used Ninja, and the template already passes -DCMAKE_INTERMEDIATE_DIR_STRATEGY=SHORT (CMake ≥ 4.2, supported by the Ninja generator), the fix that article describes. RoboStack's bld_cmake.bat already builds with Ninja and that setting; only bld_ament_cmake.bat still used the Visual Studio workaround. I switched it back to Ninja (and dropped the --parallel %CPU_COUNT% that was only needed without Ninja), and corrected the PR description. We'll confirm it with a RoboStack Windows build of the long-named ament_cmake packages.

— Claude (on behalf of @Tobias-Fischer)

Tobias-Fischer added a commit to Tobias-Fischer/robostack-fork that referenced this pull request Nov 14, 2026
…ges on Windows

Forces a rebuild of builtin_interfaces (where the long-path failure appeared in
RoboStack/ros-humble#229), control_msgs, nav2_msgs, rcl_interfaces and
rosbag2_interfaces, to check bld_ament_cmake.bat with Ninja and
CMAKE_INTERMEDIATE_DIR_STRATEGY=SHORT (prefix-dev/pixi#7540 review).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Tobias-Fischer

Copy link
Copy Markdown
Contributor Author

Follow-up: a RoboStack Windows build confirms Ninja works here. With bld_ament_cmake.bat from this PR (-G "Ninja" and CMAKE_INTERMEDIATE_DIR_STRATEGY=SHORT), five long-named humble interface packages were rebuilt on win-64, including builtin_interfaces, whose generated Python extension hit MSVC's path limit in RoboStack/ros-humble#229. All built, with no C1083 errors: https://github.com/RoboStack/robostack/actions/runs/43545144727

— Claude (on behalf of @Tobias-Fischer)

@Tobias-Fischer

Copy link
Copy Markdown
Contributor Author

So it was just trying to be a good agent .. vinca has VS currently: https://github.com/RoboStack/vinca/blob/18ac8f5fc252bc819efc55633b8ac6e39078dd0a/vinca/templates/bld_ament_cmake.bat.in#L27

anyhow, fixed now :)

@Tobias-Fischer
Tobias-Fischer force-pushed the pixi-build-ros-build-scripts branch 2 times, most recently from 823b80c to 15a59d8 Compare November 14, 2026 06:02
Tobias-Fischer added a commit to Tobias-Fischer/robostack-fork that referenced this pull request Nov 14, 2026
vinca-rs from its templates-check branch (pixi-build-ros at prefix-dev/pixi#7540's
unified ament_cmake/cmake templates, built from source), and rebuilds of
ros_workspace, ament_cmake_test, rcutils, builtin_interfaces, fastcdr and ruckig
on every platform.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tobias-Fischer added a commit to Tobias-Fischer/robostack-fork that referenced this pull request Nov 14, 2026
vinca-rs from its templates-check branch (pixi-build-ros at prefix-dev/pixi#7540's
unified ament_cmake/cmake templates, built from source), and rebuilds of
ros_workspace, ament_cmake_test, rcutils, builtin_interfaces, fastcdr and ruckig
on every platform.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Tobias-Fischer
Tobias-Fischer force-pushed the pixi-build-ros-build-scripts branch from 15a59d8 to 140a142 Compare November 14, 2026 07:51
Tobias-Fischer added a commit to Tobias-Fischer/vinca-rs that referenced this pull request Nov 14, 2026
pixi-build-ros (prefix-dev/pixi#7540) now has one template per ROS 2 build type
for every platform: ament_cmake and cmake share one, with ${CMAKE_ARGS}, and on
Windows the recipe runs the bash script with brush (`script: {file, interpreter:
brush}`, brush in the build requirements). The source directory is in bash syntax
on Windows too, except for ROS 1's catkin batch file. A recipe whose script is a
mapping counts as having a build script in the workflow.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Tobias-Fischer

Copy link
Copy Markdown
Contributor Author

Results for the latest push, where ament_cmake and cmake share one template and Windows runs the ROS 2 bash scripts with brush:

  • CI on all five platforms (RoboStack/robostack run 43573597330, from One bash build script per build type, run with brush on Windows RoboStack/robostack#225): ros_workspace, ament_cmake_test, rcutils, builtin_interfaces, fastcdr, ruckig, ament_index_python and demo_nodes_py were rebuilt and tested on linux-64, linux-aarch64, osx-64, osx-arm64 and win-64, along with about 25 other humble recipes in the same jobs. No CMake or MSVC errors.
    • On Windows the bash script ran under brush: the Python modules land in Lib/site-packages (PYTHON_INSTALL_DIR=../Lib/site-packages), and ros_workspace installs its .bat/.ps1/.sh activation scripts.
  • Locally on osx-arm64, I built the same eight packages with the previous scripts, with the new scripts under bash, and with the new scripts under brush. They contain the same files at the same sizes and have the same run dependencies.
  • A brush 0.4.0 bug turned up: with both -e and -x set (rattler-build runs brush with -euxo pipefail), an assignment in an else branch after a failed condition ends the script. The scripts turn command tracing off under brush (set +x when BRUSH_VERSION is set) until that's fixed upstream.

— Claude

Tobias-Fischer added a commit to RoboStack/robostack that referenced this pull request Nov 14, 2026
vinca-rs with prefix-dev/pixi#7540's build scripts: ament_cmake and cmake share
one template (with conda's ${CMAKE_ARGS}, BUILD_TESTING=OFF and a
PYTHON_INSTALL_DIR relative to the prefix for both), and the ROS 2 scripts are
bash scripts on every platform, run with brush on Windows. rtabmap's Windows link
flags are in bash syntax ($LIBRARY_LIB).

vinca-rs comes from its branch (a pixi source dependency) until 0.1.10 is released.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tobias-Fischer added a commit to RoboStack/robostack that referenced this pull request Nov 14, 2026
vinca-rs with prefix-dev/pixi#7540's build scripts: ament_cmake and cmake share
one template (with conda's ${CMAKE_ARGS}, BUILD_TESTING=OFF and a
PYTHON_INSTALL_DIR relative to the prefix for both), and the ROS 2 scripts are
bash scripts on every platform, run with brush on Windows. rtabmap's Windows link
flags are in bash syntax ($LIBRARY_LIB).

vinca-rs comes from its branch (a pixi source dependency) until 0.1.10 is released.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Tobias-Fischer
Tobias-Fischer force-pushed the pixi-build-ros-build-scripts branch from 140a142 to 6506d3a Compare November 14, 2026 12:15
Tobias-Fischer added a commit to Tobias-Fischer/vinca-rs that referenced this pull request Nov 14, 2026
pixi-build-ros (prefix-dev/pixi#7540) now has one template per ROS 2 build type
for every platform: ament_cmake and cmake share one, with ${CMAKE_ARGS}, and on
Windows the recipe runs the bash script with brush (`script: {file, interpreter:
brush}`, brush in the build requirements). The source directory is in bash syntax
on Windows too, except for ROS 1's catkin batch file (pixi-build-ros's
`is_batch` decides, as for the interpreter); the templates put it in double
quotes, so it has no trailing separator (a batch file's `\"` would not close
them). A recipe whose script is a mapping counts as having a build script in
the workflow.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tobias-Fischer added a commit to RoboStack/robostack that referenced this pull request Nov 14, 2026
vinca-rs with prefix-dev/pixi#7540's build scripts: ament_cmake and cmake share
one template (with conda's ${CMAKE_ARGS}, BUILD_TESTING=OFF and a
PYTHON_INSTALL_DIR relative to the prefix for both), and the ROS 2 scripts are
bash scripts on every platform, run with brush on Windows. rtabmap's Windows link
flags are in bash syntax ($LIBRARY_LIB).

vinca-rs comes from its branch (a pixi source dependency) until 0.1.10 is released.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Tobias-Fischer
Tobias-Fischer marked this pull request as ready for review November 17, 2026 07:21
@Tobias-Fischer

Copy link
Copy Markdown
Contributor Author

@Hofer-Julian this is ready for review (after #7539 )

Tobias-Fischer and others added 3 commits November 20, 2026 19:24
The backend's templates were an old copy of RoboStack's, which have since
been fixed while building every package of RoboStack's distributions on five
platforms. Adopt those (they live here now, and RoboStack's recipe generator
renders them through this crate): among others the cross-compilation fixups,
the real `ROS_PYTHON_VERSION` (the old script exported `PY_VER=3`), conda's
macOS deployment target instead of a hardcoded 11.0, a `PYTHON_INSTALL_DIR`
that works without GNU realpath, short object paths on Windows
(`CMAKE_INTERMEDIATE_DIR_STRATEGY=SHORT`, so every Windows template builds with
Ninja; RoboStack's ament_cmake one had switched to Visual Studio for MSVC's
path limit).

ROS 2 cmake/catkin packages used the ROS 1 catkin template; they now share the
ament_cmake one, whose copies had drifted apart. The shared template takes
the ament_cmake settings for both (`BUILD_TESTING=OFF`, a `PYTHON_INSTALL_DIR`
relative to the prefix, as ament_cmake_python expects, and on Windows
`ROS_BUILD_SHARED_LIBS`/`NOGDI` for both) plus conda's `${CMAKE_ARGS}`, which
only the catkin-derived one passed. Dropped: `BOOST_ERROR_CODE_HEADER_ONLY` (a
no-op since Boost 1.69), `CMAKE_FIND_FRAMEWORK=LAST` (in conda's
`${CMAKE_ARGS}` on macOS), an always-empty `CMAKE_AUTOMOC_EXECUTABLE` and
`CMAKE_NO_SYSTEM_FROM_IMPORTED`. The catkin templates use conda's macOS
deployment target too.

Special cases by package name are gone where they are no longer needed: the
emscripten source directories (vinca-rs puts an additional recipe's folder in
`source_dir`), eigenpy's single-job Windows build (from 2021's 7 GB runners),
and the activation scripts of ros_workspace / catkin, which are now whatever
activation scripts a recipe ships. ROS 1's catkin cookie, `environment`
profile scripts and Makefile generator for euslisp stay.

What differs between `pixi build` and a RoboStack recipe is a parameter:
- `ScriptVars::source_dir`: the manifest directory here, the checked-out source
  in a recipe (the templates no longer set `SRC_DIR`)
- `BuildMode::Incremental` keeps the build directory (the configure step is
  incremental too, so the `build.ninja` guard is gone) and lists what the
  build installed in `$RATTLER_BUILD_PACKAGE_FILES`, as before;
  `BuildMode::Clean` is RoboStack's fresh build
- `ScriptVars::generator`: the scripts' first line names what wrote them
  (`pixi-build-ros` here)

The catkin templates stay for ROS 1, with `@(package_prefix)-` instead of the
hardcoded `ros-noetic-` names. `BuildType::of` decides the template (no build
type plus an ament_cmake build tool is ament_cmake; catkin/cmake is the cmake
template in ROS 2 and the catkin one in ROS 1).

The templates put the source directory in double quotes, and `pixi build`
escapes the manifest directory for them (`escape_quoted`), so a path with spaces,
`$` or `%` works. Which template a package gets follows the host platform's
operating system (it was the machine generating the recipe, which made the
recipe depend on where it was generated; `pixi build` does not cross-compile
between operating systems). The gcc/g++ links on Linux are forced, so a
rebuild in the same build prefix works. The templates are left out of shfmt:
their `@[...]` line prefixes are not shell.

Tests: `test_every_template_is_filled` renders every template in both modes
for Unix and Windows; `test_build_type` covers the template choice;
`test_escape_quoted` the escaping.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…brush on Windows

The ROS 2 templates were a bash script and a batch file per build type, kept in
sync by hand. Now each build type has one bash script for every platform: on
Windows the recipe runs it with brush (https://github.com/reubeno/brush, a
bash-compatible shell; rattler-build runs the compiler activation with cmd.exe
first), and `brush` is a build requirement there (`script_interpreter`). ROS 1's
catkin template keeps its batch file (`is_batch`).

The Windows differences are branches in the scripts: the `%LIBRARY_PREFIX%`
layout, MSVC (`CC`/`CXX=cl.exe`, `CL` for `ROS_BUILD_SHARED_LIBS`/`NOGDI`), short
object paths, and on Windows the ament_python script installs with `setup.py`
into `%LIBRARY_PREFIX%`. brush has no coreutils builtins, so file operations use
`cmake -E` and bash builtins, and `PYTHON_INSTALL_DIR` is computed with Python's
`os.path.relpath` on every platform. brush 0.4.0 exits a script when an
assignment follows a failed condition under `-e` and `-x` (fixed upstream in
reubeno/brush#1248, not yet released), so the scripts turn off `-x` under brush.

The brush requirement and the interpreter follow the same platform as the
template, the host platform's operating system.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…mplates

A Windows checkout with autocrlf embedded them with CRLF, which brush
rejects (syntax error), failing every ROS build on Windows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Tobias-Fischer
Tobias-Fischer force-pushed the pixi-build-ros-build-scripts branch from 2e74ca4 to 3148f5b Compare November 20, 2026 19:26
@Tobias-Fischer

Copy link
Copy Markdown
Contributor Author

@Hofer-Julian - this is ready for review now that #7539 has been merged :)

… scripts

build_catkin.sh and bld_catkin.bat never used @(cmake_args), so a package's
additional CMake arguments (vinca-rs's additional_cmake_args) were dropped for
ROS 1 packages. Add them, and check in the template test that every CMake
build type gets them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 2d90277db29b13ad7638f82ae1978cff79da0ed8)
Tobias-Fischer added a commit to RoboStack/robostack that referenced this pull request Nov 21, 2026
vinca-rs with prefix-dev/pixi#7540's build scripts: ament_cmake and cmake share
one template (with conda's ${CMAKE_ARGS}, BUILD_TESTING=OFF and a
PYTHON_INSTALL_DIR relative to the prefix for both), and the ROS 2 scripts are
bash scripts on every platform, run with brush on Windows. rtabmap's Windows link
flags are in bash syntax ($LIBRARY_LIB).

vinca-rs comes from its branch (a pixi source dependency) until 0.1.10 is released.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tobias-Fischer added a commit to RoboStack/robostack that referenced this pull request Nov 21, 2026
vinca-rs with prefix-dev/pixi#7540's build scripts: ament_cmake and cmake share
one template (with conda's ${CMAKE_ARGS}, BUILD_TESTING=OFF and a
PYTHON_INSTALL_DIR relative to the prefix for both), and the ROS 2 scripts are
bash scripts on every platform, run with brush on Windows. rtabmap's Windows link
flags are in bash syntax ($LIBRARY_LIB).

vinca-rs comes from its branch (a pixi source dependency) until 0.1.10 is released.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants