Repository navigation
ci: build protobuf from source for the macOS wheels - #50
Conversation
The macOS wheel's support floor was set by the runner image: Homebrew's protobuf/abseil bottles are built for the runner's macOS, and delocate refuses a dylib whose minimum target is newer than the wheel's. That broke the wheel when macos-latest moved to macOS 26 (#18), and #19 fixed it by pinning the runner to macos-15. Build protobuf and abseil from source at MACOSX_DEPLOYMENT_TARGET and link them statically, as the Linux leg already does. delocate then has no dylib to check, and the runners go back to macos-latest. The Linux script becomes scripts/cibw_install_protobuf.sh for both OSes. macOS needs three differences, each found by building on macOS 27: - protobuf v36.2, not v25.3. v25.3 bundles abseil LTS 20230802, which passes "-Xarch_x86_64 -msse4.1" as separate options; CMake drops the repeated -Xarch_x86_64 and clang refuses the bare -msse4.1 on arm64. Abseil 20240722 fixed it. v36.2 is what Homebrew ships, so the macOS test legs already build against it. - protobuf_FORCE_FETCH_DEPENDENCIES=ON, so abseil is never Homebrew's dylib. - CMAKE_CXX_STANDARD=17. Abseil pins absl::string_view to the std type only if a test compile at the default standard is C++17; Apple clang's default is older, and protowire-cpp then fails to compile. Linux stays on v25.3: the PR's checks build only its x86_64 leg, not the aarch64 one the release builds. Fixes #20
The smoke log showed the extension built as macosx_11_0 (cibuildwheel's default for arm64) and retagged macosx_15_0 by delocate only because the linked protobuf code targets 15.0. Main's last run shows the same. The floor came out right, but by way of delocate rather than the setting. Pass MACOSX_DEPLOYMENT_TARGET through CIBW_ENVIRONMENT_MACOS so the build itself targets it.
This reverts commit c0002de. The cause that commit gave was wrong. cibuildwheel already passes the job's MACOSX_DEPLOYMENT_TARGET to the build (env.setdefault, cibuildwheel/platforms/macos.py:360 at v4.3.5). The macosx_11_0 name in the log comes from _PYTHON_HOST_PLATFORM, which cibuildwheel sets to macosx-11.0-arm64 for every arm64 build (line 372). scikit-build-core names the wheel from it, and delocate renames it from the binaries' real target. The re-run with the change still showed "Created ... macosx_11_0", so it changed nothing.
|
CI result at A detour, now reverted. The log also showed the wheel first created as |
Fixes #20.
What changes
The macOS wheel's support floor (macOS 15.0) no longer depends on which macOS the CI runner has. protobuf and abseil are now built from source at
MACOSX_DEPLOYMENT_TARGETand linked statically, so delocate has no Homebrew dylib to compare against the wheel's target. The wheel-building runners go back tomacos-latestinci.ymlandpublish.yml. The floor itself is unchanged.scripts/cibw_install_protobuf_linux.shbecomesscripts/cibw_install_protobuf.sh, used by both OSes. On macOS it differs in three ways, each found by building locally:-Xarch_x86_64 -msse4.1as two options, CMake drops the repeated-Xarch_x86_64, and clang refuses the bare-msse4.1. The failing target'sflags.makeshows-Xarch_x86_64 -maes -msse4.1. v36.2 fetches abseil 20250512.1, which has the fix. It's also the version Homebrew ships, so the macOS test legs already build against it.protobuf_FORCE_FETCH_DEPENDENCIES=ON), never taken from Homebrew.CMAKE_CXX_STANDARD=17. Without it the installed abseil uses its ownabsl::string_viewrather thanstd::string_view, and protowire-cpp fails to compile (no viable conversion from 'absl::string_view' to 'std::string_view'inencode.ccandwellknown.cc).The script refuses to run on macOS without
MACOSX_DEPLOYMENT_TARGET, so it can't quietly build for the runner's macOS again.Linux stays on v25.3. This PR's checks build only the Linux x86_64 wheel. The aarch64 one is built only at release, so a version change there couldn't be proven here.
Evidence (local, macOS 27.0.1 arm64, Apple clang 21)
This host's macOS is newer than the 15.0 target, which is the case that broke in #18. It also has Homebrew's protobuf/abseil installed and a stale
/usr/local/include/google/protobuf, a dirtier setup than the runner.MACOSX_DEPLOYMENT_TARGET=15.0: exit 0,libprotoc 36.2. 94 static libraries, 0 dylibs.minos 15.0onprotocand on the sampled objects inlibprotobuf.aand abseil'sranden_hwaes. Log: "Fallback to downloading Abseil 20250512.1".pip wheelwithCMAKE_PREFIX_PATHat that prefix: CMake reportsFound Protobuf: <prefix>/lib/libprotobuf.a. Wheel:protowire_python-1.1.0-cp315-cp315-macosx_15_0_arm64.whl.delocate-wheel --require-archs arm64: exit 0.delocate-listdeps --alllists only system libraries (CoreFoundation, libSystem, libc++, libz).otool -lon_protowire.so:minos 15.0.pytest tests: 200 passed. A probe test printed_protowireloading from the venv's site-packages, not fromsrc/.Not yet shown: the
macos-latestcibuildwheel leg on GitHub, the last done-when item of #20. This PR'scibuildwheel smoke / macos-latestjob is that check.