Switch flate2 backend from zlib-ng to zlib-rs - #48
Merged
Conversation
flate2's zlib-ng feature pulls in libz-ng-sys, which compiles zlib-ng's C source via CMake. On BinaryBuilder's aarch64-apple-darwin toolchain (Clang 22), this fails: zlib-ng's CMake ARMv8 CRC32 feature-detection incorrectly concludes the compiler lacks native __crc32b/__crc32h/ __crc32w intrinsics, so it includes its own polyfill header (acle_intrins.h) -- which then collides with Clang's own arm_acle.h, which already defines them, producing "redefinition of '__crc32b'" compile errors. zlib-rs (a pure-Rust zlib reimplementation, no C compiler needed) is already used elsewhere in this same dependency graph -- parquet (via iceberg-rust, a downstream consumer of this crate through iceberg_rust_ffi) enables flate2's zlib-rs feature. Before this change both backends were being compiled into the same binary simultaneously, which flate2's own documentation flags as producing undefined ordering behavior when more than one backend is enabled at once. Switching to zlib-rs here removes the C dependency entirely and unifies on the backend already in use. `cargo update -p flate2` was needed since the lockfile still had the pre-zlib-rs 1.0.28 release pinned; it resolved to 1.1.10, dropping `cmake` and `libz-ng-sys` from the dependency tree and adding `zlib-rs`, `miniz_oxide`, `adler2`, and `simd-adler32` (all pure Rust). Verified: `cargo test --lib` (37 passed), `cargo build --release`, both clean. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
3 tasks
gbrgr
added a commit
to RelationalAI/RustyIceberg.jl
that referenced
this pull request
Aug 28, 2026
## Summary - Points at [RelationalAI/object_store_ffi@2031546](RelationalAI/object_store_ffi@2031546) ([object_store_ffi#48](RelationalAI/object_store_ffi#48), still open as of this PR), which switches `object_store_ffi`'s `flate2` dependency from the `zlib-ng` backend to `zlib-rs` (pure Rust, no C compiler needed). - `zlib-ng` was failing to cross-compile on BinaryBuilder's `aarch64-apple-darwin` toolchain: `zlib-ng`'s CMake ARMv8 CRC32 feature-detection incorrectly concluded Clang 22 lacks native `__crc32b`/`__crc32h`/`__crc32w` intrinsics, so it included its own polyfill header, which then collided with Clang's own `arm_acle.h` (which already defines them) — `redefinition of '__crc32b'` errors. Found while bumping `iceberg_rust_ffi_jll` in Yggdrasil ([JuliaPackaging/Yggdrasil#14575](JuliaPackaging/Yggdrasil#14575)). - This also removes a latent issue: `parquet` (via `iceberg-rust`) already uses `flate2`'s `zlib-rs` backend, so both `zlib-ng` and `zlib-rs` were being compiled into the same binary simultaneously — `flate2`'s own docs flag enabling more than one backend at once as producing undefined ordering behavior. - Bumps `iceberg_rust_ffi`'s own version 0.9.1 → 0.9.2 (its `Cargo.toml` changed) and `Project.toml` to match.⚠️ This PR's `rev` will need to be updated to the merge commit once `object_store_ffi#48` lands on its `main`, same as the pattern used for the `iceberg-rust` bump in #110. ## Test plan - [x] `cargo test` (iceberg_rust_ffi) — 40 passed, 0 failed - [x] `cargo clippy --all-targets -- -D warnings` / `cargo fmt --all -- --check` — clean - [x] `make test-dev` — **27935 passed, 0 failed, 0 errored**, fully green, including the GZIP compression test (exercising the new `zlib-rs` backend path) Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
flate2'szlib-ngfeature pulls inlibz-ng-sys, which compileszlib-ng's C source via CMake. On BinaryBuilder'saarch64-apple-darwintoolchain (Clang 22), this fails:zlib-ng's CMake ARMv8 CRC32 feature-detection incorrectly concludes the compiler lacks native__crc32b/__crc32h/__crc32wintrinsics, so it includes its own polyfill header (acle_intrins.h) — which then collides with Clang's ownarm_acle.h(which already defines them), producingredefinition of '__crc32b'compile errors. Found while bumpingiceberg_rust_ffi_jllin Yggdrasil (see JuliaPackaging/Yggdrasil#14575).zlib-rs(a pure-Rust zlib reimplementation, no C compiler needed) is already used elsewhere in the same dependency graph —parquet(viaiceberg-rust, a downstream consumer of this crate throughiceberg_rust_ffi) enablesflate2'szlib-rsfeature. Before this change, both backends were being compiled into the same binary simultaneously, whichflate2's own documentation flags as producing undefined ordering behavior when more than one backend is enabled at once. Switching tozlib-rshere removes the C dependency entirely and unifies on the backend already in use elsewhere in the tree.cargo update -p flate2was needed since the lockfile still had the pre-zlib-rs1.0.28release pinned; it resolved to1.1.10, droppingcmakeandlibz-ng-sysfrom the dependency tree and addingzlib-rs,miniz_oxide,adler2, andsimd-adler32(all pure Rust).Test plan
cargo test --lib— 37 passed, 0 failedcargo build --release— clean (only pre-existing, unrelated warnings)cargo check --all-targets— cleanCo-Authored-By: Claude Sonnet 5 noreply@anthropic.com