refactor: make Iceberg codec fixture options explicit - #703
Merged
alexanderbianchi merged 1 commit intoSep 8, 2026
Conversation
This was referenced Sep 5, 2026
alexanderbianchi
changed the title
alexanderbianchi
force-pushed
the
iceberg/test-storage-options
branch
from
September 5, 2026 20:54
969ad8a to
45193f8
Compare
alexanderbianchi
force-pushed
the
codex/iceberg-runtime-metadata
branch
from
September 5, 2026 21:22
71a906b to
3445b6a
Compare
alexanderbianchi
force-pushed
the
iceberg/test-storage-options
branch
2 times, most recently
from
September 5, 2026 22:57
d9160c1 to
9ceb25f
Compare
alexanderbianchi
force-pushed
the
iceberg/test-storage-options
branch
from
September 6, 2026 00:18
9ceb25f to
069fa19
Compare
alexanderbianchi
force-pushed
the
codex/iceberg-runtime-metadata
branch
from
September 6, 2026 00:18
158065b to
cdf47f7
Compare
alexanderbianchi
marked this pull request as ready for review
September 7, 2026 14:19
alexanderbianchi
merged commit Sep 8, 2026
eafc234
into
datafusion-contrib:codex/iceberg-runtime-metadata
32 checks passed
Collaborator
Author
|
Merging before pulling the fork since it's just a simple few lines of test coverage |
This was referenced Sep 8, 2026
alexanderbianchi
added a commit
that referenced
this pull request
Sep 8, 2026
Reapplies the reviewed and merged changes from #703 to the intended base, `iceberg-0.10`. #703 was still targeting `codex/iceberg-runtime-metadata` when it merged, after #700 had already merged into `iceberg-0.10`. Consequently, its codec-test improvements landed only on the old topic branch. This PR cherry-picks #703's merge commit (`eafc234`) onto `iceberg-0.10`, without additional code changes. It retains the original 15-line diff in `iceberg/src/codec.rs`: explicit storage properties, including a quote-containing value, and assertions that those properties survive the codec round trip. Validation: - `cargo test -p datafusion-distributed-iceberg --locked --lib roundtrips_data_source_plan` — passed. - `cargo fmt --all -- --check` and `git diff --check` passed. Targets `iceberg-0.10` directly; it does not depend on the separate reapplication of #704.
alexanderbianchi
added a commit
that referenced
this pull request
Sep 8, 2026
Reapplies the reviewed and merged changes from #704 to the intended base, `iceberg-0.10`. #704 was still targeting `codex/iceberg-runtime-metadata` when it merged, after #700 had already merged into `iceberg-0.10`. Consequently, the estimator and harness changes landed only on the old topic branch. This PR cherry-picks #704's merge commit (`d3ba554`) onto `iceberg-0.10`. There are no additional code changes; the resulting tree is identical to #704's final head (`12e1512`). Includes the source-statistics-based task estimator, explicit estimation tests, distributed execution coverage, and the private-context harness with `configure_session(...)`. Validation: - `cargo test -p datafusion-distributed-iceberg --features integration --locked --test desired_task_count` — all 9 tests passed. - `cargo fmt --all -- --check` and `git diff --check` passed. - Verified exact tree equality with #704's final head. Targets `iceberg-0.10` directly; it does not depend on the separate reapplication of #703.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Extends
roundtrips_data_source_plan, the existing codec test from #684, with two explicit storage properties supplied through #700's harness builder. Checks their decoded values, including a quote-containing value that exercises SQL literal escaping, while retaining the existing schema, partitioning, fetch, feed, property-map, and statistics assertions.The diff against #700 remains 15 changed lines in
iceberg/src/codec.rs, with no new harness methods or test functions.Stacked on #700, now rebased onto
iceberg-0.10at6507bf8after #687 merged. Only #700 is a prerequisite.Validation:
cargo test -p datafusion-distributed-iceberg --locked— 90 tests passed, including the codec roundtrip test and doctestcargo clippy -p datafusion-distributed-iceberg --tests --locked -- -D warningscargo fmt --all -- --checkgit diff --check