Repository navigation
fix: preserve supplied VALUES field metadata [skip ci] - #174
Merged
osipovartem merged 1 commit intoNov 1, 2026
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Narrow runtime-branch backport of metadata preservation already present upstream in Apache DataFusion apache#21714 (commit
49c67c296b9b43d98768fcd1592c6ee1bcde4dfb). No duplicate upstream contribution is needed.Rationale for this change
LogicalPlanBuilder::values_with_schemacurrently drops the supplied fields' metadata. This loses logical annotations such as canonical timestamp-with-offset and UUID even though the requested storage type and nullability remain intact. Downstream validation can consequently see a generic struct rather than its declared extension type.What changes are included in this PR?
FieldMetadata::new_from_fieldAPI.ValuesFields::pushpath for empty metadata. Metadata work is per annotated column, not per row.Are these changes tested?
cargo clippy --all-targets --all-features -- -D warningspassed with the locked offline CI profile.dev/rust_lint.shsuite passed, including formatting, strict workspace clippy, TOML, licenses, spelling, Markdown and strict rustdoc. The unrelated preexisting configuration-table formatting repair is isolated in chore: format runtime configuration table [skip ci] #173.These are medians of per-process medians in the CI profile, not release or end-to-end SQL/HTTP/Iceberg/SPCS measurements. Both equivalent-output empty controls meet the predeclared 5% median budget. Individual adverse pairs remain visible: the one-row empty control ranges from +0.76% to +65.69%, so no zero-overhead or statistical non-regression claim is made. Annotated baseline output is incorrect; those rows measure the cost of retaining requested metadata, not an equivalent-output speedup. The added annotation work is proportional to annotated columns/metadata bytes, not VALUES row count.
Are there any user-facing changes?
Supplied field annotations now survive VALUES construction as requested. There is no public API signature change. This backport alone does not establish complete Rustice timestamp alias, storage, CTAS or wire compatibility; those require separate downstream qualification.