Skip to content

perf(parquet): prune IN predicates with page stats - #704

Merged
JingsongLi merged 1 commit into
apache:mainfrom
QuakeWang:in-stats-check
Aug 12, 2026
Merged

perf(parquet): prune IN predicates with page stats#704
JingsongLi merged 1 commit into
apache:mainfrom
QuakeWang:in-stats-check

Conversation

@QuakeWang

Copy link
Copy Markdown
Member

Purpose

Parquet page pruning reused the shared statistics evaluator, but ParquetPageStats did not enable min/max pruning for IN. As a result, page indexes were loaded for IN predicates while non-matching pages were still retained.

Brief change log

  • Enable min/max pruning for IN predicates using Parquet page statistics.
  • Add a multi-page regression test verifying that only the matching page is retained.

Tests

  • cargo check
  • cargo fmt --all -- --check
  • cargo build
  • cargo clippy --locked --all-targets --workspace --features fulltext,vortex -- -D warnings
  • cargo test -p paimon --all-targets --features fulltext,vortex
  • cargo test -p paimon-rest-server --all-targets

API and Format

Documentation

Signed-off-by: QuakeWang <wangfuzheng0814@foxmail.com>

@JingsongLi JingsongLi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@JingsongLi
JingsongLi merged commit 85c9127 into apache:main Aug 12, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants