Skip to content

[VL][Iceberg] Support MERGE INTO - #12964

Open
infvg wants to merge 1 commit into
apache:mainfrom
infvg:feature/iceberg-merge-into
Open

infvg wants to merge 1 commit into
apache:mainfrom
infvg:feature/iceberg-merge-into

Conversation

@infvg

@infvg infvg commented Sep 3, 2026 •

Copy link
Copy Markdown
Contributor

Enable native scans and writes for Iceberg MERGE INTO in copy-on-write mode.

Iceberg uses SparkCopyOnWriteScan and file-path/row-position metadata for these operations. The native scan path previously did not support this scan type or handle the required metadata.

This PR:

  • Supports SparkCopyOnWriteScan in Iceberg scan detection, table lookup, and scan-task extraction.
  • Maps copy-on-write file-path and row-position columns to native metadata and row-index columns.
  • Retrieves the underlying Iceberg table from the scan so validation works when row-level operations wrap the Spark table.
  • Extends row-index detection to include Iceberg row-position columns and columns at schema index zero, while preserving Spark’s schema validation and backend capability checks.

@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@github-actions github-actions Bot added CORE works for Gluten Core VELOX DATA_LAKE labels Sep 3, 2026
@infvg
infvg force-pushed the feature/iceberg-merge-into branch from 5f3ae2c to 8dc6d45 Compare September 15, 2026 13:45
@github-actions

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@infvg
infvg force-pushed the feature/iceberg-merge-into branch from 8dc6d45 to fc68bac Compare September 16, 2026 08:25
@github-actions

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@infvg
infvg marked this pull request as ready for review September 16, 2026 10:56
@infvg
infvg force-pushed the feature/iceberg-merge-into branch from fc68bac to 00efdbc Compare September 16, 2026 11:42
@github-actions

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@zhouyuan

Copy link
Copy Markdown
Member

Cc: @wangyum

@wangyum

wangyum commented Sep 18, 2026

Copy link
Copy Markdown
Member

Will test and verify it next week.

@infvg

infvg commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

Hi @wangyum, could you please take a look whenever you can?

@jinchengchenghh

Copy link
Copy Markdown
Contributor

Please write the PR description

@infvg
infvg force-pushed the feature/iceberg-merge-into branch from 00efdbc to a82be99 Compare October 15, 2026 14:49
@github-actions github-actions Bot removed the VELOX label Oct 15, 2026
@github-actions

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CORE works for Gluten Core DATA_LAKE

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants