Skip to content

[GLUTEN-13330][VL] Remap generated build key aliases in the driver-side hash table build - #13332

Open
LuciferYang wants to merge 1 commit into
apache:mainfrom
LuciferYang:gcfix-driver-build-key-alias
Open

LuciferYang wants to merge 1 commit into
apache:mainfrom
LuciferYang:gcfix-driver-build-key-alias

Conversation

@LuciferYang

Copy link
Copy Markdown
Contributor

What changes are proposed in this pull request?

When spark.gluten.sql.columnar.backend.velox.driverSideBroadcastHashTableBuild is enabled, buildHashTableWithRuntime in ColumnarBuildSideRelation and UnsafeColumnarBuildSideRelation builds the hash table on the driver. It names each join key as it appears in the join, but when a key reaches the build side under a generated alias, the build output names that column differently and the native build fails with Field not found: key#329. Available fields are: p_partkey#197, _post_11#329. The failure is caught and the join falls back to building the table on each executor, so results are correct but the driver-side build never takes effect for these joins.

#12407 fixed the same problem in the executor-side buildHashTable by mapping each key to the build output attribute with the same exprId. This applies that mapping in buildHashTableWithRuntime as well.

How was this patch tested?

Added two cases to VeloxHashJoinSuite that run the #12407 query with the driver-side build enabled, once with the on-heap and once with the off-heap build relation, and assert the broadcast relation is a SerializedHashTableBroadcastRelation, that is, the driver-side build did not fall back. Both fail without this change. The #12407 query moves into a shared val used by its original test and the new ones.

Was this patch authored or co-authored using generative AI tooling?

Yes. Generated-by: Claude Opus.

Related issue: #13330.

Copilot AI balanced review requested due to automatic review settings October 15, 2026 17:41

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@github-actions github-actions Bot added the VELOX label Oct 15, 2026
@zhouyuan
zhouyuan requested a review from JkSelf October 16, 2026 15:38

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants