Skip to content

fix(hstore): preserve range-index ordering across partitions - #3140

Open
contrueCT wants to merge 5 commits into
apache:masterfrom
contrueCT:task/issue-3053-hstore-range-index-ordering
Open

fix(hstore): preserve range-index ordering across partitions#3140
contrueCT wants to merge 5 commits into
apache:masterfrom
contrueCT:task/issue-3053-hstore-range-index-ordering

Conversation

@contrueCT

Copy link
Copy Markdown
Contributor

Purpose of the PR

HStore range-index queries with a limit, offset, or page cursor require globally
ordered backend keys. The previous multi-partition path exposed partition
iteration order and an internal storage cursor, which could return the wrong
limited slice or skip entries on continuation.

This PR adds an explicit ordered range-scan path while leaving unbounded count,
joint-index, and cleanup scans on the existing path.

Main Changes

flowchart LR
    subgraph Store1["Store 1"]
        P1["Partition 1"] --> M1["Local K-way merge"]
        P2["Partition 2"] --> M1
    end
    subgraph Store2["Store 2"]
        P3["Partition 3"] --> M2["Local K-way merge"]
        P4["Partition 4"] --> M2
    end
    M1 --> G["Client global K-way merge"]
    M2 --> G
    G --> R["Ordered range-index results"]
Loading
  • Preserve the next unread physical index key as the HugeGraph page cursor and
    preserve ordered index IDs through the following element lookup.
  • Add ORDER_BY_KEY to the HStore scan protocol. Each Store merges its local
    leader-partition iterators, and the client globally merges one stream per
    Store instead of opening one stream per partition.
  • Use fixed 64-entry pages, request later pages only when a Store's current page
    is exhausted, and bound concurrent first-page initialization to eight
    streams.
  • Keep legacy scan ordering and cursor behavior unchanged for requests that do
    not explicitly select the ordered path. Client and Store nodes must be
    upgraded together before using ordered range scans.

Verifying these changes

  • Trivial rework / code cleanup without any test coverage. (No Need)
  • Already covered by existing tests, such as (please modify tests here).
  • Need tests and can be verified as follows:
    • ClientSuiteTest: 13 tests passed.
    • ServerSuiteTest: 6 tests passed.
    • HstoreSessionsImplTest and HstoreTableTest: 8 tests passed.
    • GraphIndexTransactionTest, IdHolderTest, and QueryResultsTest: 4 tests passed.
    • git diff --check passed.

Does this PR potentially affect the following parts?

Documentation Status

  • Doc - TODO
  • Doc - Done
  • Doc - No Need

@contrueCT
contrueCT marked this pull request as ready for review August 6, 2026 15:08
@dosubot dosubot Bot added size:XXL This PR changes 1000+ lines, ignoring generated files. api Changes of API perf store Store module labels Aug 6, 2026

@imbajin imbajin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: yes. Summary: The ordered range-index path has a multi-batch ordering defect and is unsafe during mixed-version Store upgrades. Evidence: static review of QueryList/QueryResults and NodeTxSessionProxy/store_stream_meta.proto at head b45fded.

@imbajin imbajin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: yes. Summary: The ordered range-index path still has correctness, cursor, resource, and compatibility defects beyond the existing review coverage. Evidence: exact-head static review of QueryResults, ScanUtil, OrderedKvIterator, OrderedMultiPartitionIterator, and public scan contracts at b45fded.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api Changes of API perf size:XXL This PR changes 1000+ lines, ignoring generated files. store Store module

Projects

Status: In progress

Development

Successfully merging this pull request may close these issues.

[Bug] HStore range-index scans don't guarantee global ordering or stable paging

2 participants