Skip to content

[To dev/1.3] [Pipe] Respect history enable with source time range (#18362) - #18385

Merged
jt2594838 merged 1 commit into
apache:dev/1.3from
Caideyipi:cp-18362-dev-1.3
Aug 4, 2026
Merged

[To dev/1.3] [Pipe] Respect history enable with source time range (#18362)#18385
jt2594838 merged 1 commit into
apache:dev/1.3from
Caideyipi:cp-18362-dev-1.3

Conversation

@Caideyipi

Copy link
Copy Markdown
Collaborator

Description

Backport #18362 (154644107a82ba69431725390b8346149263bd0e) to dev/1.3.

A global source.start-time or source.end-time previously enabled historical extraction unconditionally. As a result, source.history.enable=false could still transfer data that existed before pipe creation.

This backport:

  • computes historical enablement independently from the global time range;
  • preserves the default behavior when source.history.enable is omitted;
  • preserves historical recovery for pipe restarts and newly added DataRegions;
  • keeps global time-range precedence over history-specific ranges and updates the validation warning;
  • adapts the historical implementation and regression test to the 1.3 PipeHistoricalDataRegionTsFileSource API.

Verification

  • mvn spotless:apply -pl iotdb-core/datanode
  • mvn spotless:apply -pl integration-test -P with-integration-tests
  • PipeHistoricalDataRegionTsFileSourceTest#testGlobalTimeRangeRespectsHistoryEnable: 1 test, 0 failures
  • git diff --check

The full DataNode Maven test was also attempted, but this checkout has unrelated pre-existing compilation errors in Pipe builder/sink classes; the changed sources were isolated-compiled and the targeted test passed.


This PR has:

  • been self-reviewed.
  • added or modified unit tests.
  • added an integration-test regression case.

Key changed/added classes
  • PipeHistoricalDataRegionTsFileSource
  • IoTDBDataRegionSource
  • PipeHistoricalDataRegionTsFileSourceTest
  • IoTDBPipeSourceIT

@jt2594838
jt2594838 merged commit 5b08947 into apache:dev/1.3 Aug 4, 2026
15 of 16 checks passed
@jt2594838
jt2594838 deleted the cp-18362-dev-1.3 branch August 4, 2026 04:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants