Skip to content

Fix single-threaded rowset data loss (VTA-118774, VTA-118542) - #25

Merged
SandeepDave2 merged 1 commit into
mainfrom
VTA-118774-VTA-118542-Fix_data_loss_running_COPY_with_random_rowset
Oct 7, 2026
Merged

SandeepDave2 merged 1 commit into
mainfrom
VTA-118774-VTA-118542-Fix_data_loss_running_COPY_with_random_rowset

Conversation

@SandeepDave2

Copy link
Copy Markdown
Collaborator

Fix:
The single-connection loader previously yielded after 10,000 rows, even mid-rowset, causing unprocessed rows to be skipped.

Move the yield check after the complete fetched rowset and use >= ROWS_PER_BREAK. Cooperative yielding is preserved, the multithreaded worker/queue path is unchanged.

Addresses VTA-118774 and VTA-118542.

Verification:
Manually tested in fenced mode on Vertica 26.4.0 with PostgreSQL 15:

  • Exact 10K reproduction: rowset=9999 now loads all 19,998 rows instead of 10,000, with no missing or duplicate IDs.
  • 100,000-row loads pass with rowsets 100, 110, 123 and 9999.
  • Single-connection fallback and four-thread range loading pass.
  • Mixed-type direct COPY and external-table/CTAS values match the control.
  • Empty results, cancellation rollback and same-session recovery pass.

@mkottakota1 mkottakota1 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed with Sandeep and changes are looking good

@SandeepDave2
SandeepDave2 merged commit f6669a6 into main Oct 7, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants