Repository navigation
fix: a T&E re-upload that fails every row changes nothing (#215) - #220
Merged
Merged
Conversation
upload_te deleted the earlier load's registry record (and dropped a prior dynamic table of the same name) before inserting any rows, and skipped the #170 all-failed rollback for re-uploads. A re-upload where every row failed therefore replaced "6 loaded" with "0 loaded, 10 failed" while the rows stayed in the table, returned status "ok", and, for a same-named dynamic table, dropped it: #170's original scenario still reproduced on main. The permission check for overwriting stays up front. The cleanup now runs only after at least one row has landed, and the all-failed rollback applies to re-uploads too, so a fully failed re-upload returns status "error" and leaves the registry and any prior table alone. Tests: tests/test_te_reupload_registry.py (all-failed re-upload keeps the record; a re-upload that inserts rows still replaces it; #170's prior-dynamic- table case). Two older tests re-uploaded an identical single-row file whose unique name made every row fail, so they encoded the old behaviour; they now re-upload a file that can succeed. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011T6Wqm1jLdq3ZEfuKdp9aM
This was referenced Nov 9, 2026
The "Test catalogue check" CI step failed because docs/TEST_CASES.md did not list the new T&E re-upload tests. Regenerated with scripts/test_catalogue.py. Also gives the three new tests a one-line docstring, and makes the edited test's docstring open with a complete sentence: the catalogue shows only the first docstring line, which was cutting it off mid-sentence. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011T6Wqm1jLdq3ZEfuKdp9aM
Owner
Author
|
CI: While there I gave the three new tests a one-line docstring and made the edited test's docstring open with a complete sentence, since the catalogue shows only the first line. Waiting on the re-run. Generated by Claude Code |
amar-python
marked this pull request as ready for review
November 9, 2026 10:42
amar-python
pushed a commit
that referenced
this pull request
Nov 9, 2026
#220 added tests/test_te_reupload_registry.py at the top of tests/, where this layout would leave it unclassified; moved to tests/system/ (it drives the API against PostgreSQL) and the docstring that names its path updated. The test catalogue was the only conflict: taken from main and regenerated. Full pytest suite passes against a freshly deployed PostgreSQL 16. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VBrxqChRJtxdvSpFhiUWUy
1 task done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A T&E re-upload that fails every row now changes nothing: the earlier load's registry record survives, and any prior table of the same name is kept. Closes #215.
The bug
upload_tedeleted the earlier load's registry record, and dropped a same-named dynamic table, before inserting any rows. It also skipped the #170 all-failed rollback for re-uploads (and not is_reupload). So a re-upload where every row failed replaced "6 loaded" with "0 loaded, 10 failed" while the 6 rows stayed in the table, and returnedstatus: ok.#170's original scenario still reproduced on
main(same root cause, so fixed here). Before this change, onmain: load a dynamic file, then upload an all-invalid T&E file with the same filename:mainatad39ddf)ok, inserted 0error, inserted 0te / 0 rowsdynamic / 1 rowThe fix
In
api/services/te_loader.py(net -3 lines):_cleanup_prior_registry_entriesnow runs only after at least one row has landed.Behaviour change (approved on #215)
A re-upload where every row fails now returns
status: error(the same response a first-time all-failed upload already gets), notokwith 0 inserted. Re-uploads that insert at least one row behave as before.Tests
tests/test_te_reupload_registry.py(3 tests, live PostgreSQL): all-failed re-upload keeps the registry record and the table agrees with it; a re-upload that inserts rows still replaces the record (guards against over-correcting); RV-F3 (BR-34): all-failed T&E import runs registry cleanup and removes prior dataset #170's prior-dynamic-table case. Two of the three failed onmainfor the stated reason before the fix; the third (replace-on-insert) is a guard that passes both ways.test_te_reupload_same_file_replaces_registry(test_api_coverage.py) andtest_reupload_cleans_prior_registry_entry(test_te_loader_integration.py). Both re-uploaded an identical one-row file whose uniquenamemade every row fail, so they were asserting the old behaviour by accident. They now re-upload a file that can succeed, which keeps their actual purpose (a re-upload replaces the record rather than adding a second).Verification
Full suite against PostgreSQL 16 with all four environments deployed, as CI does: 391 tests, 388 passed, 0 failed, 0 skipped, 3 errors. The 3 errors are
test_csv_loader_arbitrary_shapes(build/config.local.env not present); they fail identically onmainwithout this change. Not run: CI (this is a draft).Heads-up
#214 moves the test files into category folders, including the two I edited. Whichever merges second will need a small conflict resolution in those files, and my new test file will need moving into
tests/system_integration/.Generated by Claude Code