Skip to content

RV-F3 (BR-34): all-failed T&E import runs registry cleanup and removes prior dataset #170

Closed

Description

@amar-python

What happened

External repository review at commit c7bebd1 reproduced a scenario where a completely failed T&E import still runs registry cleanup and removes a prior dataset. BR-34's current claim is that all-failed imports save nothing — this is violated in the T&E path, which also destroys existing data.

Reproduction (from the review):

  • Upload an invalid organisation record to a T&E table in a tenant that already has a prior dynamic table with the same filename.
  • API returns status: ok, insertedRows: 0, failedRows: 1.
  • The prior dynamic table with the same filename is dropped as part of the T&E loader's cleanup step.
  • The frontend shows "0 inserted" with no indication that prior data was lost.

Reproduction path per the review: api/services/te_loader.py L87–112 (cleanup) and L294–338 (registration/commit).

Why this matters

The dynamic path at api/services/dynamic_loader.py L323–334 has an all-failed rollback protection. The T&E path doesn't — it was never carried across when T&E mode was added. This is current data loss, not a staged-import gap.

Decision taken 3 Nov 2026

Fix pre-M6 (option b of D11). Reasoning: no production tenants yet, but the hazard is immediate for any first external tenant. Fix is small (~1 hour) and does not need M6's staged-import architecture. The staged importer will subsume it later; interim cost is a few lines of guard code plus a test.

Proposed

In api/services/te_loader.py, before the cleanup/registration path runs, check inserted_rows > 0. If zero, roll back the cleanup, preserve the prior dataset, and return a clean failure response that reflects what actually happened.

Also: the response schema should distinguish "all failed, prior data preserved" from "all failed, nothing to preserve" so the frontend can message correctly. That is a smaller follow-up.

Done when

  • Reproduction test added: uploads an invalid-only CSV against a tenant with a prior dynamic table of the same filename, asserts the prior table and its rows persist after the all-failed import.
  • The test fails on main at c7bebd1 without the fix.
  • The test passes with the fix.
  • CI replica run quoted on the PR (counts in passed / failed / skipped form).
  • No change to the dynamic-mode happy path (regression test).

Scope

T&E path only. Dynamic-mode all-failed behaviour is already correct and must not change.

Reference

Tracker row G46 (main tab). Decision D11. Related: G44 (RV-F1), G45 (RV-F2), both P1. ART extension milestone.

Activity

  1. added this to the ART extension milestone on Nov 3, 2026
  2. amar-python commented on Nov 3, 2026

    @amar-python
    OwnerAuthor

    Cross-link from #175

    #175 (RV-S5: upload overwrite and mode-switch bypass delete-permission gate) has been bundled with this issue by the product owner's decision on 3 Nov 2026.

    Both touch api/services/te_loader.py L87–112. The fix for #175 proposes a new api/services/destructive_ops.py with a safe_drop_with_outcome(user, tenant, table, outcome) function that combines:

    See #175 for the full fix proposal. One PR will close both.

  3. amar-python commented on Nov 9, 2026

    @amar-python
    OwnerAuthor

    Fixed in #202 (merged as 203c8ae).


    Generated by Claude Code

  4. amar-python commented on Nov 9, 2026

    @amar-python
    OwnerAuthor

    Note for the record: this issue's own scenario (a prior dynamic table with the same filename, then an all-failed T&E upload) still reproduced on main at ad39ddf after #202. The all-failed guard was skipped when the upload counted as a re-upload, which is exactly when the filename matches, and the cleanup ran before any row was inserted. The dynamic table was dropped (404 afterwards) and the registry replaced by a 0-row T&E record. #220 fixes it, with a regression test for this exact case (test_all_failed_te_upload_keeps_a_prior_dynamic_table_of_the_same_name). Leaving this closed.


    Claude Code


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions