[amar-python]

Summary

Work on the "04 Enhancements" batch (#167, #168, #169, #191, #192, #193, #194, #195).

While I was building this, main moved. Merged #202 already delivered #169, #193 and #194, and open draft #213 implements #195. So this PR is rebased onto current main and contains only what main lacks, plus one defect the new tests found. Nothing here overrides upstream's #169, #193 or #194 implementations.

Issue	Status in this PR
#167 tests into category folders	Done here
#168 accessibility CI job	Done here (needs a CI run, see below)
#191 registry race	Done here, and fixes a real defect
#192 ragged rows	Done here, as a delta on upstream's tests
#193 XLSX formula guard	Upstream's guard kept. Adds the "stored as text" assertion the issue asks for, plus the spreadsheet check
#169 catalogue, #194 coverage floors	Not touched. Already on main and equivalent to what I built; I dropped mine. Catalogue regenerated for the new layout
#195 confirm types	Not included. Collides with #213; see "Decision needed"

Changes

#167. Six folders (unit, system, system_integration, performance, accessibility, uat), each a package. tests/conftest.py gives every test its folder's marker, so pytest -m sit and pytest tests/system_integration select the same set, and adds the shared fixtures (live_api, registry_cleanup, te_org_cleanup, destructive_enabled, unique_tag). New markers, make test-* targets, tests/README.md. scripts/lint.sh now collects every test file with find, so the list can't drift again. The three test files that landed flat on main after I branched are moved too.

#168. tests/accessibility/ (Playwright + axe-core, WCAG 2.1 AA) and a new accessibility job. / and /audit redirect to /auth when signed out, so a plain scan would check the sign-in page three times and report a false green. The tests seed the te.session sessionStorage entry, stub /api/**, and assert they landed on the route they asked for. The integration job and the coverage gate step now exclude the marker.

#191. Concurrency test: uploads released together by a barrier, 40 rounds, for identical file, identical content under different names, and same name with different content. It asserts one winner, one registry row, an attributable rejection for every loser, and no orphan tables. Upstream's test tolerated "error" as a valid loser outcome, so it could not catch this: the losing request can get a generic status: "error" ("relation … already exists" / "type … already exists") instead of duplicate_file, because the loader catches only UniqueViolation. It now also catches DuplicateTable and DuplicateObject, and names the winner rather than echoing the loser's own file. This replaces upstream's test_registry_race.py.

#192. Upstream's tests check the accounting identity and a reason per ragged row. Adds what they don't: the reason is also stored (GET /api/csv/uploads/{id}), and the same for T&E mode.

#193. Asserts the XLSX cell is a text cell with no formula, for =cmd, +cmd, -cmd, @cmd and a tab-prefixed value, in the error and audit reports.

Evidence

All run against PostgreSQL 16 with all four environments deployed, on this branch rebased onto main (ad39ddf).

• Backend, as CI runs it (test_report.py --markers "not accessibility" --strict): 391 passed, 0 failed, 0 skipped
• Test count: main 388; this branch 391 non-accessibility + 7 accessibility. Reconciles exactly (388 − 2 upstream race tests + 3 mine + 2 ragged), so the reorg lost nothing
• Upstream's coverage gate, unchanged: csv_parse ≥ 79, dynamic_loader 87.6% (≥ 84), te_loader 97.3% (≥ 96), pass
• make lint, bandit, make health (89), test_catalogue.py --check: pass. Frontend: vitest 165 passed, tsc clean
• Registry race: concurrent uploads of the same file need a regression test (F1) #191 on today's main: with the loader fix, 5/5 runs pass. With the fix reverted (upstream behaviour), 3/3 fail
• Mutation checks (each fails when the guard is removed): dropping UNIQUE(file_hash) fails the different-names race test 10/10 (the same-name test does not catch it, since UNIQUE(file_name) also guards that case; documented in the test); extra_cells() returning [] fails 6 of 9 ragged tests; disabling upstream's XLSX guard fails 11 of 38 export tests
• Spreadsheet check (Formula injection in exports: guard XLSX as CSV, and test both (F3) #193): the XLSX produced by downloadErrorReport, opened in LibreOffice 24.2.7.2, shows '=1+1, '=cmd, '+cmd, '-cmd, '@cmd literally, not evaluated; -5 and plain untouched

First local run of the accessibility scans against the Vite dev server: 4 pass, 3 fail. The failures are real frontend defects, as #168 expects: an unlabelled select on /audit, and no <main> landmark on /auth and /audit.

Not verified, and decisions needed

• CI has not run on this PR. Add accessibility CI job (Playwright + axe-core WCAG 2.1 AA scans) #168's "runs green on a trivial PR" and the job wiring are unverified. The scan step is continue-on-error on purpose, because of the known frontend defects.
• Microsoft Excel was not available. Formula injection in exports: guard XLSX as CSV, and test both (F3) #193's check covers LibreOffice only, and the sample XLSX is not attached to the issue.
• Dynamic mode: confirm inferred types before table creation, with a provenance flag #195, decision needed. feat: type provenance (#195), prod DB naming (#172), tenant T&E migrations (#184) #213 does not require confirmation (it records inferred_unconfirmed and flags strictness as "Decision for the owner"). The issue text says "require". I built the strict version: dynamic upload without typesConfirmed creates nothing and returns the inferred types; migration 0009, provenance returned and shown in the files list; a retry reuses confirmed types rather than re-inferring; plus the T&E-rejects-dynamic-table test. It breaks every API caller that omits the flag (~40 test call sites and smoke_test.sh were updated). It is preserved, not proposed, as commit 0fcedaf on branch enh/04-all-eight-original, which also holds my dropped Add test catalogue generator (scripts/test_catalogue.py + docs/TEST_CASES.md) #169/Gate: measure api/services coverage with floors for the three loaders #194 versions for comparison. Both designs claim migration 0009, so whichever merges second will need renumbering.
• Gate: measure api/services coverage with floors for the three loaders #194 floors. Still the issue's 79/84/96; the product owner is to confirm. dynamic_loader now measures about 87.6%.
• Pre-existing, found on the way: csv-table-hub-main/bun.lock is stale (zod ^3 vs ^4), so bun install --frozen-lockfile fails. The new job uses npm ci for that reason.

Closes #167, #191, #192. Refs #168, #193 (CI run and Excel outstanding).

---

Generated by Claude Code