[amar-python]

Summary

tests/ held 41 Python test files in one folder, so there was no way to run a single category and new tests landed unclassified. They are now grouped by what a test needs to run:

Folder	Needs	Files
unit/	no database, no network	12
system/	the API against a live PostgreSQL	23
system_integration/	whole pipeline, every environment, parity	3 + environments/ (4)
performance/, accessibility/, uat/	placeholders, each a package	0

Same tests. At the time of the move the set of collected test ids was identical before and after (379, diffed, not just counted), and the full suite, evals and SQL suites gave the same results. main has since added tests (#215); the one new file, test_te_reupload_registry.py, is placed in system/ as part of the merge below.

Commits: the move and everything it requires; the additive parts (fixtures, make targets, README); and a merge of main. 39 files show as renames (git diff -M); the four tiny test_env_* files show as delete + add because they are 6 lines each.

What the move needed

• 20 files compute paths from __file__; they now use parents[2], and the snapshot directory stays at tests/snapshots. The cross-test import (test_password_reset → test_accounts) and the environments package imports use the new module paths. (I broke the snapshot path once while doing this and caught it in the diff review.)
• scripts/health_check.py globbed tests/test_*.py non-recursively, so make health (run by CI's free-tier job) would have failed. Now recursive.
• select_tests.py matched the fragment tests/test_; it now maps each category folder. scripts/run_qa.ps1 named two test files by path (edited in bytes, CRLF kept).
• scripts/lint.sh listed 14 of the 38 test files by hand. It now finds every Python file under tests/ (50). That surfaced 40 cosmetic flake8 findings in 10 files, fixed.
• pytest.ini registers system, sit, performance, accessibility, uat; conftest.py adds the marker for a test's folder automatically, so -m system and -m "not accessibility" agree with the layout. unit stays explicit (the free tier and the Windows job select on it).
• Live docs and code comments that name test paths are updated; historical records (bug reports, audits, handoffs) are left as written.

Added

• Shared fixtures (live_api, destructive_enabled, registry_cleanup, te_org_cleanup) for tests that drive the live API. Two worked examples: system/test_failed_uploads.py drops its setUp/tearDown boilerplate and now cleans up the registry rows it creates (it never did); one test in system/test_review.py loses a try/finally. After running them, 0 allfail_* registry rows and 0 TE_REV_* organisations remain.
• make test-unit | test-system | test-sit | test-performance | test-accessibility | test-uat. An empty category reports that and succeeds.
• tests/README.md: the layout, where a new test goes, the fixtures.

Not done

• 19 other files that start the API by hand are not converted to the fixtures. Converting them would bury the move in a refactor; the README says so.
• requirements-dev.txt is not touched. The issue lists playwright and axe-playwright-python, but they are only used by the accessibility job (Add accessibility CI job (Playwright + axe-core WCAG 2.1 AA scans) #168, PR ci: accessibility job, Playwright + axe-core WCAG 2.1 AA scans (#168) #222), which adds them in a separate requirements-a11y.txt.
• Six files in system/ also hold database-free classes (marked unit). They live in the category of their heaviest need and still run in the free tier, because it selects on the marker. So -m unit and -m system overlap by those classes; the README explains it.

Merged with main, and what other open PRs add

main moved while this was open (#215 changed upload_te's tests and added tests/test_te_reupload_registry.py), so the PR stopped merging cleanly. main is merged in: Git followed the rename of test_api_coverage.py on its own; the new flat file is moved to tests/system/ (and the one docstring that names its path updated); the generated docs/TEST_CASES.md is regenerated. The full suite passes on the merged tree against a freshly deployed PostgreSQL 16.

Merge order matters for other PRs. I simulated the merge with each. #204 and #208 merge cleanly. #213 conflicts only in the generated docs/TEST_CASES.md (regenerate with python3 scripts/test_catalogue.py); Git followed the renames through test_migrations.py and test_tenant_isolation.py.

Those PRs add test files at the top of tests/, which this layout leaves unclassified. Suggested homes once they land: test_prod_db_naming.py → unit/, test_docker_image_secrets.py → unit/, test_tenant_te_migrations.py, test_type_provenance.py, test_runtime_role_per_env.py → system/. Nothing enforces the layout, so a straggler is harmless but unclassified until moved.

Verification (local, PostgreSQL 16, all four environments deployed)

• Collected test ids: identical to the 379 on main at the time of the move (test_oidc.py excluded locally only: an unrelated cffi import problem in this sandbox)

• On a clean database, in CI order: evals/runner.py --tiers p,i,s,e 26/26; tests/run_tests.sh dev test 159/159 each; full pytest passes (also after the merge of main); coverage floors hold (csv_parse 82.1% ≥ 79, dynamic_loader 87.3% ≥ 84, te_loader 97.3% ≥ 96)

• make lint clean (50+ test files), make health 89 passed, make test-unit 122 passed, test_catalogue.py --check current

• The database-free marker set (the Windows job's selection) passes: 148 tests

• CI green on the current head (1f195a0): free tier, integration (postgres service) and the Windows python-validator-tests job.

---

Generated by Claude Code