You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
RV-S7: SQL test wrapper exits 0 on failed assertions (run_tests.sh) #181
External repo review at commit c7bebd1 reproduced: tests/framework/test_framework.sql records FAIL rows without raising; tests/run_tests.sh exits 0 unless psql itself errors. Scratch run with a forced assertion failure returned exit 0.
Tier S eval parses report results separately, which limits CI exposure, but the documented standalone run_tests.sh verification command is unreliable for automation.
Why this matters
Affects confidence in #60 offline install evidence: the PASS lines in the clean-VM dry run come from this wrapper. Until fixed, a passing exit is not evidence that the assertions passed.
Proposed
Make the SQL orchestrator raise (or exit non-zero) when any assertion recorded FAIL.
Shell wrapper propagates that exit status.
Negative control: a forced assertion failure in a test fixture that CI confirms makes the wrapper exit non-zero.
Done when
Forced FAIL assertion makes tests/run_tests.sh exit non-zero.
Existing passing suite still exits 0.
Negative control committed and runs in CI.
Reference
Tracker row G40. Repo review finding RV-S7. Sub-issue of #60 (offline install evidence trust). Related to RV-S6 (Tier I ON_ERROR_STOP).
Activity
fix: address 8 open issues — security, data integrity, test reliability
amar-python commentedon Nov 9, 2026
Fixed in #202 (merged as 203c8ae).
Generated by Claude Code
test: drop the #181 expected-fail test; CI fails any skip