perf(ci): pr-review-receipt is dispatch-only — 96 h of fleet runner-time for a job that gates nothing (PMAT-1078) - #3046
perf(ci): pr-review-receipt is dispatch-only — 96 h of fleet runner-time for a job that gates nothing (PMAT-1078)#3046noahgift wants to merge 5 commits into
Conversation
…ime for a job that gates nothing (PMAT-1078) BSE-15 measured this job as the fleet's single largest consumer of runner-hours: 96 h of aprender PR time in the 30 days to 2026-09-07, inside a fleet that threw away 42.1 % of everything it spent. It is 150 minutes of two mutation sweeps on a clean-room runner, on every push to every open PR. It gates nothing. It is not a required context (`ci / gate` and `workspace-test` are), no job `needs:` it, and `gate` deliberately stopped reading it (PP-066 C0-5, PRQ-013, #2982) because a head-defined receipt job is a check the PR under review can edit. What does gate the receipt is pr-review-quorum.yml, which runs from the BASE on pull_request_target + merge_group and invokes check_pr_review_arm4.sh itself — it consumes no artifact from this job, so this job stopping changes no verdict anywhere. The job stays fully wired and runnable on demand: gh workflow run ci.yml --ref <branch> On a dispatch there is no pull_request context, so PR_HEAD_SHA falls back to github.sha; every sha-consuming step is already fail-closed and sweeps rather than skips, so the fallback cannot turn a dispatch into a silent no-op. check_pr_review_wiring.sh R4 moves WITH the policy rather than being routed around: EVENTS_TRUE/EVENTS_FALSE swap, both failure messages say why, and a new self-test row carries the old `if: github.event_name == 'pull_request'` verbatim and asserts FAIL — so putting the job back on every PR is a red check, not a quiet reversion. Self-test 26/26; discrimination confirmed by restoring the old `if:` on the real ci.yml (guard exits 1, both polarities) and restoring the new one (exit 0). Refs PMAT-1078, BSE-15, BSE-001 §4 wave 5 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B2EenRiCc3FbvW5GJe2xFh
|
§13.11 rung 1 — quorum shadow verdict Shadow mode: this records a verdict and merges nothing. A refusal |
|
quorum-review (AD-04): NOT agreed {
"ticket": "PMAT-1078",
"head": "77ba7d2fd14ad6d9c33d8f97ce5c5fc8041373ad",
"width": 3,
"executor": "agy",
"agreed": false,
"lanes": [
{
"lane": 1,
"verdict": "FAIL",
"findings": 2
},
{
"lane": 2,
"verdict": "FAIL",
"findings": 2
},
{
"lane": 3,
"verdict": "FAIL",
"findings": 1
}
]
} |
|
Quorum: NOT AGREED — 3 lanes, 3 FAIL ( 1. The diff gives On Remedy: give 2. The roadmap diff is a The 2 "deleted" are legacy prose-id entries whose id strings were re-folded ( Remedy the guard itself names: Note on the ticket id. PMAT-1078 is this PR's, minted and |
…er-time for a job nothing reads (PMAT-1078)
Supersedes this branch's first commit, which made the job dispatch-only. Three
independent quorum lanes and the orchestrator each found that the dispatch path
could not work: the diff gave PR_HEAD_SHA a `|| github.sha` fallback and left
PR_NUMBER bound to `github.event.pull_request.number`, which is empty on a
dispatch, while check_pr_review_arm4.sh:372 is `${PR_NUMBER:?...}` — a hard
refusal. The job would have been reachable and broken. So it is deleted.
THE QUERY THE DELETION IS GATED ON, recorded at the deletion site in ci.yml:
$ grep -rn 'needs:' .github/workflows/*.yml | grep pr-review-receipt # no match
$ grep -rn 'download-artifact' .github/workflows/*.yml # nightly.yml:141 only
$ gh api repos/paiml/aprender/branches/main/protection # ["ci / gate", "workspace-test"]
The job also uploaded no artifact, so no consumer could read one, and `gate` had
already stopped reading it (PP-066 C0-5, PRQ-013, #2982).
WHAT STILL RUNS: check_pr_review_receipt.sh, check_pr_review_counts.sh and
check_pr_review_arm4.sh are `scripts/check_*.sh`, so guard_tree.sh's derived
universe runs them in `guard-tree`, which `gate` needs. The PR's own receipt is
judged from the BASE by pr-review-quorum.yml, which consumed nothing from here.
WHAT NO LONGER RUNS IN CI, stated rather than discovered later: the two mutation
sweeps (scripts/mutate-guard.sh, scripts/mutate_quorum_arm.sh) and the 43-row
bats fixture table. No other workflow invokes them and they are not check_*.sh,
so guard_tree does not reach them. They remain runnable by hand.
check_pr_review_wiring.sh INVERTS rather than dying with its subject. R4 (the
`if:` polarity table) and its evaluator are deleted with the job; R1 becomes the
standing falsifier — no ci.yml job may invoke the receipt guard — and R3 is new:
the guard must stay a tracked scripts/check_*.sh, or R1 would hold vacuously
over a family that runs nowhere. The case table carries the old
`if: github.event_name == 'pull_request'` wiring VERBATIM, asserted FAIL, and
also refuses a dispatch-only job, so restoring it is a change to this rule plus
a case-table row rather than an edit to ci.yml alone. 10/10, both polarities on
every rule, R3 driven against real git trees.
scripts/ci_job_minutes.sh makes the 96 h re-derivable instead of asserted: Σ per
JOB of (completed_at − started_at) from the jobs API — runner time, not wall
time; started_at, not created_at, so queue wait is excluded; skipped jobs
contribute nothing. It refuses rather than returning 0 h when gh is absent or
the window is past run retention. --self-test 5/5 offline, including the
polarity that would catch a sum that had silently used created_at.
roadmap.yaml: `python3 scripts/roadmap_trim.py` collapses the `pmat work`
re-serialisation the first commit carried (369 entries rewritten, two legacy
prose-id entries re-folded) back to base bytes — now 1 added, 0 deleted, 26
lifecycle edits.
Refs PMAT-1078, BSE-15, BSE-001 §4 wave 5, #3046
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X1cW8TnmNe3hgjragNjUxr
|
quorum-review (AD-04): NOT agreed {
"ticket": "PMAT-1078",
"head": "3115c74e1bd77ae4bbf019f3428b579f35179e91",
"width": 3,
"executor": "agy",
"agreed": false,
"lanes": [
{
"lane": 1,
"verdict": "FAIL",
"findings": 2
},
{
"lane": 2,
"verdict": "FAIL",
"findings": 2
},
{
"lane": 3,
"verdict": "FAIL",
"findings": 2
}
]
} |
… a total (PMAT-1078)
Found by running the script rather than reading it. The first derivation asked for
400 runs and `gh run list` returned exactly 400 — the cap bound, the window was not
fully covered, and the sum came out as a floor wearing the units of a total:
=== runner-hours per job — paiml/aprender, runs created 2026-08-08..2026-09-07 (400 runs, event=pull_request) ===
hours runs job
326.8 261 pr-review-receipt
210.2 272 workspace-test
98.6 259 guard-runner-labels
850.6 TOTAL
`gh run list --limit N` truncates with no signal, so equality with the limit is the
only evidence available and it is now treated as truncation: exit 2, naming a larger
--limit to re-run with. This is the same class as the silent caps this repository
already refuses elsewhere — a number that quietly answers a smaller question than
the one asked.
Self-test 6/6. The new row drives the real script against a window whose run count
reaches --limit and requires exit 2 with TRUNCATED; it SKIPS with a printed line,
never a silent pass, where gh cannot answer.
Refs PMAT-1078, BSE-15, #3046
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X1cW8TnmNe3hgjragNjUxr
|
quorum-review (AD-04): NOT agreed {
"ticket": "PMAT-1078",
"head": "8a2f0656e969b23a0fd644341890d1e920aaeed5",
"width": 3,
"executor": "agy",
"agreed": false,
"lanes": [
{
"lane": 1,
"verdict": "FAIL",
"findings": 2
},
{
"lane": 2,
"verdict": "FAIL",
"findings": 2
},
{
"lane": 3,
"verdict": "FAIL",
"findings": 2
}
]
} |
…s with the policy (PMAT-1078)
Two findings from the second 3/3-FAIL quorum, both correct, both acted on.
1. THE TICKET WAS STILL THE OLD POLICY. All three lanes failed the diff for
contradicting its own ticket: PMAT-1078 read "move the job to workflow_dispatch
only, keep it fully wired and runnable on demand", and the diff deletes it. The
ticket is the standard a quorum judges against, so it moves with the operator's
rescope rather than leaving the diff arguing with it. Its acceptance criteria
now carry the deletion, the recorded query it is gated on, the guard inversion,
the measurement script, and the STATED LOSS of the two mutation sweeps.
2. THE ROADMAP STILL CARRIED A VOCABULARY MIGRATION, and G-6 could not see it.
`roadmap_trim.py` collapsed the re-serialisation, but 26 unrelated entries kept
a `pmat work` rewrite of their status VALUES — `done` -> `completed`,
`in_progress` -> `inprogress` (19 entries), `todo` -> `planned`. G-6 classifies
`status` as a lifecycle field, so a mass vocabulary migration is
indistinguishable from one ticket's honest status flip and the guard passed it:
`lifecycle=26 reserialised=0 deleted=0 PASS`. Three lanes saw it anyway.
The file is now main's bytes plus one inserted entry, so the diff is what the
PR is about:
added: ['PMAT-1078'] changed: [] deleted: []
check_roadmap_ids_unique.sh PASS (819 ids, nested records included)
check_roadmap_sorted.sh PASS (sorted within each id-prefix)
Rebuilding it this way also removed a defect the previous restore introduced:
keying entries by their id TEXT left the re-folded twins of two legacy prose-id
entries ("Sovereign AI Integration Specification v1.1 …", "Phase 2: Bayesian
Examples & Documentation …") alongside main's originals — two duplicate ids that
only the tree-parsing uniqueness guard caught, since the sorted guard reads
PREFIX-NUMBER ids and saw nothing.
Refs PMAT-1078, BSE-15, BSE-001 §4 wave 5, #3046
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X1cW8TnmNe3hgjragNjUxr
|
quorum-review (AD-04): NOT agreed {
"ticket": "PMAT-1078",
"head": "bb2dca50b153ae9c3e425c2d9e47af2d1497e605",
"width": 3,
"executor": "agy",
"agreed": false,
"lanes": [
{
"lane": 1,
"verdict": "FAIL",
"findings": 1
},
{
"lane": 2,
"verdict": "FAIL",
"findings": 1
},
{
"lane": 3,
"verdict": "FAIL",
"findings": 1
}
]
} |
…iew-shadow (PMAT-1078)
Third 3/3-FAIL quorum, and all three lanes named the same leftover: the first
commit on this branch gave `PR_HEAD_SHA` a `|| github.sha` fallback at TWO sites,
and deleting the `pr-review-receipt` job removed only one of them. The other sits
in `pr-review-shadow`, a job this PR never had a reason to touch, which still runs
on `pull_request` — where the fallback is dead weight, and where its comment
("on workflow_dispatch there is no pull_request context") describes an event the
job does not run on.
It is the pattern the rescoped ticket explicitly rejects, surviving in the one
place the deletion did not reach. Reverted to main's bytes, so ci.yml's diff
against main is now the deleted job and nothing else.
grep -c 'github.event.pull_request.head.sha || github.sha' .github/workflows/ci.yml # 0
python3 -c 'import yaml; yaml.safe_load(...)' # parses, 9 jobs
bash scripts/check_pr_review_wiring.sh # PASS
actionlint .github/workflows/ci.yml # exit 0
Refs PMAT-1078, BSE-15, #3046
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X1cW8TnmNe3hgjragNjUxr
|
quorum-review (AD-04): three PASS — agreed {
"ticket": "PMAT-1078",
"head": "6d641b982aa7f796aad8c20e1dbd34154ad8341d",
"width": 3,
"executor": "agy",
"agreed": true,
"lanes": [
{
"lane": 1,
"verdict": "PASS",
"findings": 4
},
{
"lane": 2,
"verdict": "PASS",
"findings": 0
},
{
"lane": 3,
"verdict": "PASS",
"findings": 4
}
]
} |
What
The
pr-review-receiptjob is deleted fromci.yml, andcheck_pr_review_wiring.shinverts to keep its falsifier alive.scripts/ci_job_minutes.shis added so the runner-hours figure this rests on is re-derivable rather than asserted.This supersedes the branch's first commit, which made the job dispatch-only. A three-lane quorum and an independent re-read both found that path broken: the diff gave
PR_HEAD_SHAa|| github.shafallback and leftPR_NUMBER: ${{ github.event.pull_request.number }}untouched at three sites. Onworkflow_dispatchthere is no pull-request context, soPR_NUMBERis empty andscripts/check_pr_review_arm4.sh:372(: "${PR_NUMBER:?PR_NUMBER is required}") refuses. The job would have been reachable and non-functional — worse than absent.The query the deletion is gated on
Recorded at the deletion site in
ci.yml, not only here:The job uploaded no artifact, so no consumer could read one, and
gatehad already stopped reading it (PP-066 C0-5, PRQ-013, #2982 — a head-defined receipt job is a check the PR under review can edit).What still runs, and what does not
check_pr_review_receipt.sh,check_pr_review_counts.sh,check_pr_review_arm4.shscripts/check_*.sh, soguard_tree.sh's derived universe (git ls-files 'scripts/check_*.sh') runs them inguard-tree, whichgateneedspr-review-quorum.yml, which consumed nothing from this jobscripts/mutate-guard.sh(Arm 3),scripts/mutate_quorum_arm.sh(Arm 6), the 43-row bats table (Arm 2)check_*.sh, soguard_treedoes not reach them. Runnable by hand; the natural content of a nightlyThat loss is the 150 minutes. It is stated here and at the deletion site rather than left to be discovered.
Deviation, stated: the standing falsifier outlived its subject
R4 asserted the job's
if:polarity per event. Deleting the job removes R4's subject — and R1's and R3's, since they asserted that a job invoking the guard exists and carries a job-levelif:. Asserting the presence of a deleted job is not a weaker guard, it is a broken one, so the file inverts instead of dying with it:ci.ymljob may invoke the receipt guard. The case table carries the oldif: github.event_name == 'pull_request'wiring verbatim, asserted FAIL, and also refuses a dispatch-only job — so restoring it is a deliberate change to this rule plus a new case-table row, never an edit toci.ymlalone.paths:filter in the file that carries both required checks).scripts/check_*.sh. Without it, renaming or untracking the guard would leave R1 holding vacuously over a family that runs nowhere.if:evaluator are deleted with the job.Both polarities on every rule; R3's two rows are driven against real git trees, since the rule is about
git ls-files.The figure is now measured, and it is 3.4x larger than the claim
scripts/ci_job_minutes.shsums per JOB(completed_at - started_at)from the jobs API: runner time, not wall time (a run with four parallel jobs occupies four runners);started_at, notcreated_at, so queue wait is excluded; skipped jobs contribute nothing. It refuses rather than printing0 hwhenghis absent or the window is past run retention.326.8 h, not 96 h - and that is a floor, not the total: the run list returned exactly
--limitruns, so the window is not fully covered. The claim this PR inherited understated its own case by more than 3x. The deletion does not rest on the figure either way; it rests on the query above.Running the script is also what exposed a defect in it:
gh run list --limit Ntruncates with no signal, so a sum over a capped list reads as a total. Equality with the limit is nowexit 2 TRUNCATEDnaming a larger limit to retry with (self-test 6/6; the new row drives the real script and SKIPS with a printed line, never a silent pass, whereghcannot answer). The full-window re-derivation at--limit 1200is still running; it can only move the number up.Other checks
The first commit's
pmat workre-serialisation (369 entries rewritten, two legacy prose-id entries re-folded) is collapsed bypython3 scripts/roadmap_trim.py.Refs PMAT-1078, BSE-15, BSE-001 §4 wave 5
🤖 Generated with Claude Code
https://claude.ai/code/session_01X1cW8TnmNe3hgjragNjUxr