perf(B5): pr-review-receipt moves off pull_request to ad-hoc dispatch — 39.4% of the merge path for ONE verdict in 30 days - #3052
Open
noahgift wants to merge 4 commits into
Open
perf(B5): pr-review-receipt moves off pull_request to ad-hoc dispatch — 39.4% of the merge path for ONE verdict in 30 days#3052noahgift wants to merge 4 commits into
noahgift wants to merge 4 commits into
Conversation
… — 39.4% of the merge path for ONE verdict in 30 days (#3049) MEASURED, machines/clean-room/ci-occupancy.sh, 30 days of aprender PR-event runs (302-run newest-first sample): pr-review-receipt 76 runs 95.7 min median 112.88 runner-h 39.4% -------------------------------------------------- of ALL PR-event hours cancelled n=71 (110.83 h) success n=1 (2.05 h) cancelled durations (min): p25=34.1 med=95.7 p75=150.3 p90=150.4 max=151.1 PR-event CI costs 286.56 runner-hours over that window and this job is 112.88 of them. It returned ONE verdict. The cancels are not all supersedes. p75 sits ON the declared timeout-minutes: 150, and GitHub reports a timeout kill as `cancelled`, not `failed` — so 32 of the 71 ran the full 2.5 hours holding a clean-room slot and were then killed. ~80 runner-hours a month of pure timeout on a 16-slot fleet. PRREV-015's own comment above that timeout predicted exactly this — "a job that times out the first time the runner is busy, and a cancelled job reads as a failure nobody can distinguish from a real one" — and this is that prediction coming true 32 times a month. A sweep that completes 1 run in 76 is not mutation coverage. It is the "0 violations over 0 files" shape aimed at this repo's own mutation gate. THE THREE QUERIES, recorded at the change site rather than in a commit nobody re-reads: 1. required? the RULESET (not the classic branch-protection API, which names a DIFFERENT job) says ["gate","workspace-test"]. Not this one. 2. needs:-ed? 7 grep hits across every workflow on origin/main; six are comments, the seventh is its own job key. Nothing consumes it. 3. covered? yes, and for 5% of the price: `present` in pr-review-quorum.yml (pull_request_target, base-owned) is 98 runs at a 1.7-min median = 5.93 h, and pr-review-sign still signs on every PR. What moves is the mutation sweep OVER those guards, nothing else. MOVED, NOT DELETED: deleting removes mutation coverage of the receipt guards. Moving lets it finish. THE TRAP THIS AVOIDS. PR_HEAD_SHA is REMOVED from the job-level env. On workflow_dispatch there is no `github.event.pull_request`, so it would be the empty string — and the sweep does BASE=$(git merge-base origin/main "${PR_HEAD_SHA}" 2>/dev/null) || BASE="" which TOLERATES an empty base. The job would have swept nothing and read green: the exact defect class this change is motivated by, reintroduced by the change itself. A first step resolves the head from the PR number, asserts a 40-char sha, asserts it is fetchable, and REFUSES otherwise. It runs before both consumers. gh workflow run ci.yml -f pr=<number> actionlint clean. Refs #3049, PMAT-1091. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EeUX4ymyt8XhjXbtTuNH8L
Contributor
Author
|
quorum-review (AD-04): three PASS — agreed {
"ticket": "PMAT-1091",
"head": "0e8dd9df80b5beec44ce621e967f8acd4c364cff",
"width": 3,
"executor": "agy",
"agreed": true,
"lanes": [
{
"lane": 1,
"verdict": "PASS",
"findings": 0
},
{
"lane": 2,
"verdict": "PASS",
"findings": 0
},
{
"lane": 3,
"verdict": "PASS",
"findings": 0
}
]
} |
Three independent agy lanes judged the diff against PMAT-1091. AGREED: lane 1=PASS, lane 2=PASS, lane 3=PASS, no findings. All three independently confirmed the part that matters — that the new resolve step runs BEFORE both PR_HEAD_SHA consumers and refuses rather than letting an empty sha reach a merge-base that tolerates an empty base. Refs #3049, PMAT-1091. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EeUX4ymyt8XhjXbtTuNH8L
noahgift
enabled auto-merge
September 8, 2026 12:43
|
§13.11 rung 1 — quorum shadow verdict Shadow mode: this records a verdict and merges nothing. A refusal |
…ly where a PR subject exists" (#3049) guard-tree caught the ci.yml change, correctly, and the diagnosis is better than a parser gap: FAIL R4: this guard cannot evaluate `github.event_name == 'workflow_dispatch' && inputs.pr != ''` R4 asserted a fixed pair — TRUE on pull_request, FALSE on push, merge_group and workflow_dispatch — with the reason stated in its own message: "There is no PR number on this event". That was TRUE until this PR gave workflow_dispatch a `pr` input. A rule whose premise has changed is worse than no rule: it refuses the one wiring that makes the receipt path well-defined. So R4 now states the property directly, as a QUANTIFIER over events rather than a list: every event the `if:` selects must carry a subject for evidence/pr-review/<pr>/<sha>/, and it must select at least one. pull_request always (the pull_request payload) workflow_dispatch iff the `if:` REQUIRES a non-empty input AND that input is DECLARED under on.workflow_dispatch.inputs push, merge_group never, on any wiring TWO HOLES THE CASE TABLE FOUND, neither of which I set out to fix: * An undeclared input is ALWAYS empty, so `inputs.prr != ''` naming an input the workflow does not declare is a job that never runs and reports nothing — dark, and indistinguishable from a deliberate disable. Now RED by name. * `inputs.*` is populated on workflow_dispatch and EMPTY on every other event, so `github.event_name == 'pull_request' && inputs.pr != ''` never runs on anything. Shape B is admissible only with the workflow_dispatch literal; anything else is refused rather than evaluated. AND ONE IN MY OWN FIRST DRAFT, which is why the property is written as a quantifier: I initially wrote a want-per-event table with `pull_request -> runs` hardcoded, reintroducing the exact staleness I was removing. It refused the dispatch wiring it existed to admit. The evaluator is still narrow and still refuses rather than guesses — shape B is a SEPARATE pattern, not a widened regex, per this guard's own header: "Extending the evaluator is a code change with a new case-table row, not an accident." 29/29 rows (was 25), four new, one per FORM VARIANT. Three mutations, each caught by the row built for it: drop the declared-input check -> row 17 allow shape B on any literal -> row 18 dispatch with no required input OK -> row 16 Refs #3049, PMAT-1091. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EeUX4ymyt8XhjXbtTuNH8L
Contributor
Author
|
quorum-review (AD-04): three PASS — agreed {
"ticket": "PMAT-1091",
"head": "80ea9ce0780fc7a2ac0ae62602350b524182e774",
"width": 3,
"executor": "agy",
"agreed": true,
"lanes": [
{
"lane": 1,
"verdict": "PASS",
"findings": 0
},
{
"lane": 2,
"verdict": "PASS",
"findings": 0
},
{
"lane": 3,
"verdict": "PASS",
"findings": 0
}
]
} |
…l merge The first verdict judged a tree without the check_pr_review_wiring.sh change, so it was void the moment guard-tree caught R4 and the fix landed. Re-run against the actual head: lane 1=PASS, lane 2=PASS, lane 3=PASS. Refs #3049, PMAT-1091. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EeUX4ymyt8XhjXbtTuNH8L
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ticket — PMAT-1091, issue #3049 (
bse-001). Claim — B5: the largest measured waste on the merge path, moved so it can actually finish.Measured before (basis=)
machines/clean-room/ci-occupancy.sh, 30 days of aprender PR-event runs, 302-run newest-first sample. PR-event CI costs 286.56 runner-hours:What those 112.88 hours bought:
One verdict in thirty days. And the cancels are not all supersedes: p75 sits on the declared
timeout-minutes: 150, and GitHub reports a timeout kill ascancelled, notfailed— so 32 of 71 ran the full 2.5 hours holding a clean-room slot and were then killed. ~80 runner-hours a month of pure timeout on a 16-slot fleet.PRREV-015's own comment above that timeout predicted it: "a job that times out the first time the runner is busy, and a cancelled job reads as a failure nobody can distinguish from a real one." This is that, 32 times a month.
Change
if: github.event_name == 'pull_request'→workflow_dispatch && inputs.pr != '', plus aprinput onworkflow_dispatch.Moved, not deleted. Deleting removes mutation coverage of the receipt guards. Moving lets it finish.
The trap this change would otherwise have walked into
PR_HEAD_SHAis removed from the job-levelenv. Onworkflow_dispatchthere is nogithub.event.pull_request, so it would be the empty string — and the sweep does:which tolerates an empty base. The job would have swept nothing and read green: the exact defect class this PR is motivated by, reintroduced by the fix. A first step resolves the head from the PR number, asserts a 40-char sha, asserts it is fetchable, and refuses otherwise. It runs before both
PR_HEAD_SHAconsumers (verified by line order: resolve at job-relative 61, consumers at 201 and 214).Queries at the change site
All three are written into
ci.ymlabove theif:, not left in a commit message:gh api repos/paiml/aprender/rules/branches/main→["gate","workspace-test"]. Not this job. (The classic branch-protection API saysci / gate— a different job. The ruleset is what blocks.)needs:-ed? grep over every workflow onorigin/main→ 7 hits, six are comments, the seventh is its own job key. Nothing consumes it.presentinpr-review-quorum.yml(pull_request_target, base-owned) is 98 runs, 1.7-min median, 5.93 h, andpr-review-signstill signs on every PR. What moves is the mutation sweep over those guards.Predicted after
~112.9 runner-hours per 30 days off the PR merge path, ~80 h of it pure timeout. Second-largest leg (
workspace-test, 68.0 h) becomes the top row and the next measurement.Writes
.github/workflows/ci.yml(one job's trigger + one new step + the input),docs/roadmaps/roadmap.yaml(+19/−0,added=1 reserialised=0 deleted=0, guard PASS). No script, no crate, no PP-066 row.actionlint clean.
🤖 Generated with Claude Code
https://claude.ai/code/session_01EeUX4ymyt8XhjXbtTuNH8L