Skip to content

perf(ci): pr-review-receipt is dispatch-only — 96 h of fleet runner-time for a job that gates nothing (PMAT-1078) - #3046

Open
noahgift wants to merge 5 commits into
mainfrom
bse-prr-dispatch-only
Open

perf(ci): pr-review-receipt is dispatch-only — 96 h of fleet runner-time for a job that gates nothing (PMAT-1078)#3046
noahgift wants to merge 5 commits into
mainfrom
bse-prr-dispatch-only

Conversation

@noahgift

@noahgift noahgift commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

What

The pr-review-receipt job is deleted from ci.yml, and check_pr_review_wiring.sh inverts to keep its falsifier alive. scripts/ci_job_minutes.sh is added so the runner-hours figure this rests on is re-derivable rather than asserted.

This supersedes the branch's first commit, which made the job dispatch-only. A three-lane quorum and an independent re-read both found that path broken: the diff gave PR_HEAD_SHA a || github.sha fallback and left PR_NUMBER: ${{ github.event.pull_request.number }} untouched at three sites. On workflow_dispatch there is no pull-request context, so PR_NUMBER is empty and scripts/check_pr_review_arm4.sh:372 (: "${PR_NUMBER:?PR_NUMBER is required}") refuses. The job would have been reachable and non-functional — worse than absent.

The query the deletion is gated on

Recorded at the deletion site in ci.yml, not only here:

$ grep -rn 'needs:' .github/workflows/*.yml | grep pr-review-receipt      # no match
$ grep -rn 'download-artifact' .github/workflows/*.yml                    # nightly.yml:141 only (its own build assets)
$ gh api repos/paiml/aprender/branches/main/protection --jq '...'         # ["ci / gate", "workspace-test"]

The job uploaded no artifact, so no consumer could read one, and gate had already stopped reading it (PP-066 C0-5, PRQ-013, #2982 — a head-defined receipt job is a check the PR under review can edit).

What still runs, and what does not

where
check_pr_review_receipt.sh, check_pr_review_counts.sh, check_pr_review_arm4.sh still run — they are scripts/check_*.sh, so guard_tree.sh's derived universe (git ls-files 'scripts/check_*.sh') runs them in guard-tree, which gate needs
the PR's own receipt (Arm 4 live) already judged from the BASE by pr-review-quorum.yml, which consumed nothing from this job
scripts/mutate-guard.sh (Arm 3), scripts/mutate_quorum_arm.sh (Arm 6), the 43-row bats table (Arm 2) no longer run in CI — no other workflow invokes them and they are not check_*.sh, so guard_tree does not reach them. Runnable by hand; the natural content of a nightly

That loss is the 150 minutes. It is stated here and at the deletion site rather than left to be discovered.

Deviation, stated: the standing falsifier outlived its subject

R4 asserted the job's if: polarity per event. Deleting the job removes R4's subject — and R1's and R3's, since they asserted that a job invoking the guard exists and carries a job-level if:. Asserting the presence of a deleted job is not a weaker guard, it is a broken one, so the file inverts instead of dying with it:

  • R1 is now the standing falsifier: no ci.yml job may invoke the receipt guard. The case table carries the old if: github.event_name == 'pull_request' wiring verbatim, asserted FAIL, and also refuses a dispatch-only job — so restoring it is a deliberate change to this rule plus a new case-table row, never an edit to ci.yml alone.
  • R2 unchanged (no workflow-level paths: filter in the file that carries both required checks).
  • R3 is new and is R1's other half: the guard must remain a tracked scripts/check_*.sh. Without it, renaming or untracking the guard would leave R1 holding vacuously over a family that runs nowhere.
  • R4 and its if: evaluator are deleted with the job.
bash scripts/check_pr_review_wiring.sh --self-test    # SELF-TEST PASSED (10/10)
bash scripts/check_pr_review_wiring.sh                # PASS against the real ci.yml

Both polarities on every rule; R3's two rows are driven against real git trees, since the rule is about git ls-files.

The figure is now measured, and it is 3.4x larger than the claim

scripts/ci_job_minutes.sh sums per JOB (completed_at - started_at) from the jobs API: runner time, not wall time (a run with four parallel jobs occupies four runners); started_at, not created_at, so queue wait is excluded; skipped jobs contribute nothing. It refuses rather than printing 0 h when gh is absent or the window is past run retention.

$ bash scripts/ci_job_minutes.sh --since 2026-08-08 --until 2026-09-07 --event pull_request --limit 400
=== runner-hours per job - paiml/aprender, runs created 2026-08-08..2026-09-07 (400 runs, event=pull_request) ===
     hours    runs  job
     326.8     261  pr-review-receipt
     210.2     272  workspace-test
      98.6     259  guard-runner-labels
      34.3     272  ci / lint
     850.6          TOTAL

326.8 h, not 96 h - and that is a floor, not the total: the run list returned exactly --limit runs, so the window is not fully covered. The claim this PR inherited understated its own case by more than 3x. The deletion does not rest on the figure either way; it rests on the query above.

Running the script is also what exposed a defect in it: gh run list --limit N truncates with no signal, so a sum over a capped list reads as a total. Equality with the limit is now exit 2 TRUNCATED naming a larger limit to retry with (self-test 6/6; the new row drives the real script and SKIPS with a printed line, never a silent pass, where gh cannot answer). The full-window re-derivation at --limit 1200 is still running; it can only move the number up.

bash scripts/ci_job_minutes.sh --self-test    # SELF-TEST PASSED (6/6)

Other checks

bash scripts/check_roadmap_diff_additive.sh origin/main HEAD
  roadmap-diff: base=818 head=821 added=3 lifecycle=26 reserialised=0 deleted=0    PASS
bash scripts/check_receipt_gate_base_owned.sh    # exit 0
actionlint .github/workflows/ci.yml              # exit 0
python3 -c "import yaml; yaml.safe_load(open('.github/workflows/ci.yml'))"   # parses; 9 jobs; no needs: edge to the deleted job

The first commit's pmat work re-serialisation (369 entries rewritten, two legacy prose-id entries re-folded) is collapsed by python3 scripts/roadmap_trim.py.

Refs PMAT-1078, BSE-15, BSE-001 §4 wave 5

🤖 Generated with Claude Code

https://claude.ai/code/session_01X1cW8TnmNe3hgjragNjUxr

…ime for a job that gates nothing (PMAT-1078)

BSE-15 measured this job as the fleet's single largest consumer of runner-hours:
96 h of aprender PR time in the 30 days to 2026-09-07, inside a fleet that threw
away 42.1 % of everything it spent. It is 150 minutes of two mutation sweeps on a
clean-room runner, on every push to every open PR.

It gates nothing. It is not a required context (`ci / gate` and `workspace-test`
are), no job `needs:` it, and `gate` deliberately stopped reading it (PP-066 C0-5,
PRQ-013, #2982) because a head-defined receipt job is a check the PR under review
can edit. What does gate the receipt is pr-review-quorum.yml, which runs from the
BASE on pull_request_target + merge_group and invokes check_pr_review_arm4.sh
itself — it consumes no artifact from this job, so this job stopping changes no
verdict anywhere.

The job stays fully wired and runnable on demand:
    gh workflow run ci.yml --ref <branch>
On a dispatch there is no pull_request context, so PR_HEAD_SHA falls back to
github.sha; every sha-consuming step is already fail-closed and sweeps rather than
skips, so the fallback cannot turn a dispatch into a silent no-op.

check_pr_review_wiring.sh R4 moves WITH the policy rather than being routed
around: EVENTS_TRUE/EVENTS_FALSE swap, both failure messages say why, and a new
self-test row carries the old `if: github.event_name == 'pull_request'` verbatim
and asserts FAIL — so putting the job back on every PR is a red check, not a
quiet reversion. Self-test 26/26; discrimination confirmed by restoring the old
`if:` on the real ci.yml (guard exits 1, both polarities) and restoring the new
one (exit 0).

Refs PMAT-1078, BSE-15, BSE-001 §4 wave 5

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B2EenRiCc3FbvW5GJe2xFh
@github-actions

github-actions Bot commented Sep 8, 2026

Copy link
Copy Markdown

§13.11 rung 1 — quorum shadow verdict

S13-SHADOW pr=3046 head=6d641b982aa7f796aad8c20e1dbd34154ad8341d verdict=REFUSE class=Q1 arm_rc=1

Shadow mode: this records a verdict and merges nothing. A refusal
to arm is not a block (§13 adds zero rows to §7) — the pull request is
exactly as green as it was.

@noahgift

noahgift commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

quorum-review (AD-04): NOT agreed

{
 "ticket": "PMAT-1078",
 "head": "77ba7d2fd14ad6d9c33d8f97ce5c5fc8041373ad",
 "width": 3,
 "executor": "agy",
 "agreed": false,
 "lanes": [
  {
   "lane": 1,
   "verdict": "FAIL",
   "findings": 2
  },
  {
   "lane": 2,
   "verdict": "FAIL",
   "findings": 2
  },
  {
   "lane": 3,
   "verdict": "FAIL",
   "findings": 1
  }
 ]
}

@noahgift

noahgift commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

Quorum: NOT AGREED — 3 lanes, 3 FAIL (agy, width 3, head 77ba7d2f; artifact posted above). pmat-merge 3046 --auto refuses at this head, so the PR is not armed. Both findings were re-read against the branch by the orchestrator rather than taken from the lanes.

1. PR_NUMBER has no dispatch fallback — the job becomes reachable but cannot run (all three lanes; confirmed here)

The diff gives PR_HEAD_SHA a fallback and leaves its neighbour alone:

+      PR_HEAD_SHA: ${{ github.event.pull_request.head.sha || github.sha }}
       PR_NUMBER:   ${{ github.event.pull_request.number }}      # ci.yml:1860, :2185, :2337 — unchanged

On workflow_dispatch there is no pull_request context, so PR_NUMBER is empty, and scripts/check_pr_review_arm4.sh:372 is : "${PR_NUMBER:?PR_NUMBER is required (the pull request number)}" — a hard refusal on empty. So the "Still reachable / gh workflow run ci.yml --ref <branch>" section is not backed: after this change the job is dispatchable and fails at the step it exists to run. The PR body already notes arm4 "requires PR_NUMBER from CI context" and calls that pre-existing; it is pre-existing only while the job runs on pull_request, which is exactly what this PR removes.

Remedy: give PR_NUMBER a dispatch input (inputs.pr) or resolve it from github.sha, and add a self-test row asserting a dispatch run reaches arm4 with a non-empty PR number.

2. The roadmap diff is a pmat work re-serialisation — ci / gate will be RED (all three lanes; measured here)

$ bash scripts/check_roadmap_diff_additive.sh c04eda87d origin/bse-prr-dispatch-only
roadmap-diff: base=818 head=819 added=3 lifecycle=26 reserialised=369 deleted=2      # exit 1

The 2 "deleted" are legacy prose-id entries whose id strings were re-folded (Sovereign AI Integration Specification v1.1 …, Phase 2: Bayesian Examples & Documentation …), so at the YAML level nothing is lost — but G-6 (PMAT-980, #2874) refuses the churn either way, and since BSE-001 PR-A this guard runs through guard_tree.sh's universe (git ls-files 'scripts/check_*.sh') in guard-tree, which gate needs. The one real addition is PMAT-1078.

Remedy the guard itself names: python3 scripts/roadmap_trim.py, then re-run the guard.

Note on the ticket id. PMAT-1078 is this PR's, minted and InProgress. agent/R-5 also stamps it in contracts/apr-publish-cascade-v1.yaml:4; that one is the stray and is being renumbered (#2908).

…er-time for a job nothing reads (PMAT-1078)

Supersedes this branch's first commit, which made the job dispatch-only. Three
independent quorum lanes and the orchestrator each found that the dispatch path
could not work: the diff gave PR_HEAD_SHA a `|| github.sha` fallback and left
PR_NUMBER bound to `github.event.pull_request.number`, which is empty on a
dispatch, while check_pr_review_arm4.sh:372 is `${PR_NUMBER:?...}` — a hard
refusal. The job would have been reachable and broken. So it is deleted.

THE QUERY THE DELETION IS GATED ON, recorded at the deletion site in ci.yml:

  $ grep -rn 'needs:' .github/workflows/*.yml | grep pr-review-receipt   # no match
  $ grep -rn 'download-artifact' .github/workflows/*.yml                 # nightly.yml:141 only
  $ gh api repos/paiml/aprender/branches/main/protection                 # ["ci / gate", "workspace-test"]

The job also uploaded no artifact, so no consumer could read one, and `gate` had
already stopped reading it (PP-066 C0-5, PRQ-013, #2982).

WHAT STILL RUNS: check_pr_review_receipt.sh, check_pr_review_counts.sh and
check_pr_review_arm4.sh are `scripts/check_*.sh`, so guard_tree.sh's derived
universe runs them in `guard-tree`, which `gate` needs. The PR's own receipt is
judged from the BASE by pr-review-quorum.yml, which consumed nothing from here.

WHAT NO LONGER RUNS IN CI, stated rather than discovered later: the two mutation
sweeps (scripts/mutate-guard.sh, scripts/mutate_quorum_arm.sh) and the 43-row
bats fixture table. No other workflow invokes them and they are not check_*.sh,
so guard_tree does not reach them. They remain runnable by hand.

check_pr_review_wiring.sh INVERTS rather than dying with its subject. R4 (the
`if:` polarity table) and its evaluator are deleted with the job; R1 becomes the
standing falsifier — no ci.yml job may invoke the receipt guard — and R3 is new:
the guard must stay a tracked scripts/check_*.sh, or R1 would hold vacuously
over a family that runs nowhere. The case table carries the old
`if: github.event_name == 'pull_request'` wiring VERBATIM, asserted FAIL, and
also refuses a dispatch-only job, so restoring it is a change to this rule plus
a case-table row rather than an edit to ci.yml alone. 10/10, both polarities on
every rule, R3 driven against real git trees.

scripts/ci_job_minutes.sh makes the 96 h re-derivable instead of asserted: Σ per
JOB of (completed_at − started_at) from the jobs API — runner time, not wall
time; started_at, not created_at, so queue wait is excluded; skipped jobs
contribute nothing. It refuses rather than returning 0 h when gh is absent or
the window is past run retention. --self-test 5/5 offline, including the
polarity that would catch a sum that had silently used created_at.

roadmap.yaml: `python3 scripts/roadmap_trim.py` collapses the `pmat work`
re-serialisation the first commit carried (369 entries rewritten, two legacy
prose-id entries re-folded) back to base bytes — now 1 added, 0 deleted, 26
lifecycle edits.

Refs PMAT-1078, BSE-15, BSE-001 §4 wave 5, #3046

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X1cW8TnmNe3hgjragNjUxr
@noahgift

noahgift commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

quorum-review (AD-04): NOT agreed

{
 "ticket": "PMAT-1078",
 "head": "3115c74e1bd77ae4bbf019f3428b579f35179e91",
 "width": 3,
 "executor": "agy",
 "agreed": false,
 "lanes": [
  {
   "lane": 1,
   "verdict": "FAIL",
   "findings": 2
  },
  {
   "lane": 2,
   "verdict": "FAIL",
   "findings": 2
  },
  {
   "lane": 3,
   "verdict": "FAIL",
   "findings": 2
  }
 ]
}

… a total (PMAT-1078)

Found by running the script rather than reading it. The first derivation asked for
400 runs and `gh run list` returned exactly 400 — the cap bound, the window was not
fully covered, and the sum came out as a floor wearing the units of a total:

  === runner-hours per job — paiml/aprender, runs created 2026-08-08..2026-09-07 (400 runs, event=pull_request) ===
       hours    runs  job
       326.8     261  pr-review-receipt
       210.2     272  workspace-test
        98.6     259  guard-runner-labels
       850.6          TOTAL

`gh run list --limit N` truncates with no signal, so equality with the limit is the
only evidence available and it is now treated as truncation: exit 2, naming a larger
--limit to re-run with. This is the same class as the silent caps this repository
already refuses elsewhere — a number that quietly answers a smaller question than
the one asked.

Self-test 6/6. The new row drives the real script against a window whose run count
reaches --limit and requires exit 2 with TRUNCATED; it SKIPS with a printed line,
never a silent pass, where gh cannot answer.

Refs PMAT-1078, BSE-15, #3046

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X1cW8TnmNe3hgjragNjUxr
@noahgift

noahgift commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

quorum-review (AD-04): NOT agreed

{
 "ticket": "PMAT-1078",
 "head": "8a2f0656e969b23a0fd644341890d1e920aaeed5",
 "width": 3,
 "executor": "agy",
 "agreed": false,
 "lanes": [
  {
   "lane": 1,
   "verdict": "FAIL",
   "findings": 2
  },
  {
   "lane": 2,
   "verdict": "FAIL",
   "findings": 2
  },
  {
   "lane": 3,
   "verdict": "FAIL",
   "findings": 2
  }
 ]
}

…s with the policy (PMAT-1078)

Two findings from the second 3/3-FAIL quorum, both correct, both acted on.

1. THE TICKET WAS STILL THE OLD POLICY. All three lanes failed the diff for
   contradicting its own ticket: PMAT-1078 read "move the job to workflow_dispatch
   only, keep it fully wired and runnable on demand", and the diff deletes it. The
   ticket is the standard a quorum judges against, so it moves with the operator's
   rescope rather than leaving the diff arguing with it. Its acceptance criteria
   now carry the deletion, the recorded query it is gated on, the guard inversion,
   the measurement script, and the STATED LOSS of the two mutation sweeps.

2. THE ROADMAP STILL CARRIED A VOCABULARY MIGRATION, and G-6 could not see it.
   `roadmap_trim.py` collapsed the re-serialisation, but 26 unrelated entries kept
   a `pmat work` rewrite of their status VALUES — `done` -> `completed`,
   `in_progress` -> `inprogress` (19 entries), `todo` -> `planned`. G-6 classifies
   `status` as a lifecycle field, so a mass vocabulary migration is
   indistinguishable from one ticket's honest status flip and the guard passed it:
   `lifecycle=26 reserialised=0 deleted=0  PASS`. Three lanes saw it anyway.

   The file is now main's bytes plus one inserted entry, so the diff is what the
   PR is about:

     added: ['PMAT-1078']   changed: []   deleted: []
     check_roadmap_ids_unique.sh   PASS (819 ids, nested records included)
     check_roadmap_sorted.sh       PASS (sorted within each id-prefix)

   Rebuilding it this way also removed a defect the previous restore introduced:
   keying entries by their id TEXT left the re-folded twins of two legacy prose-id
   entries ("Sovereign AI Integration Specification v1.1 …", "Phase 2: Bayesian
   Examples & Documentation …") alongside main's originals — two duplicate ids that
   only the tree-parsing uniqueness guard caught, since the sorted guard reads
   PREFIX-NUMBER ids and saw nothing.

Refs PMAT-1078, BSE-15, BSE-001 §4 wave 5, #3046

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X1cW8TnmNe3hgjragNjUxr
@noahgift

noahgift commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

quorum-review (AD-04): NOT agreed

{
 "ticket": "PMAT-1078",
 "head": "bb2dca50b153ae9c3e425c2d9e47af2d1497e605",
 "width": 3,
 "executor": "agy",
 "agreed": false,
 "lanes": [
  {
   "lane": 1,
   "verdict": "FAIL",
   "findings": 1
  },
  {
   "lane": 2,
   "verdict": "FAIL",
   "findings": 1
  },
  {
   "lane": 3,
   "verdict": "FAIL",
   "findings": 1
  }
 ]
}

…iew-shadow (PMAT-1078)

Third 3/3-FAIL quorum, and all three lanes named the same leftover: the first
commit on this branch gave `PR_HEAD_SHA` a `|| github.sha` fallback at TWO sites,
and deleting the `pr-review-receipt` job removed only one of them. The other sits
in `pr-review-shadow`, a job this PR never had a reason to touch, which still runs
on `pull_request` — where the fallback is dead weight, and where its comment
("on workflow_dispatch there is no pull_request context") describes an event the
job does not run on.

It is the pattern the rescoped ticket explicitly rejects, surviving in the one
place the deletion did not reach. Reverted to main's bytes, so ci.yml's diff
against main is now the deleted job and nothing else.

  grep -c 'github.event.pull_request.head.sha || github.sha' .github/workflows/ci.yml   # 0
  python3 -c 'import yaml; yaml.safe_load(...)'                                          # parses, 9 jobs
  bash scripts/check_pr_review_wiring.sh                                                 # PASS
  actionlint .github/workflows/ci.yml                                                    # exit 0

Refs PMAT-1078, BSE-15, #3046

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X1cW8TnmNe3hgjragNjUxr
@noahgift

noahgift commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

quorum-review (AD-04): three PASS — agreed

{
 "ticket": "PMAT-1078",
 "head": "6d641b982aa7f796aad8c20e1dbd34154ad8341d",
 "width": 3,
 "executor": "agy",
 "agreed": true,
 "lanes": [
  {
   "lane": 1,
   "verdict": "PASS",
   "findings": 4
  },
  {
   "lane": 2,
   "verdict": "PASS",
   "findings": 0
  },
  {
   "lane": 3,
   "verdict": "PASS",
   "findings": 4
  }
 ]
}

@noahgift
noahgift enabled auto-merge September 8, 2026 10:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant