From 48e0aea1faa3c5750f1d34690e39a623449a22c4 Mon Sep 17 00:00:00 2001 From: REPPL <77722411+REPPL@users.noreply.github.com> Date: Mon, 28 Sep 2026 16:14:49 +0100 Subject: [PATCH 1/5] =?UTF-8?q?chore:=20capture=20iss-2609281514435020=20?= =?UTF-8?q?=E2=80=94=20the=20merge=20queue=20cancels=20a=20passing=20macOS?= =?UTF-8?q?=20check=20at=2030=20minutes?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit #728, #730 (twice) and #733 were cancelled at the cap while their code was fine; ruling Z (2026-09-28) raises both limits to 45 minutes and asks for a warning at 35. Refs: iss-2609281514435020 Assisted-by: Claude:claude-opus-5-5 --- ...eue-fails-a-group-whose-macos-check-job-runs.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100644 .abcd/work/issues/open/iss-2609281514435020-the-merge-queue-fails-a-group-whose-macos-check-job-runs.md diff --git a/.abcd/work/issues/open/iss-2609281514435020-the-merge-queue-fails-a-group-whose-macos-check-job-runs.md b/.abcd/work/issues/open/iss-2609281514435020-the-merge-queue-fails-a-group-whose-macos-check-job-runs.md new file mode 100644 index 000000000..2eb34d09b --- /dev/null +++ b/.abcd/work/issues/open/iss-2609281514435020-the-merge-queue-fails-a-group-whose-macos-check-job-runs.md @@ -0,0 +1,14 @@ +--- +schema_version: 1 +id: "iss-2609281514435020" +slug: "the-merge-queue-fails-a-group-whose-macos-check-job-runs" +severity: "minor" +category: "process" +source: "review-followup" +found_during: "autonomous run A resumed 2026-09-25: ruling Z" +origin: researcher-authored +production_mode: hand-written +found_at: ".github/workflows/ci.yml" +--- + +The merge queue fails a group whose macOS check job runs past 30 minutes, cancelling pull requests whose code is fine: #728 was cancelled at 30.3 minutes, #730 twice at the cap, and #733 once. The check job's timeout-minutes and the main ruleset's merge-queue check_response_timeout_minutes are both 30, while the macOS leg of a source change measures 23 to 30 minutes, so ordinary runner variance cancels a passing run, and nothing warns before the cancellation arrives. Ruling Z (2026-09-28, the technical facilitator) raises both limits to 45 minutes and asks for a warning once the macOS check passes 35 minutes, so a speed lane opens before any cancellation. From b248cb647b4bfde2d729bad79ebe08b9f9377d8d Mon Sep 17 00:00:00 2001 From: REPPL <77722411+REPPL@users.noreply.github.com> Date: Mon, 28 Sep 2026 16:18:59 +0100 Subject: [PATCH 2/5] ci: the macOS check and the merge-queue cap go to 45 minutes, with a warning at 35 Ruling Z (2026-09-28, the technical facilitator) raises the macOS leg of the check job and the ruleset's check_response_timeout_minutes from 30 to 45: passing macOS runs of 23 to 30 minutes were being cancelled at the cap. The job's timeout-minutes is one matrix expression, so the ubuntu leg keeps 30; the ruleset mirror records 45. The live ruleset is changed after this merges, by the run's orchestrator with gh api. The cancel policy adds a step at the end of the macOS leg that warns, in the log and the step summary, once the check has run past 35 minutes, naming the rerun-once rule and the speed lane. It runs under always(), carries continue-on-error, and reads the start epoch from a first step's output through env, never an expression inside run:. TestRaceLaneBudgetIsDeclaredAndFitsItsJob reads the ceiling leg by leg and holds each at or below the mirror's cap (watched red with the macOS leg at 45 against a mirror of 30); TestCheckJobWarnsBeforeTheQueueCap pins the threshold at 35, below the leg's ceiling and the cap, and the step's shape (watched red before the step existed). Refs: iss-2609281514435020 Assisted-by: Claude:claude-opus-5-5 --- .abcd/work/rulesets/main-protection.json | 2 +- .github/workflows/ci.yml | 78 +++++++-- internal/core/lint/racelanebudget_test.go | 158 ++++++++++++++++-- .../surface/cli/history_capture_cap_test.go | 2 +- 4 files changed, 209 insertions(+), 31 deletions(-) diff --git a/.abcd/work/rulesets/main-protection.json b/.abcd/work/rulesets/main-protection.json index 45c51d777..65492bea8 100644 --- a/.abcd/work/rulesets/main-protection.json +++ b/.abcd/work/rulesets/main-protection.json @@ -55,7 +55,7 @@ }, { "parameters": { - "check_response_timeout_minutes": 30, + "check_response_timeout_minutes": 45, "grouping_strategy": "ALLGREEN", "max_entries_to_build": 5, "max_entries_to_merge": 5, diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index a96ff0aa9..a35debfb2 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -223,24 +223,39 @@ jobs: check: needs: changes if: ${{ !cancelled() }} - # Equal to the merge queue's check response timeout (30 minutes, recorded in - # .abcd/work/rulesets/main-protection.json), which is the outer ceiling on a - # merge-group run: the queue fails the group when this required check has - # not concluded by then, so a job ceiling above it is unreachable there. The - # race step's -timeout lifts go test's 10m per-package default, so a slow - # package runs on to this ceiling instead of failing at ten minutes; it does - # not fit inside the ceiling on the macOS leg, which spends about 18m before - # the slowest package starts, so a hang there is cancelled without a - # goroutine dump. Raising the queue's cap is an admin act on the live - # ruleset, left to the technical facilitator; this job may follow it then. - # TestRaceLaneBudgetIsDeclaredAndFitsItsJob holds this at or below the cap. - timeout-minutes: 30 + # The macOS leg's ceiling is equal to the merge queue's check response + # timeout (45 minutes, recorded in .abcd/work/rulesets/main-protection.json), + # which is the outer ceiling on a merge-group run: the queue fails the group + # when this required check has not concluded by then, so a job ceiling above + # it is unreachable there. Both are 45 under ruling Z (2026-09-28, the + # technical facilitator), because a 30-minute cap cancelled passing macOS + # runs of 23 to 30 minutes (iss-2609281514435020). Raising either again is + # an admin act on the live ruleset, left to the technical facilitator, and + # the remedy for a leg near the cap is a faster suite. The ubuntu leg runs + # well inside 30 minutes and keeps that ceiling. The race step's -timeout lifts go test's 10m + # per-package default, so a slow package runs on to this ceiling instead of + # failing at ten minutes; it does not fit inside the ceiling on the macOS + # leg, which spends about 18m before the slowest package starts, so a hang + # there is cancelled without a goroutine dump. + # TestRaceLaneBudgetIsDeclaredAndFitsItsJob holds every leg at or below the + # cap, and TestCheckJobWarnsBeforeTheQueueCap holds the warning step at the + # end of the job below it. + timeout-minutes: ${{ matrix.os == 'macos-latest' && 45 || 30 }} strategy: fail-fast: false matrix: os: [ubuntu-latest, macos-latest] runs-on: ${{ matrix.os }} steps: + # The clock for the warning step at the end of the job. It is the first + # step, so the time the runner spent before it (queueing, set-up) is not + # counted, and the elapsed time the warning reads is a lower bound on + # what the merge queue's clock shows. + - name: Record the job's start + id: started + if: matrix.os == 'macos-latest' + run: echo "epoch=$(date +%s)" >>"$GITHUB_OUTPUT" + # Full history: the record-lint step below arms `-agent-diff` against the # base commit, and a three-dot range needs the base object AND a merge # base, neither of which a depth-1 clone holds — with the default shallow @@ -301,9 +316,9 @@ jobs: # unrelated pull request from the merge queue (iss-2609260319483365). 20m # is about twice the slowest passing run, so a real hang still fails with # a goroutine dump where the enclosing ceiling leaves room for it: in - # release.yml's verify job it does, while this job's 30 minutes, held to - # the merge queue's response cap, are the binding ceiling here (see the - # job header). The Makefile's preflight recipe and release.yml's verify + # release.yml's verify job it does, while this job's own ceiling, held to + # the merge queue's response cap, is the binding one here (see the job + # header). The Makefile's preflight recipe and release.yml's verify # job carry the same flag; TestRaceLaneBudgetIsDeclaredAndFitsItsJob # holds all three. - name: Test (race, internal) @@ -392,6 +407,39 @@ jobs: go run ./cmd/abcd site build --out "$RUNNER_TEMP/site-render-check" go run ./cmd/abcd lint site --out "$RUNNER_TEMP/site-render-check" + # Warn before the cap cancels (ruling Z's cancel policy, 2026-09-28). A + # macOS check that runs past WARN_AFTER_MINUTES is ten minutes from the + # merge queue's cap, and a cancellation there carries nothing but the + # cancellation. The rule for a cancelled check is to rerun it once; a + # second cancellation for the same reason stops that pull request and + # opens a speed lane, never another raise of the timeout. This warning + # moves that signal ahead of the first cancellation. It never fails the + # job: `always()` runs it however the steps before it ended, and + # `continue-on-error` holds even a broken script to a warning. + # TestCheckJobWarnsBeforeTheQueueCap pins the threshold below the leg's + # ceiling. + - name: Warn when the check nears the queue's cap + if: always() && matrix.os == 'macos-latest' + continue-on-error: true + env: + STARTED_AT: ${{ steps.started.outputs.epoch }} + WARN_AFTER_MINUTES: 35 + run: | + case "${STARTED_AT:-}" in + '' | *[!0-9]*) + echo "::notice::no job start was recorded, so the elapsed-time warning did not run" + exit 0 + ;; + esac + elapsed=$(($(date +%s) - STARTED_AT)) + minutes=$((elapsed / 60)) + echo "check (macos-latest) has run ${minutes}m $((elapsed % 60))s; the warning line is ${WARN_AFTER_MINUTES}m" + if [ "$elapsed" -gt $((WARN_AFTER_MINUTES * 60)) ]; then + msg="check (macos-latest) ran ${minutes} minutes, past the ${WARN_AFTER_MINUTES}-minute warning line and close to the merge queue's cap. If it is cancelled at the cap, rerun it once; a second cancellation for the same reason stops the pull request: open a speed lane to make the suite faster, and do not raise the timeout." + echo "::warning title=macOS check near the queue's cap::${msg}" + printf '### macOS check near the merge queue cap\n\n%s\n' "$msg" >>"$GITHUB_STEP_SUMMARY" + fi + # Secrets never land in history: the history each run would land is scanned — # a pull request's own commits, and the full history of the pushed or queued # commit — never every ref the checkout fetched. Run as a diff --git a/internal/core/lint/racelanebudget_test.go b/internal/core/lint/racelanebudget_test.go index b2adecda8..5aacd4fe8 100644 --- a/internal/core/lint/racelanebudget_test.go +++ b/internal/core/lint/racelanebudget_test.go @@ -27,13 +27,16 @@ import ( // queue fails the group when a required check has not concluded within the // ruleset's check_response_timeout_minutes. That cap is the outer ceiling on // the event the lane was ejected on, and a job ceiling above it is unreachable -// there: the queue gives up first, with no job failure to read. So the check -// job's timeout-minutes may not exceed the cap the ruleset mirror records. The -// step's -timeout still earns its place under that cap: it lifts the ten-minute -// per-package default, so a slow package fails on the job's clock rather than -// at ten minutes. Raising the queue's cap is an admin act on the live ruleset, -// left to the technical facilitator; this test reads the mirror, so the day the -// cap is raised the check job may follow it. +// there: the queue gives up first, with no job failure to read. So no leg of +// the check job may carry a timeout-minutes above the cap the ruleset mirror +// records. Under ruling Z (2026-09-28, the technical facilitator) the cap is 45 +// minutes and the macOS leg follows it, while the ubuntu leg keeps 30; the job +// declares that as one matrix expression, which checkLegTimeouts reads leg by +// leg. The step's -timeout still earns its place under that cap: it lifts the +// ten-minute per-package default, so a slow package fails on the job's clock +// rather than at ten minutes. Raising the queue's cap is an admin act on the +// live ruleset, left to the technical facilitator; this test reads the mirror, +// so a job ceiling can rise only after the mirror records the raise. // // release.yml's verify job is not a merge-queue job, so its own timeout-minutes // is its ceiling. A package timeout that reaches past it never fires: the @@ -75,14 +78,15 @@ func TestRaceLaneBudgetIsDeclaredAndFitsItsJob(t *testing.T) { if check, _, ok := raceJob(".github/workflows/ci.yml", "check"); ok { const where = ".github/workflows/ci.yml job check" - capMin := jobTimeoutMinutes(t, where, check) queue := mergeQueueResponseTimeout(t, root) - if capMin > queue { - t.Errorf("%s: timeout-minutes is %s, above the merge queue's %s check response "+ - "timeout in %s; on a merge-group run the queue fails the group first, so the "+ - "job ceiling is unreachable there and any budget sized against it is false. "+ - "Raise the live ruleset's cap (an admin act) before the job's", - where, capMin, queue, rulesetMirror) + for _, leg := range checkLegTimeouts(t, where, check) { + if leg.minutes > queue { + t.Errorf("%s: the %s leg's timeout-minutes is %s, above the merge queue's %s check "+ + "response timeout in %s; on a merge-group run the queue fails the group first, "+ + "so the job ceiling is unreachable there and any budget sized against it is "+ + "false. Raise the live ruleset's cap (an admin act) before the job's", + where, leg.os, leg.minutes, queue, rulesetMirror) + } } } @@ -102,6 +106,132 @@ func TestRaceLaneBudgetIsDeclaredAndFitsItsJob(t *testing.T) { } } +// checkWarnAfterMinutes is where the macOS leg of ci.yml's check job warns that +// it is nearing its ceiling: ruling Z's cancel policy (2026-09-28) sets it at 35 +// minutes, ten under the 45-minute cap, so a speed lane opens before the queue +// cancels anything. +const checkWarnAfterMinutes = 35 + +// checkWarnStep is the name of that warning step in the check job. +const checkWarnStep = "Warn when the check nears the queue's cap" + +// The macOS leg warns before the merge queue's cap cancels it (ruling Z's +// cancel policy, 2026-09-28). A cancellation at the cap carries nothing but the +// cancellation, and the rule for it is rerun once, then stop that pull request +// and open a speed lane; the warning moves that signal ahead of the first +// cancellation. It is a warning and nothing else: it may not fail the job, it +// runs however the steps before it ended, and its threshold sits below the +// leg's ceiling, or it could never fire before the cancellation it announces. +func TestCheckJobWarnsBeforeTheQueueCap(t *testing.T) { + root := filepath.Join("..", "..", "..") + const where = ".github/workflows/ci.yml job check" + job, ok := workflowJobBlock(readRepoFile(t, root, ".github/workflows/ci.yml"), "check") + if !ok { + t.Fatalf("%s: no such job; the parser or the workflow changed shape", where) + } + step, ok := workflowStepBlock(job, checkWarnStep) + if !ok { + t.Fatalf("%s: no %q step; the macOS leg would reach the merge queue's cap with no "+ + "warning ahead of the cancellation", where, checkWarnStep) + } + + m := regexp.MustCompile(`(?m)^\s+WARN_AFTER_MINUTES: "?(\d+)"?\s*$`).FindStringSubmatch(step) + if m == nil { + t.Fatalf("%s step %q: no WARN_AFTER_MINUTES in its env", where, checkWarnStep) + } + warn, _ := strconv.Atoi(m[1]) + if warn != checkWarnAfterMinutes { + t.Errorf("%s step %q warns after %d minutes; the cancel policy of ruling Z sets %d", + where, checkWarnStep, warn, checkWarnAfterMinutes) + } + warnAfter := time.Duration(warn) * time.Minute + for _, leg := range checkLegTimeouts(t, where, job) { + if leg.os == "macos-latest" && warnAfter >= leg.minutes { + t.Errorf("%s step %q warns after %s, not below the macos-latest leg's %s ceiling; "+ + "the job is cancelled before the warning can fire", where, checkWarnStep, warnAfter, leg.minutes) + } + } + if queue := mergeQueueResponseTimeout(t, root); warnAfter >= queue { + t.Errorf("%s step %q warns after %s, not below the merge queue's %s cap in %s", + where, checkWarnStep, warnAfter, queue, rulesetMirror) + } + + ifLine := regexp.MustCompile(`(?m)^\s+if: (.*)$`).FindStringSubmatch(step) + if ifLine == nil || !strings.Contains(ifLine[1], "always()") || !strings.Contains(ifLine[1], "macos-latest") { + t.Errorf("%s step %q: want an `if:` naming always() and the macos-latest leg, so it runs "+ + "on that leg however the steps before it ended", where, checkWarnStep) + } + if !regexp.MustCompile(`(?m)^\s+continue-on-error: true\s*$`).MatchString(step) { + t.Errorf("%s step %q: want `continue-on-error: true`; a warning may never fail the job", where, checkWarnStep) + } + at := strings.Index(step, "run:") + if at < 0 { + t.Fatalf("%s step %q has no run: script", where, checkWarnStep) + } + run := step[at:] + for _, want := range []string{"::warning", "GITHUB_STEP_SUMMARY", "rerun", "speed lane"} { + if !strings.Contains(run, want) { + t.Errorf("%s step %q: its script never mentions %q; the warning names the rerun-once "+ + "rule and the speed lane, in the log and the step summary", where, checkWarnStep, want) + } + } + if strings.Contains(run, "${{") { + t.Errorf("%s step %q interpolates an expression inside run:; pass values through env", where, checkWarnStep) + } +} + +// checkLeg is one matrix leg's job ceiling. +type checkLeg struct { + os string + minutes time.Duration +} + +// checkLegTimeouts reads the check job's `timeout-minutes:` per matrix leg. The +// value is a plain number, which every leg shares, or the one expression +// `${{ matrix.os == '' && || }}`, which gives its own ceiling +// and every other leg . Legs come from the job's `os: [...]` matrix line. +func checkLegTimeouts(t *testing.T, where, job string) []checkLeg { + t.Helper() + osLine := regexp.MustCompile(`(?m)^\s+os: \[([^\]]*)\]\s*$`).FindStringSubmatch(job) + if osLine == nil { + t.Fatalf("%s: no `os: [...]` matrix line", where) + } + var legs []checkLeg + for _, name := range strings.Split(osLine[1], ",") { + legs = append(legs, checkLeg{os: strings.TrimSpace(name)}) + } + minutes := func(s string) time.Duration { + n, err := strconv.Atoi(s) + if err != nil { + t.Fatalf("%s: timeout-minutes %q: %v", where, s, err) + } + return time.Duration(n) * time.Minute + } + if m := regexp.MustCompile(`(?m)^ timeout-minutes: (\d+)\s*$`).FindStringSubmatch(job); m != nil { + for i := range legs { + legs[i].minutes = minutes(m[1]) + } + return legs + } + m := regexp.MustCompile(`(?m)^ timeout-minutes: \$\{\{ matrix\.os == '([a-z0-9.-]+)' && (\d+) \|\| (\d+) \}\}\s*$`).FindStringSubmatch(job) + if m == nil { + t.Fatalf("%s: timeout-minutes is neither a number nor "+ + "`${{ matrix.os == '' && || }}`", where) + } + named := false + for i := range legs { + if legs[i].os == m[1] { + legs[i].minutes, named = minutes(m[2]), true + } else { + legs[i].minutes = minutes(m[3]) + } + } + if !named { + t.Fatalf("%s: timeout-minutes names %q, which is not a leg of the matrix", where, m[1]) + } + return legs +} + // rulesetMirror is the tree's record of the live branch ruleset on main. const rulesetMirror = ".abcd/work/rulesets/main-protection.json" diff --git a/internal/surface/cli/history_capture_cap_test.go b/internal/surface/cli/history_capture_cap_test.go index 03e671d05..a22f768f7 100644 --- a/internal/surface/cli/history_capture_cap_test.go +++ b/internal/surface/cli/history_capture_cap_test.go @@ -31,7 +31,7 @@ import ( // detector has nothing of this package's to watch, and instrumenting // those passes cost 125s against 6s uninstrumented on the same machine: a third // of this package's race time, on the macOS leg whose job is held to the merge -// queue's 30-minute cap (iss-2609261924541555). The uninstrumented lane asserts +// queue's 45-minute cap (iss-2609261924541555). The uninstrumented lane asserts // the same cap on both CI legs. func TestHistoryCaptureAcceptsWhatTheHooksAccept(t *testing.T) { if raceEnabled { From 27a1f97a0309df1efb47a0443625837990532535 Mon Sep 17 00:00:00 2001 From: REPPL <77722411+REPPL@users.noreply.github.com> Date: Mon, 28 Sep 2026 16:19:01 +0100 Subject: [PATCH 3/5] docs(decisions): record rulings Z, AR and the cancel policy of 2026-09-28 The user, as technical facilitator, at 15:10:37Z: both limits to 45 minutes (the one amendment of the never-raise rule), all three CI speed-ups built, and a warning at 35 minutes beside the rerun-once rule. Assisted-by: Claude:claude-opus-5-5 --- .abcd/work/DECISIONS.md | 1 + 1 file changed, 1 insertion(+) diff --git a/.abcd/work/DECISIONS.md b/.abcd/work/DECISIONS.md index 3202ff7d9..ac10c1319 100644 --- a/.abcd/work/DECISIONS.md +++ b/.abcd/work/DECISIONS.md @@ -2567,3 +2567,4 @@ together (the script's header says why there is no escape hatch). - 2026-09-25 — itd-2609211913453478's acceptance criterion 4 ships under two readings the intent's scope line does not state. A glossary entry's `not_to_be_confused_with` passes when at least one member names a family row on the record-families page or the page itself, where the scope line says the field "may name only a family on the page"; the stricter reading would force nonsense pairs such as warm against intent, and the entries keep their real confusion pairs. The family-key rule (`record_family_key`, warn) reports a record frontmatter key only when the glossary already marks that word superseded or forbidden, so a brand-new grouping word with no row (the intent's own Mechanism case, e.g. an `initiative:` key) is not detected by construction, and the stores the page does not row (adr, rdi, dsp, rdg, adm, srp) are not reported. The six `grandfathered_at_phase` warnings on itd-20, 27, 28, 63, 69 and 72 are history and stay. Recorded for the product thinker to confirm or widen (autonomous run A, glossary lane review, orchestrator abcd-39). - 2026-09-26 — The lab store is keyed `~/.abcd/lab///`, with one `index.jsonl` registry per root-sha lane beside the lab homes (lane implementer, autonomous run A, on review-lab's third finding against spc-2609212141418943 for itd-2609212137128014). This supersedes two recorded texts: the spec's literal `~/.abcd/lab//` (scope item 1), and the 2026-08-31 lab-convention entry's hand-run keying `~/.abcd/lab/-/` with a single top-level `~/.abcd/lab/index.jsonl`, whose stated divergence from root-sha keying is withdrawn. Why: the intent's scope condition keys the store "as the other machine-scoped stores are", and the worktree and transcript stores key on the repository's root commit, because a checkout moves, is renamed and is cloned twice on one machine while its root commit does none of that; a lab's identity is still its intention, carried by its id `lab--` (the UTC mint time and the pin), so several labs share one baseline inside one lane. The hand-run labs that predate the verb stay where they are, beside the root-sha lanes, and the verb neither reads nor writes them or the top-level registry, so no real lab is moved or migrated by the change. A later text naming `~/.abcd/lab//` (the open spc-2609221011151661's `pairs.jsonl` among them) means the lab home inside its root-sha lane. - 2026-09-26 — The build loop's worktree store is keyed on the FULL root sha: a lane lives at `~/.abcd/worktrees//-` with the 40-hex root commit, the form the history, transcript and voyage stores use and the one the store's draft (itd-2609091014076309) specifies. The lanes of autonomous run A made by hand under the abbreviated key (`~/.abcd/worktrees/488a0aa9//`) are the pre-verb convention, not a second form of the store: the loop never reads or adopts a lane under that key, and those worktrees are retired with `git worktree remove` like any other (implementer of fix round fix2-loop2, autonomous run A, on item 4 of the loop2 review; spc-2609202134338445 piece 6). +- 2026-09-28 — Rulings Z, AR and the cancel policy, given by the user as technical facilitator at 15:10:37Z (autonomous run A, recorded by lane cap45 for orchestrator abcd-a8). (Z) The macOS leg of ci.yml's `check` job and the main ruleset's merge-queue `check_response_timeout_minutes` both rise from 30 to 45 minutes, because a 30-minute cap cancelled passing macOS runs (#728, #730 twice, #733; iss-2609281514435020). This amends the standing rule that never raises the check job's timeout or edits the ruleset, for exactly this one change: 45 minutes, those two settings; every other timeout, every required-check name or split and every other ruleset field stays as it is, and the ubuntu leg keeps 30. The workflow and the `.abcd/work/rulesets/main-protection.json` mirror change through a reviewed pull request; the live ruleset is changed with `gh api` by the run's orchestrator after that pull request merges, and until then the live queue still fails a group at 30 minutes. (AR) All three speed-ups of iss-2609261924541555 are built: a test-only switch that skips the disk flush in the atomic write code and the slowest -race package first in the race step (lane ciSpeed), and the scanner's per-identity git calls folded into one (lane scanFold), a trust path that is security-reviewed before it lands. (Cancel policy) The rerun-once rule stands: a check cancelled at the cap is rerun once, and a second cancellation for the same reason stops that pull request and opens a speed lane, never another raise of the timeout; and a step on the macOS leg warns, in the log and the step summary, once the check has run past 35 minutes, without ever failing the job, so a speed lane opens before any cancellation. From 75c5a02ca9290c88d278dc6e538dc902921e5584 Mon Sep 17 00:00:00 2001 From: REPPL <77722411+REPPL@users.noreply.github.com> Date: Mon, 28 Sep 2026 16:19:12 +0100 Subject: [PATCH 4/5] =?UTF-8?q?chore:=20resolve=20iss-2609281514435020=20?= =?UTF-8?q?=E2=80=94=20the=20macOS=20check=20and=20the=20queue=20cap=20are?= =?UTF-8?q?=2045=20minutes,=20with=20a=20warning=20at=2035?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Resolves: iss-2609281514435020 Assisted-by: Claude:claude-opus-5-5 --- ...ails-a-group-whose-macos-check-job-runs.md | 14 ------------ ...ails-a-group-whose-macos-check-job-runs.md | 22 +++++++++++++++++++ 2 files changed, 22 insertions(+), 14 deletions(-) delete mode 100644 .abcd/work/issues/open/iss-2609281514435020-the-merge-queue-fails-a-group-whose-macos-check-job-runs.md create mode 100644 .abcd/work/issues/resolved/iss-2609281514435020-the-merge-queue-fails-a-group-whose-macos-check-job-runs.md diff --git a/.abcd/work/issues/open/iss-2609281514435020-the-merge-queue-fails-a-group-whose-macos-check-job-runs.md b/.abcd/work/issues/open/iss-2609281514435020-the-merge-queue-fails-a-group-whose-macos-check-job-runs.md deleted file mode 100644 index 2eb34d09b..000000000 --- a/.abcd/work/issues/open/iss-2609281514435020-the-merge-queue-fails-a-group-whose-macos-check-job-runs.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -schema_version: 1 -id: "iss-2609281514435020" -slug: "the-merge-queue-fails-a-group-whose-macos-check-job-runs" -severity: "minor" -category: "process" -source: "review-followup" -found_during: "autonomous run A resumed 2026-09-25: ruling Z" -origin: researcher-authored -production_mode: hand-written -found_at: ".github/workflows/ci.yml" ---- - -The merge queue fails a group whose macOS check job runs past 30 minutes, cancelling pull requests whose code is fine: #728 was cancelled at 30.3 minutes, #730 twice at the cap, and #733 once. The check job's timeout-minutes and the main ruleset's merge-queue check_response_timeout_minutes are both 30, while the macOS leg of a source change measures 23 to 30 minutes, so ordinary runner variance cancels a passing run, and nothing warns before the cancellation arrives. Ruling Z (2026-09-28, the technical facilitator) raises both limits to 45 minutes and asks for a warning once the macOS check passes 35 minutes, so a speed lane opens before any cancellation. diff --git a/.abcd/work/issues/resolved/iss-2609281514435020-the-merge-queue-fails-a-group-whose-macos-check-job-runs.md b/.abcd/work/issues/resolved/iss-2609281514435020-the-merge-queue-fails-a-group-whose-macos-check-job-runs.md new file mode 100644 index 000000000..5d2058320 --- /dev/null +++ b/.abcd/work/issues/resolved/iss-2609281514435020-the-merge-queue-fails-a-group-whose-macos-check-job-runs.md @@ -0,0 +1,22 @@ +--- +schema_version: 1 +id: "iss-2609281514435020" +slug: "the-merge-queue-fails-a-group-whose-macos-check-job-runs" +severity: "minor" +category: "process" +source: "review-followup" +found_during: "autonomous run A resumed 2026-09-25: ruling Z" +origin: researcher-authored +production_mode: hand-written +found_at: ".github/workflows/ci.yml" +resolution: "The macOS leg of ci.yml's check job carries a 45-minute ceiling (the ubuntu leg keeps 30) and the ruleset mirror's merge-queue check_response_timeout_minutes records 45, under ruling Z of 2026-09-28; the live ruleset follows with gh api after the merge. A step at the end of the macOS leg warns in the log and the step summary once the check passes 35 minutes, naming the rerun-once rule and the speed lane, and never fails the job. TestRaceLaneBudgetIsDeclaredAndFitsItsJob holds every leg at or below the mirror's cap and TestCheckJobWarnsBeforeTheQueueCap pins the 35-minute threshold below it." +impact: internal +resolved_by: + commit: "b248cb647" +--- + +The merge queue fails a group whose macOS check job runs past 30 minutes, cancelling pull requests whose code is fine: #728 was cancelled at 30.3 minutes, #730 twice at the cap, and #733 once. The check job's timeout-minutes and the main ruleset's merge-queue check_response_timeout_minutes are both 30, while the macOS leg of a source change measures 23 to 30 minutes, so ordinary runner variance cancels a passing run, and nothing warns before the cancellation arrives. Ruling Z (2026-09-28, the technical facilitator) raises both limits to 45 minutes and asks for a warning once the macOS check passes 35 minutes, so a speed lane opens before any cancellation. + +## Grounds + +- pursued: a passing macOS check that runs between 30 and 45 minutes concludes and merges instead of being cancelled by the queue, and one that passes 35 minutes shows the warning; a merge-group run cancelled at 30 minutes after the live ruleset is raised, a check failed by the warning step, or a run past 35 minutes with no warning would show it wrong. From 2cdcca99f27630dda44b173096643c05af839aad Mon Sep 17 00:00:00 2001 From: REPPL <77722411+REPPL@users.noreply.github.com> Date: Mon, 28 Sep 2026 16:47:17 +0100 Subject: [PATCH 5/5] test: every check leg but macOS stays at 30 minutes Ruling Z raised the macOS leg alone, but the budget test only held each leg at or below the queue's cap, so a Linux leg raised anywhere up to 45 passed. Pin every other leg at 30, and say in the warning step's comment that the warning is emitted at the job's end, for a run that finished past the line (review-cap45, two minors). Refs: iss-2609281514435020 Assisted-by: Claude:claude-opus-5-5 --- .github/workflows/ci.yml | 5 ++++- internal/core/lint/racelanebudget_test.go | 6 ++++++ 2 files changed, 10 insertions(+), 1 deletion(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index a35debfb2..091a4f398 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -413,7 +413,10 @@ jobs: # cancellation. The rule for a cancelled check is to rerun it once; a # second cancellation for the same reason stops that pull request and # opens a speed lane, never another raise of the timeout. This warning - # moves that signal ahead of the first cancellation. It never fails the + # moves that signal ahead of the first cancellation: it is emitted at the + # job's end, for a run that finished past the line, so a check that + # passed between 35 and 45 minutes says so before one is cancelled at + # the cap. It never fails the # job: `always()` runs it however the steps before it ended, and # `continue-on-error` holds even a broken script to a warning. # TestCheckJobWarnsBeforeTheQueueCap pins the threshold below the leg's diff --git a/internal/core/lint/racelanebudget_test.go b/internal/core/lint/racelanebudget_test.go index 5aacd4fe8..ecf33682d 100644 --- a/internal/core/lint/racelanebudget_test.go +++ b/internal/core/lint/racelanebudget_test.go @@ -87,6 +87,12 @@ func TestRaceLaneBudgetIsDeclaredAndFitsItsJob(t *testing.T) { "false. Raise the live ruleset's cap (an admin act) before the job's", where, leg.os, leg.minutes, queue, rulesetMirror) } + // Ruling Z raised the macOS leg alone; every other leg keeps 30. + if leg.os != "macos-latest" && leg.minutes != 30*time.Minute { + t.Errorf("%s: the %s leg's timeout-minutes is %s; ruling Z (2026-09-28) raised only "+ + "the macOS leg, and every other leg of the check job stays at 30m", + where, leg.os, leg.minutes) + } } }