fix(beat): the ollama decode floor was calibrated on sm_89 and is now asserted on sm_121 (#2835) - #2838
fix(beat): the ollama decode floor was calibrated on sm_89 and is now asserted on sm_121 (#2835)#2838noahgift wants to merge 3 commits into
Conversation
… asserted on sm_121 (#2835) The CUDA nightly has been red four nights. It is not a regression: the beat's silicon moved out from under its threshold, and the threshold did not move with it. #2740 retired the `ada-4090` matrix leg -- correctly, lambda-vector is the workstation and must never be a CI host -- and in the same commit the step lost the predicate that scoped it to that leg: - - name: Pillar-4 ... beat (ada-4090 only) - if: steps.decide.outputs.proceed == 'true' && matrix.name == 'ada-4090' + - name: Pillar-4 ... beat + if: steps.decide.outputs.proceed == 'true' REMOVING A MATRIX LEG RE-TARGETS EVERY STEP WHOSE PREDICATE NAMED IT. That shape is not in the existing guard-scope rule, which covers widening a guard deliberately; this was widening by subtraction. It had never run on gx10 before. On 2026-08-29 and earlier the step reports `skipped` on both legs and no beat line appears in any green log -- the green nights are green because the assertion did not execute. So the four reds are the first four measurements this gate has ever taken on this silicon, not a change in apr. Every number in the file's header is an sm_89 number: the four measurements, the 1.015 worst median, the ~300 tok/s incumbent. On GB10 ollama medians 181 and apr medians 112-118, ratio 0.579-0.647 across four nights with ollama stable within 0.8%. ## What changed SILICON_FLOORS is keyed by compute capability and each entry must carry its derivation. sm_89 keeps 0.90 with the four measurements behind it. sm_121 has NO entry, and that is the fix rather than an omission: four nights is data, not a calibration -- a threshold comes from samples, never from invention -- so the harness now has THREE outcomes and an uncalibrated silicon yields UNCALIBRATED-SILICON. That is not a pass (the `ada-4090 only` skip is what hid this gate for months) and not a claim that apr regressed. The message names the three ways to resolve it and says plainly that recalibrating and fixing a real deficit are opposite conclusions this gate must not pick by accident. The failure text was wrong the same way the threshold was. At ratio 0.619 it led with "at this depth apr is very likely not decoding on the GPU at all", whose own cited signature is ~20 tok/s / ratio ~0.065 -- a diagnosis its own arithmetic excludes. The rule against that was already three paragraphs up in this file ("a failure message that confidently names the wrong cause is worse than one that names none") and had not been applied to the ladder itself. The diagnosis is now conditional on the measured ratio. The silicon is read from `nvidia-smi --query-gpu=compute_cap`, never from the runner label: a label is a claim about provisioning, and this gate has already been wrong once about which host it was running on. ## The unit tests now actually run The header admitted these tests never execute -- an integration TARGET that `--lib` never reaches, named by one workflow that runs `-- --ignored`. That was still true today: the target appears nowhere in ci.yml. It is now in the beat chain, so the five new tests are gated rather than compiled and skipped. 5/5 mutation-verified, each by its own test: an invented GB10 entry, the ceiling moved above the observed band, the sm_89 floor lowered under the GB10 nights, and the observations emptied. The fifth -- the ceiling dropping below the collapse it exists to catch -- is a `const` block and fails the BUILD, because that invariant should not be shippable while the suite happens not to run. Whether GB10 is honestly ~0.62x or carries a real sm_121 deficit stays OPEN (#2835; #2800 argues the #2786 GB10 shortfall is real). This commit does not decide it, and deliberately does not lower a threshold to make a red gate green. Refs #2835, #2740, #2800. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…urity on every open PR (#2839) Same signature as #2805: all PRs red, nothing changed in-repo. The advisory was committed to the RustSec DB at 2026-09-01T07:50:30Z -- two seconds after crates.io yanked manzana at 07:50:28Z. `ci / security` passed on #2825 at 06:44Z and has failed on #2836/#2837/ #2838 from 09:56Z. An advisory-DB catch-up, not a regression. CONTAINMENT ONLY. The block records the removal condition, and it is NOT "when a new manzana ships" -- see below. Reachability measured on a clean origin/main worktree, x86_64 Linux, absolute cargo binary (a `cargo` shell function on the dev box shadows it and redirects CARGO_TARGET_DIR): cargo tree -p aprender-gpu | grep -c manzana -> 0 cargo tree -p aprender-gpu --features metal | grep -c manzana -> 0 cargo tree --workspace | grep -c manzana -> 0 git grep -l 'secure_enclave|SecureEnclave' -- '*.rs' -> 0 files The middle line is load-bearing: manzana is BOTH optional AND declared under [target.'cfg(target_os = "macos")'.dependencies], so enabling `metal` on Linux pulls in nothing. aprender's only three call sites are `manzana::metal::*`, all behind cfg(all(target_os = "macos", feature = "metal")). The advisory is about `secure_enclave::SecureEnclaveSigner`, which this workspace never names. As with the h2 entry above it, cargo-deny passes WITHOUT an exemption because it walks the ACTIVATED graph; cargo-audit scans Cargo.lock, which lists target- and feature-gated deps unconditionally. So the ignore goes in .cargo/audit.toml ONLY -- adding it to deny.toml would be a dead exemption and fail check_deny_exemptions_live.sh. WHY AN IGNORE AND NOT AN UPGRADE. Both published versions (0.1.0, 0.2.0) are YANKED, so crates.io reports max_version 0.0.0 and there is nothing to bump to. More importantly the advisory carries `[versions] patched = []` -- NO version is considered fixed, so publishing a corrected manzana does NOT clear this gate on its own. Only amending the upstream advisory to name a patched range does. The removal condition is written that way in the file. VERIFICATION, STATED HONESTLY. This is NOT locally verified green. cargo-audit 0.22.1 on this x86_64 host does not report RUSTSEC-2026-0273 at all -- manzana surfaces only as `Warning: yanked` -- so a local run passes with OR without this ignore and proves nothing. The negative control was run and it FAILED TO FAIL, which is how the vacuity was found rather than shipped. The fix is sound by construction (CI errors on that exact id; `--ignore <id>` suppresses that id) and CI is the verifier. Refs #2805. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
|
§13.11 rung 1 — quorum shadow verdict Shadow mode: this records a verdict and merges nothing. A refusal |
Pull request was converted to draft
|
PP-066 triage (epic #2873, ticket #2986): CARRY-0.67:W-G. Linked issues #2835 and #2786 both sit in the carried 0.67 row W-G (issue #2942, |
|
R-7 of YOGA-NIGHTLY-001 belongs in this PR, because
The defect this PR fixes has a second instance it does not close. The header here already says it out loud:
and the fix — Measured on yoga, 2026-09-09:
The change: add a device discriminator to the lookup — Nothing is asserted on yoga in the meantime, and that is enforced rather than assumed (#3060):
Prediction, on the record, so the measurement can falsify it: yoga's decode ratio will land below 0.90. Whichever way it goes it is data — refusing to take the measurement is how a 4090's floor gets inherited by silence. Refs paiml/infra PMAT-272 (YOGA-NIGHTLY-001 §7, R-7, N-4), #3060, #2835 🤖 Generated with Claude Code |
1 participant
The CUDA nightly has been red four nights. It is not a regression — the beat's silicon moved out from under its threshold, and the threshold did not move with it.
What happened
#2740 retired the
ada-4090matrix leg — correctly; lambda-vector is the workstation and must never be a CI host. In the same commit the step lost the predicate that scoped it to that leg:Removing a matrix leg re-targets every step whose predicate named it. That shape isn't in the existing guard-scope rule, which covers widening a guard deliberately. This was widening by subtraction.
It had never run there before
… (ada-4090 only)on both legsNo beat line appears in any green log. The green nights are green because the assertion did not execute. So the four reds are the first four measurements this gate has ever taken on this silicon.
Every number in the file's header is an sm_89 number — the four measurements, the 1.015 worst median, the ~300 tok/s incumbent. On GB10, ollama medians 181 and apr medians 112–118:
ollama is stable within 0.8% across all four, so this is a reproducible measurement, not a noisy rig.
The fix
SILICON_FLOORSis keyed by compute capability, and each entry must carry its derivation — that's what stops the next silicon inheriting a number nobody measured on it. sm_89 keeps 0.90 with its four measurements behind it.sm_121 has no entry, and that is the fix rather than an omission. Four nights is data, not a calibration — a threshold comes from samples, never from invention. So the harness now has three outcomes, and an uncalibrated silicon yields
UNCALIBRATED-SILICON: not a pass (theada-4090 onlyskip is what hid this gate for months), and not a claim that apr regressed. The message names the three ways to resolve it and says plainly that recalibrating and fixing a real deficit are opposite conclusions this gate must not pick by accident.This PR deliberately does not lower a threshold to make a red gate green.
The silicon is read from
nvidia-smi --query-gpu=compute_cap, never from the runner label — a label is a claim about provisioning, and this gate has already been wrong once about which host it was running on.The failure text was wrong the same way
At ratio 0.619 it led with "at this depth apr is very likely not decoding on the GPU at all" — whose own cited signature is ~20 tok/s / ratio ~0.065. A diagnosis its own arithmetic excludes. The rule against that was already three paragraphs up in this file ("a failure message that confidently names the wrong cause is worse than one that names none") and had not been applied to the ladder itself. The diagnosis is now conditional on the measured ratio.
The unit tests now actually run
The header admitted these tests never execute — an integration TARGET that
--libnever reaches, named by one workflow that runs-- --ignored. That was still true today: the target appeared nowhere inci.yml. It's now in the beat chain, so the five new tests are gated rather than compiled and skipped.Mutation-verified 5/5, each by its own test
The third is a
constblock — that invariant should not be shippable while the suite happens not to run.Still open
Whether GB10 is honestly ~0.62x or carries a real sm_121 decode deficit is not decided here (#2835; #2800 argues the #2786 GB10 shortfall is real). This PR makes the gate stop answering that question by accident.
Refs #2835, #2740, #2800.