From b63bcf86c987c80952bbba100bf3679e16a73051 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 6 Sep 2026 12:12:26 +0000 Subject: [PATCH 1/2] =?UTF-8?q?bgz17:=20aarch64=20cannot=20compile=20?= =?UTF-8?q?=E2=80=94=20unstable=20prefetch=20intrinsic,=20never=20built=20?= =?UTF-8?q?by=20any=20CI?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `prefetch_matrix_row` called `std::arch::aarch64::{_prefetch, _PREFETCH_READ, _PREFETCH_LOCALITY3}`, all gated behind the unstable `stdarch_aarch64_prefetch` feature (rust-lang/rust#117217). On the pinned stable 1.98.1 toolchain that is three hard `error[E0658]`s, not a missed optimization: aarch64 has no stable prefetch intrinsic at all, so the only correct form is the no-op path every other non-x86_64 target already took. Measured red-then-green, locally, both targets: target old code new code aarch64-unknown-linux-gnu 3x E0658, fails clean x86_64-unknown-linux-gnu clean clean The x86 row is the finding — the old code passes on the only architecture anything has ever built it on. Two independent holes hid it: bgz17 is workspace-`exclude`d, and all 14 CI jobs here are ubuntu (measured: 0 non-ubuntu runners), so no job has ever parsed this block as code. Surfaced only by q2#146 un-gating a suite that runs on macos-latest (Apple Silicon) — a downstream consumer's CI doing this repo's job, and the second aarch64 defect found that way in a week after #1200's NEON fixes. Second, independent finding in the same sweep: the example carried a `clippy::chunks_exact_to_as_chunks` warning — the lint rust-toolchain.toml's bump log records as swept in #1194 at "ten sites across four crates". bgz17 was not one of the four because `--workspace` cannot see an excluded crate. Fixed; the crate now passes `clippy -D warnings` (a CLAUDE.md Hard Rule) on both targets for the first time. Gates: fmt clean; clippy -D warnings clean on x86_64 AND aarch64; 134 tests pass. Board: EPIPHANIES E-AN-EXCLUDED-CRATE-ON-AN-X86-ONLY-FLEET-IS-CODE-NO-CI-HAS-EVER-COMPILED-1; ISSUES ISS-NO-AARCH64-RUNNER + ISS-EXCLUDED-CRATES-UNBUILT record what these point repairs do NOT fix — the holes are still open and closing them is a CI-policy call. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01AGVLyRZNEKKBSfBDJfbY3V --- .claude/board/EPIPHANIES.md | 64 +++++++++++++++++++ .claude/board/ISSUES.md | 54 ++++++++++++++++ .../examples/probe_base17_fold_ceiling.rs | 6 +- crates/bgz17/src/prefetch.rs | 19 +++--- 4 files changed, 131 insertions(+), 12 deletions(-) diff --git a/.claude/board/EPIPHANIES.md b/.claude/board/EPIPHANIES.md index abb311647..35a797ee8 100644 --- a/.claude/board/EPIPHANIES.md +++ b/.claude/board/EPIPHANIES.md @@ -1,3 +1,67 @@ +## 2026-09-06 — E-AN-EXCLUDED-CRATE-ON-AN-X86-ONLY-FLEET-IS-CODE-NO-CI-HAS-EVER-COMPILED-1 — un-gating one downstream suite found a second aarch64 defect that could never have built + +**Status:** FINDING, measured red-then-green locally. Fixed in this PR. +**Confidence:** High — every claim is a command output, not an inference. + +**What happened.** Un-gating q2's test suite (q2 #146, removing the +`github.repository == 'quarto-dev/q2'` guard) put `macos-latest` — Apple +Silicon, i.e. **aarch64** — in front of this workspace's code for the first +time. `crates/bgz17/src/prefetch.rs` failed with three `error[E0658]`: +`std::arch::aarch64::{_prefetch, _PREFETCH_READ, _PREFETCH_LOCALITY3}` are +gated behind the unstable `stdarch_aarch64_prefetch` feature +(rust-lang/rust#117217). On the pinned stable 1.98.1 toolchain that is a hard +compile error, not a missed optimization — **aarch64 has no stable prefetch +intrinsic at all**, so the only correct form is the no-op path. + +**Why nothing caught it, and the reason is two independent holes:** + +1. **`bgz17` is workspace-`exclude`d** (`Cargo.toml:31`) — lance-graph's own CI + never builds it as a member. +2. **Every lance-graph runner is `ubuntu-*`** (measured: `grep runs-on + .github/workflows/*.yml` → 14 jobs, all ubuntu). There is no aarch64 runner + anywhere in this repo, so `#[cfg(target_arch = "aarch64")]` code is never + parsed as code — on x86 it is indistinguishable from a comment. + +Either hole alone hides it. Both together mean the code had **never been +compiled by anything, ever**, since it landed (#844). + +**Measured, red-then-green, locally:** + +| target | old code | new code | +|---|---|---| +| `aarch64-unknown-linux-gnu` | 3× `E0658`, build fails | clean | +| `x86_64-unknown-linux-gnu` | clean | clean | + +The x86 row is the finding: the old code passes on the only architecture +anything ever built it on. + +**This is the SECOND instance in one week.** #1200 fixed `contract/src/mul.rs` +— NEON `_n_` intrinsics passed non-const shift operands and +`is_aarch64_feature_detected!` was imported from the wrong module — found the +same way, by the same un-gate. Two defects, one cause: **an architecture with +no runner is an architecture with no compiler.** + +**Second, independent finding in the same sweep.** `bgz17`'s example carried a +`clippy::chunks_exact_to_as_chunks` warning — the exact lint the +`rust-toolchain.toml` bump log records as swept in #1194 "at ten sites across +four crates". bgz17 was not one of the four **because it is excluded**, so the +sweep could not see it. Fixed here; the crate now passes `clippy -D warnings` +(a `CLAUDE.md` Hard Rule) on both targets for the first time. + +**Consequence — what this does NOT fix.** Both fixes are point repairs. The +holes remain: excluded crates are still unbuilt and unlinted by this repo's CI, +and there is still no aarch64 runner. Every other `#[cfg(target_arch = +"aarch64")]` block in this workspace and in `ndarray` is in exactly the state +these two were in ten minutes before they were measured — presumed fine, +never compiled. Filed as `ISS-NO-AARCH64-RUNNER` / `ISS-EXCLUDED-CRATES-UNBUILT` +rather than fixed here, because adding a runner is a CI-policy change and an +operator call, not a drive-by. + +**The transferable rule:** a `cfg` you cannot build is a claim, not code. When +a gate is removed and a new platform appears, expect the backlog of every +never-compiled branch to arrive at once — and do not read "it compiles here" as +evidence about anywhere else. + ## 2026-09-06 — E-I-CITED-THE-RIGHTMOST-REGISTER-AND-CALLED-IT-THE-ADDRESS-1 — three corrections to one entry, each because I reasoned instead of measuring **Status:** OPERATOR CORRECTION ×3 of my own same-day entry, superseded before diff --git a/.claude/board/ISSUES.md b/.claude/board/ISSUES.md index 04c7e3c8f..3ed354e99 100644 --- a/.claude/board/ISSUES.md +++ b/.claude/board/ISSUES.md @@ -1,3 +1,57 @@ +## ISS-NO-AARCH64-RUNNER (2026-09-06) — OPEN + +**Every `#[cfg(target_arch = "aarch64")]` block in this repo is unverified, because +no CI job has ever compiled one.** Measured: `grep -n runs-on .github/workflows/*.yml` +→ 14 jobs, all `ubuntu-latest` / `ubuntu-24.04`. On x86 an aarch64 `cfg` block is not +compiled, not type-checked, and not linted — it is text the parser skips, functionally +a comment. + +**Two defects found this way in one week, both by an EXTERNAL fleet, not by ours:** + +- #1200 — `lance-graph-contract/src/mul.rs`: NEON `_n_` intrinsics called with + non-const shift operands (they require const generics), and + `is_aarch64_feature_detected!` imported from the wrong module (it lives under + `std::arch`, unlike its x86 counterpart). +- #1204-adjacent (this issue's PR) — `bgz17/src/prefetch.rs`: three `E0658`s on the + unstable `stdarch_aarch64_prefetch` feature; **unbuildable on stable, ever**. + +Both were surfaced only because q2 #146 un-gated a suite that runs on `macos-latest` +(Apple Silicon). That is a downstream consumer's CI doing this repo's job. + +**What would close it:** one `runs-on: ubuntu-24.04-arm` (GitHub's free ARM runner) job +running `cargo check --workspace` — or, cheaper and covering excluded crates too, +`cargo check --target aarch64-unknown-linux-gnu` on the existing x86 runners, which +needs only `rustup target add` and catches every error above (verified locally this +session: the aarch64 cross-check reproduces all three `E0658`s exactly). + +**Not done here** — adding a CI job is a policy change and an operator call, not a +drive-by on a compile-fix PR. The point repairs shipped; the hole did not close. +Cf. `EPIPHANIES.md` `E-AN-EXCLUDED-CRATE-ON-AN-X86-ONLY-FLEET-IS-CODE-NO-CI-HAS-EVER-COMPILED-1`. + +## ISS-EXCLUDED-CRATES-UNBUILT (2026-09-06) — OPEN + +**`Cargo.toml`'s `exclude` list is also an exclude-from-CI list, and nothing says so.** +22 crates sit in `exclude` (`bgz17`, `deepnsm`, `deepnsm-v2`, `causal-edge`, +`thinking-engine`, `p64-bridge`, `highheelbgz`, …). Workspace-level `cargo check` / +`clippy` / `test` never touch them, so a lint sweep scoped `--workspace` silently +skips them while reporting success. + +**Measured instance:** the `rust-toolchain.toml` bump log records +`clippy::chunks_exact_to_as_chunks` as swept in #1194 at "ten sites across four +crates". `bgz17` carried an eleventh site and was not one of the four — not because +it was missed by hand, but because `--workspace` cannot see it. It surfaced only when +this session ran clippy with an explicit `--manifest-path`. + +**Consequence:** any claim of the form "the workspace passes `clippy -D warnings`" +(a `CLAUDE.md` Hard Rule) is scoped to members only. The Hard Rule reads as if it +covers the repo; it covers roughly half of it. + +**What would close it:** a CI matrix step over the `exclude` list running +`cargo clippy --manifest-path crates//Cargo.toml --all-targets -- -D warnings`. +Cheap — these are the zero/low-dependency crates. Requires first measuring how many +of the 22 are currently red, which is itself the useful number and is NOT yet known: +this session measured `bgz17` only. + ## ISS-PIN-RULING-PROSE-DRIFTS-BEHIND-THE-MANIFEST (2026-09-05) — OPEN **The pin ruling lives in two places and only one of them is checked.** diff --git a/crates/bgz17/examples/probe_base17_fold_ceiling.rs b/crates/bgz17/examples/probe_base17_fold_ceiling.rs index beffe7c12..a162d329c 100644 --- a/crates/bgz17/examples/probe_base17_fold_ceiling.rs +++ b/crates/bgz17/examples/probe_base17_fold_ceiling.rs @@ -283,8 +283,10 @@ fn main() { buf.len() ); let all: Vec = buf[8..] - .chunks_exact(4) - .map(|c| f32::from_le_bytes(c.try_into().unwrap())) + .as_chunks::<4>() + .0 + .iter() + .map(|c| f32::from_le_bytes(*c)) .collect(); println!("source: {path}\n {n} rows x {dim} dims (real embedding weights)"); diff --git a/crates/bgz17/src/prefetch.rs b/crates/bgz17/src/prefetch.rs index ebdbcd17d..7bac24201 100644 --- a/crates/bgz17/src/prefetch.rs +++ b/crates/bgz17/src/prefetch.rs @@ -95,17 +95,16 @@ fn prefetch_matrix_row(matrix: &DistanceMatrix, row: u8) { // _MM_HINT_T0 = prefetch into all cache levels std::arch::x86_64::_mm_prefetch(ptr as *const i8, std::arch::x86_64::_MM_HINT_T0); } - #[cfg(target_arch = "aarch64")] - unsafe { - std::arch::aarch64::_prefetch( - ptr as *const i8, - std::arch::aarch64::_PREFETCH_READ, - std::arch::aarch64::_PREFETCH_LOCALITY3, - ); - } - // On other architectures: no-op. The matrix is small enough that + // aarch64 has NO stable prefetch intrinsic. `std::arch::aarch64::_prefetch` + // and its `_PREFETCH_*` constants are gated behind the unstable + // `stdarch_aarch64_prefetch` feature (rust-lang/rust#117217), so calling + // them is a hard `error[E0658]` on the pinned stable toolchain — not a + // missed optimization. This crate therefore takes the no-op path on + // aarch64, and MUST keep doing so until that feature stabilizes. + // + // On every non-x86_64 target: no-op. The matrix is small enough that // hardware prefetch usually handles it anyway. - #[cfg(not(any(target_arch = "x86_64", target_arch = "aarch64")))] + #[cfg(not(target_arch = "x86_64"))] let _ = ptr; } } From 5d2d26ee295f94acc8290b485004c3559af3647e Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 6 Sep 2026 12:22:11 +0000 Subject: [PATCH 2/2] board: narrow three overclaims CodeRabbit caught in the aarch64 entry MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit All three review findings on #1205 were correct. None touches code; all three were my prose claiming more than I measured — the same failure this session was already corrected for twice. 1. ISSUES / ISS-NO-AARCH64-RUNNER — the substantive one. I proposed `cargo check --target aarch64-unknown-linux-gnu` at the root and wrote that it covers excluded crates "too". It does not: `bgz17` is in `exclude`, so no root-manifest invocation selects it, and the proposed gate would have been blind to the exact defect that opened the issue. I knew this — locally I used `--manifest-path` — so the proposal contradicted my own method. Now states both forms and records that ISS-NO-AARCH64-RUNNER and ISS-EXCLUDED-CRATES-UNBUILT are not independent: an aarch64 gate is only as wide as the crate list it is pointed at. 2. EPIPHANIES — "never been compiled by anything, ever" is contradicted by the entry's OWN evidence table two lines below it, which reports a deliberate local aarch64 build of the old code. Scoped to the measured claim: no CI job in this repository has ever built the aarch64 target. Same narrowing applied at the three other sites that carried the loose form. 3. EPIPHANIES — "aarch64 has no stable prefetch intrinsic at all, so the only correct form is the no-op path". The intrinsic half is right; "only correct form" is not. Stable `asm!` has been available on aarch64 since 1.59, so a hand-written `prfm` is possible — unwarranted here for an advisory hint on a small matrix, and it would need its own measurement. Reworded to "this crate's stable fallback". Each correction names the review that caught it, so the entry carries its own provenance rather than reading as if it were right the first time. No code change: `prefetch.rs` and the example are byte-identical to b63bcf86. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01AGVLyRZNEKKBSfBDJfbY3V --- .claude/board/EPIPHANIES.md | 51 +++++++++++++++++++++++++------------ .claude/board/ISSUES.md | 21 +++++++++++---- 2 files changed, 51 insertions(+), 21 deletions(-) diff --git a/.claude/board/EPIPHANIES.md b/.claude/board/EPIPHANIES.md index 35a797ee8..1ed350e9c 100644 --- a/.claude/board/EPIPHANIES.md +++ b/.claude/board/EPIPHANIES.md @@ -10,20 +10,35 @@ time. `crates/bgz17/src/prefetch.rs` failed with three `error[E0658]`: `std::arch::aarch64::{_prefetch, _PREFETCH_READ, _PREFETCH_LOCALITY3}` are gated behind the unstable `stdarch_aarch64_prefetch` feature (rust-lang/rust#117217). On the pinned stable 1.98.1 toolchain that is a hard -compile error, not a missed optimization — **aarch64 has no stable prefetch -intrinsic at all**, so the only correct form is the no-op path. +compile error, not a missed optimization: **Rust 1.98.1 exposes no stable +prefetch *intrinsic* on aarch64.** The no-op is therefore this crate's stable +fallback — not the only conceivable form. Stable `asm!` IS available on +aarch64 (since 1.59), so a hand-written `prfm` is possible; it is not +warranted here for an advisory hint on a small matrix, and would need its own +measurement to justify. (Narrowed after a CodeRabbit review on #1205 flagged +the original "only correct form" as an overclaim; the flag was right.) **Why nothing caught it, and the reason is two independent holes:** 1. **`bgz17` is workspace-`exclude`d** (`Cargo.toml:31`) — lance-graph's own CI never builds it as a member. -2. **Every lance-graph runner is `ubuntu-*`** (measured: `grep runs-on - .github/workflows/*.yml` → 14 jobs, all ubuntu). There is no aarch64 runner - anywhere in this repo, so `#[cfg(target_arch = "aarch64")]` code is never - parsed as code — on x86 it is indistinguishable from a comment. - -Either hole alone hides it. Both together mean the code had **never been -compiled by anything, ever**, since it landed (#844). +2. **Every lance-graph runner is `ubuntu-*`** (measured: `grep -h runs-on + .github/workflows/*.yml` → 14 jobs, 0 non-ubuntu), and no job passes + `--target`. So no CI job in this repo has ever built the + `aarch64-unknown-linux-gnu` target, and on an x86 host a + `#[cfg(target_arch = "aarch64")]` block is not compiled or type-checked — + it is skipped like a comment. + +Either hole alone hides it. Both together mean **no CI job in this repository +has ever compiled this block** since it landed (#844). + +**Scoped precisely, because the looser version is false.** The claim is about +*this repo's CI*, NOT about the world: this session compiled the old code +locally for aarch64 on purpose — that is exactly how the table below was +produced — and q2's macOS runner compiled it too, which is what surfaced it. +The first draft of this entry said "never compiled by anything, ever", which +its own evidence table contradicts two lines down. Corrected after a +CodeRabbit review on #1205 caught the self-contradiction. **Measured, red-then-green, locally:** @@ -38,8 +53,9 @@ anything ever built it on. **This is the SECOND instance in one week.** #1200 fixed `contract/src/mul.rs` — NEON `_n_` intrinsics passed non-const shift operands and `is_aarch64_feature_detected!` was imported from the wrong module — found the -same way, by the same un-gate. Two defects, one cause: **an architecture with -no runner is an architecture with no compiler.** +same way, by the same un-gate. Two defects, one cause: **an architecture no +CI job builds is an architecture whose code is unverified** — the compiler +exists and cross-compiling is one flag away; nothing was pointing it there. **Second, independent finding in the same sweep.** `bgz17`'s example carried a `clippy::chunks_exact_to_as_chunks` warning — the exact lint the @@ -53,14 +69,17 @@ holes remain: excluded crates are still unbuilt and unlinted by this repo's CI, and there is still no aarch64 runner. Every other `#[cfg(target_arch = "aarch64")]` block in this workspace and in `ndarray` is in exactly the state these two were in ten minutes before they were measured — presumed fine, -never compiled. Filed as `ISS-NO-AARCH64-RUNNER` / `ISS-EXCLUDED-CRATES-UNBUILT` +never built by this repo's CI. Filed as `ISS-NO-AARCH64-RUNNER` / `ISS-EXCLUDED-CRATES-UNBUILT` rather than fixed here, because adding a runner is a CI-policy change and an operator call, not a drive-by. -**The transferable rule:** a `cfg` you cannot build is a claim, not code. When -a gate is removed and a new platform appears, expect the backlog of every -never-compiled branch to arrive at once — and do not read "it compiles here" as -evidence about anywhere else. +**The transferable rule:** a `cfg` your CI never builds is a claim, not +verified code. When a gate is removed and a new platform appears, expect the +backlog of every unbuilt branch to arrive at once — and do not read "it +compiles here" as evidence about anywhere else. The corollary this entry +learned the hard way: that rule applies to the entry's OWN prose. "Never +compiled by anything" was a stronger claim than "no CI job compiled it", and +only the weaker one was measured. ## 2026-09-06 — E-I-CITED-THE-RIGHTMOST-REGISTER-AND-CALLED-IT-THE-ADDRESS-1 — three corrections to one entry, each because I reasoned instead of measuring diff --git a/.claude/board/ISSUES.md b/.claude/board/ISSUES.md index 3ed354e99..8b3980721 100644 --- a/.claude/board/ISSUES.md +++ b/.claude/board/ISSUES.md @@ -18,11 +18,22 @@ a comment. Both were surfaced only because q2 #146 un-gated a suite that runs on `macos-latest` (Apple Silicon). That is a downstream consumer's CI doing this repo's job. -**What would close it:** one `runs-on: ubuntu-24.04-arm` (GitHub's free ARM runner) job -running `cargo check --workspace` — or, cheaper and covering excluded crates too, -`cargo check --target aarch64-unknown-linux-gnu` on the existing x86 runners, which -needs only `rustup target add` and catches every error above (verified locally this -session: the aarch64 cross-check reproduces all three `E0658`s exactly). +**What would close it:** cross-compiling on the existing x86 runners is the cheap +option — `rustup target add aarch64-unknown-linux-gnu` plus +`cargo check --workspace --target aarch64-unknown-linux-gnu`. A dedicated +`runs-on: ubuntu-24.04-arm` job is the alternative and needs no target plumbing. + +**⚠ But `--workspace` alone would NOT have caught the defect that opened this +issue.** `bgz17` is in `exclude`, so no root-manifest invocation selects it — the +cross-check has to be run per excluded crate, `cargo check --manifest-path +crates//Cargo.toml --target aarch64-unknown-linux-gnu`, which is exactly how it +was reproduced locally this session (all three `E0658`s, exactly). A first draft of +this entry proposed the root-only form and claimed it covered excluded crates +"too"; that was wrong, and wrong in the specific way that would have shipped a gate +blind to its own founding example. Caught by a CodeRabbit review on #1205. So this +issue and `ISS-EXCLUDED-CRATES-UNBUILT` are **not independent**: an aarch64 gate is +only as wide as the crate list it is pointed at, and the member list is not that +list. **Not done here** — adding a CI job is a policy change and an operator call, not a drive-by on a compile-fix PR. The point repairs shipped; the hole did not close.