From 6f489ecf4b67bbfef686e0cc2b080f4bb5c4a37e Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 5 Sep 2026 22:29:08 +0000 Subject: [PATCH 01/16] =?UTF-8?q?plans:=20gemm=20consolidation=20v1.1=20?= =?UTF-8?q?=E2=80=94=20Mississippi=20Queen=20amendment=20+=20Wave=200=20st?= =?UTF-8?q?atic=20results?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two additions to the DRAFT, no code. §9 folds in the operator's Mississippi Queen metaphor, graded mechanism vs rhyme. The load-bearing one CORRECTS the plan: v1's D-GTM-F4 built the compacted row-index list "once per mask generation" — laying the whole river before any boat moves. `pack_a_f32` already walks a panel cursor, so the mask->index expansion belongs one panel AHEAD of that cursor, on the stack, not in an O(n_rows) prologue (which also quietly violated data-flow.md §1's no-alloc-in-hot-loop rule). M1b names where the cache lives: keyed by (mask generation, panel index), never by call — a per-call cache amortizes nothing. M2 turns D-GTM-0e into a lookahead ladder; M3 replaces the T2->T1 prohibition with a coal budget. The hexagon itself is marked rhyme [S]: the game's six is adjacency, the substrate's six is field carving. §10 runs the three measurement-free W0 probes. Each corrected §1.1: - 0a: both "duplicates" divergent, only one a defect. The bf16 pair is polyfill vs dispatcher and the facade already renames one `_amx`; `simd_avx2.rs:462 sgemm_blocked` is a naive scalar triple loop whose file, name and body disagree three ways. - 0b: blas_level3.rs is not empty — a six-method BlasLevel3 trait dispatching to BlasFloat::backend_gemm. A `pub fn` grep cannot see a method. This re-frames D-GTM-F3: two facades already exist, and BlasFloat's Float bound structurally excludes i8/bf16 from the generic one, so W1's first question is which is canonical. - 0f: pruned_gemm_rows and mixed_precision_gemm have zero callers anywhere. §2.3 called the former "the ONLY existing mask->GEMM bridge"; it is dead code, so D-GTM-5 is a first writer, not a migration. Also records a consumer-side iron-rule violation found incidentally: the five external bf16_tile_gemm_16x16 references resolve to two different bodies, one of them via `ndarray::hpc::*` past the facade. Both call sites are lance-graph's; reported, not fixed. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01EfrUJH3UNnv5NpDH4jDGHq --- .claude/blackboard.md | 56 +++++ .../gemm-ternlog-mask-consolidation-v1.md | 220 +++++++++++++++++- 2 files changed, 270 insertions(+), 6 deletions(-) diff --git a/.claude/blackboard.md b/.claude/blackboard.md index c1a5c3cc..b9f8388a 100644 --- a/.claude/blackboard.md +++ b/.claude/blackboard.md @@ -3,6 +3,62 @@ > **Read this first.** The "Polyglot Notebook" architecture below is a > separate/older program, not the current epoch. +## 2026-09-05 (later) — GEMM consolidation plan v1.1: Mississippi Queen amendment + Wave 0 static results + +**Operator metaphor folded in (§9).** Mississippi Queen: the river board is laid a +few hex tiles ahead of the lead boat, speed changes ±1 and is committed before the +move, extra maneuvers cost from a fixed coal budget, and a tile laid for the leader +is free for every boat behind. Graded per the mechanism-vs-rhyme rule: + +- **M1 [G] CORRECTS D-GTM-F4/D-GTM-5.** v1 said the compacted row-index list is + "built once per mask generation" — laying the whole river before any boat moves. + Wrong shape: `pack_a_f32` (`kernels_avx512.rs:552`) already walks a panel cursor, + so mask→index expansion belongs ONE PANEL AHEAD of that cursor, in that loop. + Signature changes `mask_to_row_indices(&[u64]) -> Vec` → + `next_panel_indices(&[u64], cursor, mr) -> ArrayVec` (stack, no + hot-loop alloc — which also fixes a quiet `data-flow.md` §1 violation in v1). +- **M1b [G] — the amortization itself.** Many boats, one river: the cache key is + `(mask generation, panel index)`, NOT the call. Per-call caching amortizes nothing. +- **M2 [H]** lookahead depth adapts ±1 and commits before the panel → D-GTM-0e + becomes a LADDER (lookahead 1/2/4/8 × density 10/50/90%), not one crossover. +- **M3 [H]** coal = a bounded budget for mid-stream re-chains; replaces the + T2→T1 prohibition (`membrane-tiers.md:105`) with a budget. +- **R1 [S] the hexagon is rhyme** pending one operator word: the game's six is + ADJACENCY, the substrate's six (`6×(u8:u8)` facet rails, 6-byte HHTL path = + CAM-PQ 6×256) is FIELD CARVING. Same cardinality, different mechanism. Unbuilt. + +**Wave 0 static probes run (§10) — each corrected the inventory:** + +- **0a:** both duplicate names DIVERGENT, only one a defect. `bf16_tile_gemm_16x16` + = polyfill (`simd_ops.rs`, F32x16 decode) vs dispatcher (`hpc/`, AMX/VNNI) — + legitimate, and `simd.rs:714` already renames the dispatcher `_amx`. + `simd_avx2.rs:462 sgemm_blocked` is a **naive scalar triple loop** — neither AVX2 + nor blocked; file, name and body disagree three ways. +- **0b:** `blas_level3.rs` is NOT empty — 393 lines, zero `pub fn` because it is a + **trait** (`BlasLevel3`: gemm/gemm_into/syrk/symm/trmm/trsm, blanket impl, + re-exported `simd.rs:656`) dispatching to `BlasFloat::backend_gemm` + (`backend/mod.rs:75`, impl'd **f32/f64 only**). CLAUDE.md was right; my `pub fn` + grep was blind. **Re-frames D-GTM-F3: TWO facades already exist** — the generic + trait method and the four free functions — and `BlasFloat`'s `num_traits::Float` + bound structurally excludes i8/bf16 from the generic one. W1's first question is + which is canonical, not how to build one. +- **0f caller census:** `pruned_gemm_rows` **0 callers**, `mixed_precision_gemm` + **0**, `blas_gemm` 0 external. §2.3 called `pruned_gemm_rows` "the ONLY existing + mask→GEMM bridge" — it is dead code, so D-GTM-5 is a FIRST WRITER, not a + migration. Only `bf16_tile_gemm_16x16` has real external consumers (5). + +**Incidental find, reported not fixed (lance-graph call sites):** the 5 external +references reach `bf16_tile_gemm_16x16` by two paths that resolve to two different +bodies. `symbiont/src/domino.rs:27` imports it from `ndarray::simd` alongside +`amx_available` and its doc mentions tile ops — but that name is the POLYFILL; it +wants `bf16_tile_gemm_16x16_amx`. `thinking-engine/examples/amx_bf16_probe.rs:15` +imports from `ndarray::hpc::bf16_tile_gemm::*`, reaching past the facade — the exact +form the "all SIMD from `ndarray::simd`, never `hpc::*`" iron rule forbids. ndarray's +own facade is correct; both defects are consumer-side (and symbiont is deprecated). + +**Not run:** 0c (f64/tail bench), 0d (MKL-ternlog tail hunch), 0e (the M2 ladder — +now known to measure a zero-caller kernel, fine for a probe, not production evidence). + ## 2026-09-05 — AMX f32 GEMM was silently bf16; `matmul_f32` made exact; consolidation plan filed **Finding (measured, PR #303):** `hpc::amx_matmul::matmul_f32` downcast both operands diff --git a/.claude/plans/gemm-ternlog-mask-consolidation-v1.md b/.claude/plans/gemm-ternlog-mask-consolidation-v1.md index abd7cb86..dbb047ee 100644 --- a/.claude/plans/gemm-ternlog-mask-consolidation-v1.md +++ b/.claude/plans/gemm-ternlog-mask-consolidation-v1.md @@ -1,5 +1,10 @@ # gemm-ternlog-mask-consolidation-v1 — one GEMM entry per dtype, masks as the prefilter, ternlog as the mask ALU +> **Status:** DRAFT v1.1 (2026-09-05) — §9 folds in the operator's Mississippi Queen +> metaphor: M1 CORRECTS D-GTM-F4/D-GTM-5 (panel-ahead expansion, not a whole-board +> prologue), M1b names where the cache lives, M2 re-shapes D-GTM-0e into a ladder, +> M3 turns the T2→T1 prohibition into a budget. R1 (the hexagon) is marked rhyme. +> > **Status:** DRAFT v1 (2026-09-05). Source-first: every "exists" row cites `file:line` > at ndarray `claude/great-curie-d2ufyl` HEAD (PR #303 + this doc). Every "proposed" > row carries a falsifier. Nothing in §5 is built. No kernel, no ABI symbol. @@ -64,7 +69,7 @@ Three claims, each falsifiable in §6: | Runtime re-exports | `matmul_f32`, `matmul_bf16_to_f32`, `matmul_i8_to_i32` (×2), `gemm_u8_i8` | `simd_runtime/matmul.rs:43,33,80,193,227` | feature `runtime-dispatch` mirror of the AMX API | | Misc | `matmul_vec`, `matmul_i8_to_i32_wasm` | `hpc/models/layers.rs:174`, `simd_wasm.rs:1474` | model layer GEMV; wasm arm | -**Counted, not estimated:** 54 entry points, 12 files, **4 unified.** `hpc/blas_level3.rs` (named in CLAUDE.md as "BLAS L3 gemm/syrk/trsm/symm") returned **zero** `pub fn` hits in this grep — W0 must resolve whether that module is empty, macro-generated, or misnamed. +**Counted, not estimated:** 54 entry points, 12 files, **4 unified.** ⊘ **§10 D-GTM-0b corrects this count's blind spot:** `hpc/blas_level3.rs` returned zero `pub fn` hits because its surface is a **trait** (`BlasLevel3`, six methods, blanket impl, re-exported at `simd.rs:656`) dispatching through `BlasFloat::backend_gemm` — a dtype-generic facade that already exists beside the four free functions. A `pub fn` grep cannot see a method; any re-inventory must grep `fn`. ### §1.2 — ternlog: already a T1 primitive, already chained once @@ -136,7 +141,7 @@ A 3-input `IMM` is the whole truth table (`simd.rs:561-563`), so any Boolean ove | D-GTM-F1 | `gemm_f32` default = hand-rolled F32x16 `sgemm_blocked` on `avx512f` hosts for square-ish shapes; `matrixmultiply` for skinny/wide rectangles (threshold from D-GTM-0c) and on other hosts. Exact on both. | §1.3: wins 256³–4096³ (up to 7%); LOSES 10–19% at 256×8192×256 and 64×2048×8192 | | D-GTM-F2 | AMX serves `gemm_bf16` and `gemm_i8` ONLY. Any f32 AMX path is a named opt-in carrying its measured table. | §1.3; ndarray#303 | | D-GTM-F3 | The facade is `backend::{gemm_f32, gemm_f64, gemm_bf16, gemm_i8}` (+ batched). Every other GEMM symbol is `pub(crate)`, a documented opt-in, or deleted. | §1.1 count 54→4 | -| D-GTM-F4 | Mask→GEMM prefilter consumes a compacted row-index list built once per mask generation, stored as a mask carving. Never bit-tests inside the micro-kernel. | §2.3; mask-risc §14.7 | +| D-GTM-F4 | ⊘ **AMENDED by §9 M1/M1b.** Mask→GEMM prefilter expands mask→indices ONE PANEL AHEAD of the pack cursor (stack, no hot-loop alloc), cached at `(mask generation, panel index)` — never a whole-board `Vec` prologue, never a bit-test inside the micro-kernel. | §2.3; mask-risc §14.7; §9 M1 | | D-GTM-F5 | Every accuracy test in this surface uses inputs whose significands exceed 8 bits, and tolerances at f32 grade (1e-5) for f32 APIs. | the vacuous `(i+j)*0.5` test that hid the bf16 loss (#303) | | D-GTM-F6 | Every new kernel lands with a two-sided pin: the fast path must beat the reference by a stated factor AND the reference must still be measurably slower — so a regression in either direction fails. | `three_pass_split_beats_one_bf16_pass` pattern | @@ -146,12 +151,12 @@ A 3-input `IMM` is the whole truth table (`simd.rs:561-563`), so any Boolean ove | id | probe | falsifier / gate | |---|---|---| -| D-GTM-0a | Diff `simd_ops.rs:587 bf16_tile_gemm_16x16` vs `hpc/bf16_tile_gemm.rs:45`; diff `simd_avx2.rs:462 sgemm_blocked` vs `kernels_avx512.rs:665`. | byte-identical bodies → delete one; divergent → record which is canonical and why | -| D-GTM-0b | Resolve `hpc/blas_level3.rs`: what does it export, if anything? | zero `pub fn` → either delete the CLAUDE.md claim or find the macro | +| D-GTM-0a | ✅ **RUN — §10.** Both pairs divergent. `bf16_tile_gemm_16x16` = polyfill vs dispatcher, legitimate, facade already renames one `_amx`. `sgemm_blocked` in `simd_avx2.rs` = a naive scalar triple loop (neither AVX2 nor blocked) — a naming defect. | v1 expected "byte-identical → delete"; neither pair is | +| D-GTM-0b | ✅ **RUN — §10.** A `BlasLevel3` trait, 6 methods, blanket impl, → `BlasFloat::backend_gemm` (f32/f64 only). CLAUDE.md was right; the grep was blind. **Re-frames D-GTM-F3: two facades already exist.** | found the surface; the macro hypothesis was wrong | | D-GTM-0c | Extend `gemm_paths_bench` to f64 (`dgemm_blocked` vs `gemm_f64_tiled_fma` vs matrixmultiply) and to non-square / K-tail shapes (e.g. 1000×1000×1000, 17×33×15 scaled). | if `sgemm_blocked` loses on any tail shape, D-GTM-F1 gains a shape guard, not a revert | | D-GTM-0d | **The MKL-ternlog hunch:** in `backend/mkl.rs` `sgemm` (`:384`), time the tail-lane handling with the current idiom vs one `mask_ternlog` select, K∈{255,257,1023,1025}. | < 3% end-to-end → record as shape-only win, no speed claim; ≥ 3% → W1 item | -| D-GTM-0e | `pruned_gemm_rows` (`prefilter.rs:189`): measure per-row bit-test vs compacted-index pack at 10%/50%/90% mask density. | the crossover density decides D-GTM-F4's threshold, or proves compaction always wins | -| D-GTM-0f | Count real call sites of every §1.1 symbol across ndarray, lance-graph, lance-graph-java, burn (`grep -rn`). | symbols with zero external callers are `pub(crate)` candidates for W1 with no consumer wave | +| D-GTM-0e | ⊘ **RE-SHAPED by §9 M2 into a LADDER:** `pruned_gemm_rows` (`prefilter.rs:189`) measured over lookahead ∈ {1,2,4,8} panels × density ∈ {10%,50%,90%}, per-row bit-test as the floor. | a single crossover cannot express a ±1-adaptive depth; the ladder decides the step size, or proves depth inert | +| D-GTM-0f | ✅ **RUN — §10.** `pruned_gemm_rows` and `mixed_precision_gemm` have **0 callers anywhere**; `blas_gemm` 0 external; only `bf16_tile_gemm_16x16` has real external consumers (5), reached by two paths that resolve to two different bodies (one an iron-rule violation, reported). | D-GTM-5 is a FIRST WRITER, not a migration | ### Wave 1 — ndarray (the facade and the backends) @@ -190,3 +195,206 @@ A 3-input `IMM` is the whole truth table (`simd.rs:561-563`), so any Boolean ove 2. Does the reverse-engineered MKL path have any caller today, or is it a second facade with zero consumers? (D-GTM-0f decides whether W1 demotes it or deletes it) 3. At what mask density does compacted-index packing beat per-row bit tests? (D-GTM-0e) 4. Is there a real shape where an f32 AMX path wins? Nothing measured says yes; the opt-ins exist so the question stays cheap to re-ask. + +## §9 — The Mississippi Queen shape (operator metaphor, 2026-09-05) — v1.1 AMENDMENT + +**The game.** A paddle-steamer race whose river board does not exist in advance: +hex tiles are laid a few ahead of the lead boat, never the whole course. Each turn +you set speed by **±1 only** and commit it *before* moving that many hexes. One +direction change is free; extras are paid from a fixed **coal** budget. Several +boats race the same river, and a tile laid for the leader is free for everyone +behind. + +Graded per the workspace rule (`cross-domain-synthesizer`: shared MECHANISM is +transferable, mere rhyme is decorative and must be labelled). Three mechanisms, +one rhyme. + +### M1 — Reveal ahead of the cursor; never lay the whole river [G] — **CORRECTS D-GTM-F4 / D-GTM-5** + +D-GTM-F4 as written says the compacted row-index list is *"built once per mask +generation"*. That is laying the entire river before any boat moves: `O(n_rows)` +memory, and it pays for rows the GEMM may never reach — a pruned GEMM can stop +early, and a blocked GEMM only ever needs the panel under its cursor. + +The premise that makes the correction concrete is already in the code: +`pack_a_f32` (`backend/kernels_avx512.rs:552`) walks +`while ii + SGEMM_MR <= mc`, addressing `a[(i_start + ii + ir) * lda + …]`. +**Packing already has a cursor.** The mask→index expansion belongs at that same +cursor, in that same loop, one panel ahead — not in a prologue. + +Consequence, replacing D-GTM-5's signature: + +```rust +// WAS (v1): whole-board prologue, heap, pays for unreached rows +fn mask_to_row_indices(mask: &[u64]) -> Vec + +// IS (v1.1): one panel ahead of the pack cursor, stack, no hot-loop alloc +fn next_panel_indices(mask: &[u64], cursor: usize, mr: usize) -> ArrayVec +``` + +This also satisfies `.claude/rules/data-flow.md` §1 ("never allocate inside a hot +loop — slice into pre-allocated storage"), which the `Vec` version quietly +violated. + +### M1b — A tile serves every boat behind it [G] — this is *the* amortization + +Several boats race one river. The leader pays to reveal a tile; everyone behind +crosses it free. That is the amortization the operator named, and the game says +**where the cache lives**: on the *tile*, not on the *boat*. + +So the cache key is `(mask generation, panel index)` — **not** the call. Two GEMMs +against the same mask generation reuse the same expanded panels; a new mask +generation invalidates them wholesale (the registry already does exactly this to +`cached_carving`, `lgj-abi/registry.rs:842-849`). A per-call cache would re-lay the +river for every boat and amortize nothing. + +### M2 — Speed changes by ±1 and is committed before the move [H] + +Lookahead depth (how many panels ahead the expansion runs) is a state variable +that moves **one step at a time** and is **committed before the panel is entered**. +Two properties, both load-bearing: + +- *Hysteresis* — it cannot be re-derived per row, which is what stops a + per-call heuristic from thrashing between depths on adjacent panels. +- *Commit-ahead* — you cannot discover mid-panel that you needed a deeper + lookahead; by then the pack loop is already running. + +**Changes D-GTM-0e:** it measured a single crossover density. It now measures a +**ladder** — lookahead ∈ {1, 2, 4, 8} panels × density ∈ {10%, 50%, 90%} — because +a single crossover cannot express a ±1-adaptive depth. Graded [H]: the ladder shape +is argued, the step size is not yet measured. + +### M3 — Coal: a bounded budget for extra maneuvers [H] + +Re-chaining the ternlog predicate mid-stream (the mask gains a conjunct, or changes +shape) is a maneuver paid from a fixed budget. When the budget is spent you commit +to the mask you hold rather than re-deriving it. + +This is what keeps *chaining* from degenerating into *re-evaluate the predicate per +panel* — the exact T2→T1 violation `membrane-tiers.md:105` already forbids as a +prohibition. The game supplies the better form: **a budget rather than a ban**, so +the legitimate mid-stream re-chain stays possible and the pathological one runs out +of coal. + +### R1 — The hexagon itself is rhyme, pending one operator word [S] + +Six is conspicuous on both sides: the game moves on 6-neighbour hexes; this +substrate carves the 12-byte V3 facet as `6×(u8:u8)` and the HHTL path as 6 bytes += CAM-PQ `6×256`. **They are not obviously the same six.** The game's six is +*adjacency* (which cell may I move to next); the substrate's six is *field carving* +(which byte pair means what). Adjacency and carving are different mechanisms that +happen to share a cardinality — the textbook rhyme signature. + +Left [S] and unbuilt. If the operator meant the **rails** specifically, this is +promoted and gets its own section; nothing in M1-M3 depends on it either way. + +### What this amendment does NOT touch + +The AMX verdict (§1.3, measured), the facade consolidation (D-GTM-F1/F3), and the +W0 static probes (0a, 0b, 0f) are unaffected — the metaphor is about *when work is +done and who pays for it*, not about which kernel is fastest or how many entry +points exist. + +## §10 — WAVE 0 RESULTS (run 2026-09-05, static probes 0a / 0b / 0f) + +Three of the six W0 probes are measurement-free (greps and body diffs) and are run +here. Each corrected something in §1.1's inventory — which is the point of running +them before building anything. + +### D-GTM-0a — the two duplicate names: BOTH divergent, only ONE is a defect + +| pair | `simd_*` body | `hpc/` or `backend/` body | verdict | +|---|---|---|---| +| `bf16_tile_gemm_16x16` | `simd_ops.rs`, 37 lines — decode BF16→f32, then F32x16 + FMA. The **polyfill**. | `hpc/bf16_tile_gemm.rs`, 17 lines — `amx_available() \|\| avx512bf16` → VNNI-pack → tile tiers. The **dispatcher**. | **NOT a defect.** Backend vs dispatcher, legitimately distinct. The facade already disambiguates: `simd.rs:714` re-exports the hpc one **renamed** `bf16_tile_gemm_16x16_amx`; `simd.rs:744` exports the polyfill under the plain name. | +| `sgemm_blocked` | `simd_avx2.rs:462`, 14 lines — a **naive scalar triple loop** (`for i / for j / for p { sum += … }`). | `backend/kernels_avx512.rs:665`, 52 lines — the real packed-panel MR=6/NR=16 kernel. | **DEFECT, naming.** The `simd_avx2.rs` body is neither AVX2 nor blocked; the file name, the function name, and the body disagree three ways. | + +⊘ **Corrects v1's D-GTM-0a**, which anticipated "byte-identical bodies → delete +one". Neither pair is byte-identical and neither should be deleted. The real +finding is narrower and different: one legitimate tier pair (already handled by a +facade rename) and one mislabelled scalar fallback. + +### D-GTM-0b — `blas_level3.rs` is not empty; the inventory grep was blind + +393 lines, **zero `pub fn`** — because the surface is method-shaped: + +```rust +pub trait BlasLevel3 { + fn blas_gemm(&self, alpha: A, b: &Self, beta: A) -> Array; + fn blas_gemm_into(&self, alpha: A, b: &Self, beta: A, c: &mut Array); + fn blas_syrk (&self, uplo: Uplo, alpha: A, beta: A, c_init: Option<&Self>) -> Array; + fn blas_symm (&self, side: Side, uplo: Uplo, alpha: A, b: &Self, beta: A, c_init: Option<&Self>) -> …; + fn blas_trmm (&self, side: Side, uplo: Uplo, alpha: A, a_tri: &Self) -> Array; + fn blas_trsm (&self, side: Side, uplo: Uplo, alpha: A, b: &Self) -> Array; +} +impl BlasLevel3 for ArrayBase where A: BlasFloat + Float + AddAssign, S: Data +``` + +Re-exported at `simd.rs:656`. CLAUDE.md's "BLAS L3 (gemm, syrk, trsm, symm)" claim +was **accurate all along**; §1.1's `grep 'pub fn …gemm'` simply could not see a +trait. **Any future inventory of this crate must grep `fn`, not `pub fn`.** + +**And it dispatches through a facade that already exists.** `blas_gemm`'s body is +`A::backend_gemm(m, n, k, alpha, …)` — a method on `BlasFloat` (`backend/mod.rs:75`), +implemented for **f32 and f64 only** (`:83`, `:110`), with `f32::backend_gemm` → +`gemm_f32`. + +⊘ **This materially re-frames D-GTM-F3.** v1 said "the facade is +`backend::{gemm_f32, gemm_f64, gemm_bf16, gemm_i8}`; consolidate everything else +under it." In fact **two facades already exist side by side**: + +| | shape | dtypes | callers (0f) | +|---|---|---|---| +| `BlasFloat::backend_gemm` | dtype-**generic** trait method | f32, f64 | reached only via `BlasLevel3` | +| `backend::{cblas_sgemm, cblas_dgemm, gemm_i8, gemm_bf16}` | four **free functions** | f32, f64, i8, bf16 | 10 / 2 / 6 in-crate | + +The consolidation question is therefore **not** "build one facade" but "**which of +the two existing facades is canonical**". The constraint that decides it is already +in the source: `BlasFloat`'s impl bounds require `num_traits::Float`, so **i8 and +bf16 cannot join it** without a second trait or a bound change. A generic facade +that structurally excludes half the dtypes is not the canonical one. Recorded as +the first thing W1 must settle; no verdict claimed here. + +### D-GTM-0f — caller census: two of the plan's own load-bearing symbols are DEAD + +| symbol | in-crate | external | note | +|---|---|---|---| +| `bf16_tile_gemm_16x16` | 21 | **5** | the only symbol with real external consumers | +| `int8_gemm_vnni` | 12 | 0 | | +| `batched_gemm_f32` | 11 | 0 | | +| `cblas_sgemm` | 10 | 0 | | +| `gemm_f64_tiled_fma` | 9 | 0 | | +| `gemm_bf16` | 6 | 0 | | +| `sgemm_blocked` | 3 | 0 | `pub(crate)`-reachable only | +| `gemm_i8` | 2 | 0 | | +| `blas_gemm` | 2 | 0 | decl + impl; the trait facade is **unused** | +| **`pruned_gemm_rows`** | **0** | **0** | ⚠ | +| **`mixed_precision_gemm`** | **0** | **0** | ⚠ | + +**`pruned_gemm_rows` has zero callers.** §2.3 called it "the seed" and "the ONLY +existing mask→GEMM bridge", and §9 M1 rewrote its signature — all of that was +reasoning about **dead code**. It is still the right *shape* to build on, but the +plan must stop describing it as an existing integration: nothing integrates it. +D-GTM-5 is therefore a **first writer**, not a migration. + +### A live iron-rule violation, found incidentally by 0f + +The five external `bf16_tile_gemm_16x16` references reach it by **two different +paths**, and because 0a proved the bodies divergent, they are calling **different +functions**: + +- `lance-graph/crates/symbiont/src/domino.rs:27` — `use ndarray::simd::{amx_available, amx_report, bf16_tile_gemm_16x16}` → resolves to the **polyfill** (F32x16 decode), *not* a tile op. Its own module doc at `:8` reads *"which `bf16_tile_gemm_16x16` calls before any tile op"*, and it imports `amx_available` alongside — so the call site appears to expect the AMX dispatcher and receives the polyfill. `..._amx` is the name it wants. +- `lance-graph/crates/thinking-engine/examples/amx_bf16_probe.rs:15` — `use ndarray::hpc::bf16_tile_gemm::bf16_tile_gemm_16x16`, reaching **past the facade into `hpc::`**. That is the exact form the workspace iron rule forbids (*"all SIMD from `ndarray::simd`, never `ndarray::hpc::*`, never raw intrinsics"* — lance-graph-java `CLAUDE.md`, `abi.md` §8). It gets the dispatcher, correctly, by an illegal route. + +Both are **lance-graph** call sites, not ndarray's, and `symbiont` is +⊘ DEPRECATED (operator no-go, 2026-08-18) — so this is **reported, not fixed**, and +belongs to whoever next touches those files. ndarray's own facade is correct: the +rename at `simd.rs:714` is precisely the disambiguation these call sites failed to +use. + +### W0 remaining + +`0c` (f64 + tail-shape bench), `0d` (the MKL-ternlog tail hunch), `0e` (the M2 +lookahead × density **ladder**) are measurement probes and are not run here. Note +0e's target is now known to be dead code — the ladder measures a kernel with no +consumers, which is fine for a probe and must not be described as measuring +production behaviour. From dc89eca74e5ecbcaf31035e0be6b485dcc39b784 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 5 Sep 2026 22:37:10 +0000 Subject: [PATCH 02/16] =?UTF-8?q?plans:=20gemm=20consolidation=20v1.2=20?= =?UTF-8?q?=E2=80=94=20grey/white=20matter=20over=20one=20packed=20registe?= =?UTF-8?q?r;=20third=20D-GTM-5=20correction?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit §11 folds in the operator's statement: the hexagon field is digital grey/white matter over ONE packed-address substrate. Grey = local hex state in the existing 96-bit 6x2x8 register, where the same geometry is substrate and mask; white = trie routing read off packed location prefixes (ADDRESS & PREFIX_MASK == PREFIX), never an edge list; TERNLOGQ is the membrane algebra. Invariant: substrate == selection == routing. Reconciled with the measured hex record rather than against it: Q6/Q7/Q8 tested a learned association overlay; this is a compute-and-bytes claim, which is exactly the bar r2il §7.2 sharpened. E-Q8's degree ablation is now a mandatory control. §9 R1 re-graded [S]->[H]: the two sixes are two readings of the same 12 bytes, as le-contract §3 already states. The census finds both existing HHTL arms (blasgraph heel_search's Vec, splat3d's Vec) materializing IDs on the hot path — and my own §9 M1 ArrayVec too. D-GTM-5 is corrected a third time: the pack consumes mask words directly, zero index materialization. K0..K7 read as the SPO 2^3 TriadicProjection masks [H]. Six operator falsifiers D-GTM-0g..0l with one pre-registered kill condition. §11.9 records the Panela primitive and the hex+diamond lattice pair; the diamond's coordination 4 vs the 16-ary nibble trie is flagged [S]. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01EfrUJH3UNnv5NpDH4jDGHq --- .claude/blackboard.md | 39 +++ .../gemm-ternlog-mask-consolidation-v1.md | 222 +++++++++++++++++- 2 files changed, 259 insertions(+), 2 deletions(-) diff --git a/.claude/blackboard.md b/.claude/blackboard.md index b9f8388a..d594a7ef 100644 --- a/.claude/blackboard.md +++ b/.claude/blackboard.md @@ -3,6 +3,45 @@ > **Read this first.** The "Polyglot Notebook" architecture below is a > separate/older program, not the current epoch. +## 2026-09-05 (v1.2) — GEMM plan: grey/white matter over ONE packed register; D-GTM-5 corrected a third time + +**Operator statement folded in as §11.** Hex field = digital grey/white matter over +one packed-address substrate, not two graphs. Grey = local hex state in the +existing 96-bit `6×2×8` register (same geometry is substrate AND mask; learning +changes permeability masks, never pointers). White = trie routing through packed +location (`ADDRESS & PREFIX_MASK == PREFIX`; a tract is `(prefix, mask, learned +transition)`, never a materialized path). TERNLOGQ is the membrane algebra. +**Invariant: `substrate == selection == routing`** — expanding a mask into IDs, +materializing a neighbour list, or converting the trie to an edge table on the hot +path is the loss condition. Hypothesis is NOT "ternlog replaces GEMM": GEMM wins +when dense; hex/trie may win when cognition is successive elimination. + +**Reconciled with the measured hex record, not against it.** Q6/Q7/Q8 tested a +learned *association overlay* (recall/interference); §11 is a *compute + bytes* +claim — the bar r2il §7.2 already sharpened ("wins only as a COMPUTE topology"). +E-Q8's degree-ablation lesson is now a MANDATORY control on every new probe. §9 R1 +re-graded [S]→[H]: adjacency (grey, six neighbours) and carving (white, six rails) +are two readings of the same 12 bytes — `le-contract.md` §3 already says the +register "holds every sanctioned reading at once". + +**The census bites both existing HHTL arms AND my own §9:** blasgraph +`heel_search → Vec` (k=50 per tier) and splat3d `Vec` +both materialize IDs; `splat3d/tile.rs`'s packed `(tile_id<<32 | depth)` key already +conforms. **D-GTM-5 corrected a THIRD time:** v1 `Vec` → v1.1 `ArrayVec` +→ v1.2 `pack_a_masked_f32` consumes mask words directly (tzcnt / vpcompress), +zero index materialization. Each revision removed one layer; the invariant is the +fixed point. + +**K0..K7 [H]** = the SPO 2³ `TriadicProjection` masks (`cam-codebook-resonance- +projection.md`: 8 observation/query masks, "not decorative — the query grammar") += the 8 rows of a ternlog truth table. **Six operator falsifiers** D-GTM-0g..0l +(mask/trie vs GEMM; VPTERNLOGQ residency vs depth; density sweep; crossover; +bytes-materialized/step → 0; prefix-routing codebook entropy) with one +pre-registered kill condition. **Panela + diamond (§11.9):** the toy's +positive/negative shape IS the invariant; diamond lattice (coord 4, tetrahedral) +as white-matter geometry is [S] — it conflicts with the 16-ary nibble trie unless +read as two bits of a level; flagged for one operator word. + ## 2026-09-05 (later) — GEMM consolidation plan v1.1: Mississippi Queen amendment + Wave 0 static results **Operator metaphor folded in (§9).** Mississippi Queen: the river board is laid a diff --git a/.claude/plans/gemm-ternlog-mask-consolidation-v1.md b/.claude/plans/gemm-ternlog-mask-consolidation-v1.md index dbb047ee..f6a598ca 100644 --- a/.claude/plans/gemm-ternlog-mask-consolidation-v1.md +++ b/.claude/plans/gemm-ternlog-mask-consolidation-v1.md @@ -1,5 +1,13 @@ # gemm-ternlog-mask-consolidation-v1 — one GEMM entry per dtype, masks as the prefilter, ternlog as the mask ALU +> **Status:** DRAFT v1.2 (2026-09-05) — §11 folds in the operator's grey/white-matter +> statement: hex (grey, 6×2×8 rails, local permeability) and trie (white, 3×2 path +> prefixes, routing) are ONE 12-byte register read two ways; `substrate == selection == +> routing` is the V3 4+12 facet doctrine as a compute model. §9 R1 re-graded [S]→[H]; +> D-GTM-5 corrected a THIRD time (pack consumes mask words, zero index materialization); +> K0..K7 = the SPO 2³ triadic projections [H]; six operator falsifiers D-GTM-0g..0l with +> the E-Q8 degree-ablation control mandatory. +> > **Status:** DRAFT v1.1 (2026-09-05) — §9 folds in the operator's Mississippi Queen > metaphor: M1 CORRECTS D-GTM-F4/D-GTM-5 (panel-ahead expansion, not a whole-board > prologue), M1b names where the cache lives, M2 re-shapes D-GTM-0e into a ladder, @@ -141,7 +149,7 @@ A 3-input `IMM` is the whole truth table (`simd.rs:561-563`), so any Boolean ove | D-GTM-F1 | `gemm_f32` default = hand-rolled F32x16 `sgemm_blocked` on `avx512f` hosts for square-ish shapes; `matrixmultiply` for skinny/wide rectangles (threshold from D-GTM-0c) and on other hosts. Exact on both. | §1.3: wins 256³–4096³ (up to 7%); LOSES 10–19% at 256×8192×256 and 64×2048×8192 | | D-GTM-F2 | AMX serves `gemm_bf16` and `gemm_i8` ONLY. Any f32 AMX path is a named opt-in carrying its measured table. | §1.3; ndarray#303 | | D-GTM-F3 | The facade is `backend::{gemm_f32, gemm_f64, gemm_bf16, gemm_i8}` (+ batched). Every other GEMM symbol is `pub(crate)`, a documented opt-in, or deleted. | §1.1 count 54→4 | -| D-GTM-F4 | ⊘ **AMENDED by §9 M1/M1b.** Mask→GEMM prefilter expands mask→indices ONE PANEL AHEAD of the pack cursor (stack, no hot-loop alloc), cached at `(mask generation, panel index)` — never a whole-board `Vec` prologue, never a bit-test inside the micro-kernel. | §2.3; mask-risc §14.7; §9 M1 | +| D-GTM-F4 | ⊘ **AMENDED AGAIN by §11.4.** The pack CONSUMES MASK WORDS directly (tzcnt / vpcompress inside the panel window) — **no index list exists at any point**, not a `Vec` prologue (v1), not a panel-ahead `ArrayVec` (v1.1). Cache key stays `(mask generation, panel index)`; the reusable object generalizes to a permeability codebook entry (§11.6). | `substrate == selection == routing`; §11.4 | | D-GTM-F5 | Every accuracy test in this surface uses inputs whose significands exceed 8 bits, and tolerances at f32 grade (1e-5) for f32 APIs. | the vacuous `(i+j)*0.5` test that hid the bf16 loss (#303) | | D-GTM-F6 | Every new kernel lands with a two-sided pin: the fast path must beat the reference by a stated factor AND the reference must still be measurably slower — so a regression in either direction fails. | `three_pass_split_beats_one_bf16_pass` pattern | @@ -167,7 +175,7 @@ A 3-input `IMM` is the whole truth table (`simd.rs:561-563`), so any Boolean ove ### Wave 2 — the mask→GEMM seam (ndarray + lgj-abi, one PR each) -- D-GTM-5 (ndarray): `pruned_gemm_rows` takes a `&[u32]` compacted index list, not a mask; a `mask_to_row_indices(&[u64]) -> Vec` T1 primitive builds it (popcount + expand, one pass). Density crossover per D-GTM-0e. +- D-GTM-5 (ndarray): ⊘ corrected three times, see §11.4 — `pack_a_masked_f32(a, lda, mask, row_cursor, kc, k_start, buf) -> rows_packed` consumes mask words directly; **no index list, no `Vec`, no `ArrayVec`**. `pruned_gemm_rows` has zero callers (§10 0f) so this is a first writer. Ladder per D-GTM-0e; bytes-materialized per D-GTM-0k must be ≈ 0. - D-GTM-6 (lgj-abi, **filed against mask-risc-lowering-v1, not built here**): a carving kind whose payload is that index list, keyed by mask generation, invalidated with the mask. This is the single ask of the other plan. ### Wave 3 — consumers (last, by the STOP rule) @@ -398,3 +406,213 @@ lookahead × density **ladder**) are measurement probes and are not run here. No 0e's target is now known to be dead code — the ladder measures a kernel with no consumers, which is fine for a probe and must not be described as measuring production behaviour. + +## §11 — v1.2: the hex/trie field is ONE packed-address substrate read two ways (operator statement, 2026-09-05) + +### 11.1 The statement, compressed to its load-bearing claims + +> *Treat the hexagon field as digital grey/white matter over one packed-address +> substrate, not as two separately materialized graphs.* + +1. **Grey = local hex state.** Each cell is a fixed-width state carrier — the + existing **96-bit 6×2×8** geometry. The same geometry is substrate AND mask. + Local learning changes *admissibility/permeability masks*, never an external + pointer structure. +2. **White = trie routing through packed location.** Long-range edges are never + object lists. Location is hierarchical *in the address bits*; each prefix is a + trie level/region; navigation is successive prefix refinement. Every prefix is + itself a mask: `ADDRESS & PREFIX_MASK == PREFIX`. A tract is + `(prefix, mask, learned transition)`, not a materialized path. +3. **TERNLOGQ is the membrane algebra.** `U' = ternlog(U, local_membrane_mask, + trie_route_mask)`; the surviving bits ARE the lawful next hex / prefix + refinement. Cached masks (`current_state`, `local_hex_mask`, + `trie_prefix_mask`, `learned_relation_mask`, `attention/focus_mask`) chain in + the same native bit geometry. +4. **Learning is a codebook, not addresses.** `context mask + relation atom + + packed prefix delta → permeability`. After exposure, most structure resolves + through learned vocabulary; only residual novelty alters the membrane. The + R2IL/C64 experiment is the model. +5. **The invariant: `substrate == selection geometry == routing geometry`.** + Expanding a mask into IDs, materializing a neighbour list, or converting the + trie into an edge table *for the hot path* is the loss condition. +6. **The hypothesis is NOT "TERNLOGQ replaces GEMM."** GEMM is attractive when + information is dense; a hex/trie field may win when cognition is mostly + *successive elimination of possibility*. Grey squeezes locally; white moves + the constraint field cheaply across distance. +7. **Underlined:** white matter is not another data structure — it is an + *interpretation of packed location prefixes*. Hexagon supplies neighbourhood; + trie supplies scale; TERNLOGQ supplies permeability. + +### 11.2 Why the measured hex record (Q6 / Q7 / Q8) does NOT close this — and what it DOES bind + +Three board entries measured "hex" and found it wanting: +`E-Q6-HEX-FAILS-CONTENT-ADDRESSING-…-1` (learns less *and* interferes more), +`E-Q7-…-COMPLEMENTARY-NOT-COMPETING-1` (frequency sizing rescues learning, not +interference), `E-Q8-THE-SIX-DOES-NO-WORK-…-1` (degree-1 ablation: identical +completion at 5.5× less memory — *"the information is in the PAIR, not in the +neighbourhood's shape"*). The #1023 audit found **zero** hex adjacency by grep +anywhere in lance-graph / ndarray / OGAR. + +**Those experiments tested a different claim.** Their B-arm was a *learned +association overlay* — a co-occurrence neighbourhood graph for macro recall, +scored on completion / false resonance / interference. §11.1 is a *compute and +memory* claim: propagation cost and bytes materialized under successive +elimination, against GEMM. That is precisely the bar `r2il-machine-semantic- +contract-v1` §7.2 already sharpened: *"if hex wins, it wins as a COMPUTE +topology — never retroactively as an explanation of the 96-bit register."* +§11.1 claims exactly and only that. + +**What the record binds, non-negotiably:** +- **E-Q8's lesson becomes a mandatory control.** *"A locality claim needs a + DEGREE ablation, not only a wiring null."* Every probe in §11.7 runs the hex + arm at degree 6 **and** degree 1; any advantage that survives degree 1 is not + hexagonal and must not be reported as such. +- The ratified demarcation is retained verbatim: *"White Matter ist Wahrheit und + Zwang. Grey Matter ist Hypothese und Plastizität. … Hexagon ist noch gar + nichts außer einem Kandidaten für lokale Rechengeometrie."* Grey is + hypothesis; a learned permeability mask is never a second truth + (`E-*-a-macro-never-becomes-a-second-truth`, the B4 invariant). + +⊘ **§9 R1 is therefore RE-GRADED, [S] → [H]-with-falsifier.** R1 said the game's +six (adjacency) and the substrate's six (carving) were different mechanisms. +That distinction stands — and §11.1 resolves it by assigning them to the two +tissues: adjacency is *grey* (the six neighbours, an unproven compute topology), +carving is *white* (the six rails, the packed address). Not rhyme, not the same +six — two readings of one register (11.3). + +### 11.3 6×2×8 and 3×2 are the SAME twelve bytes — this is already canon + +| reading | what the 12-byte V3 payload means | tissue | +|---|---|---| +| **rails** — `6×(u8:u8)` = 6×2×8 = 96 bits | six `palette256:palette256` pairs; the colon carries the distribution (`E-PALETTE256-IS-A-NEEDLE-THE-COLON-IS-THE-DISTRIBUTION-1`) | **grey** — local state | +| **path** — `HEEL:HIP:TWIG` = 3 tiers × 2 axes = 6 bytes | the CAM-PQ `6×256` code; `path distance = 3 tier-table lookups, O(1)`; longest-prefix binding; `is_ancestor_of` = centroid-tree containment (OGAR `CLAUDE.md` §Tier interpretation) | **white** — routing | + +`le-contract.md` §3 already says it: *"the 12B is a dumb byte register the +ClassView projects — it holds every sanctioned reading at once."* So the +operator's invariant `substrate == selection == routing` is **the V3 +content-blind 4+12 facet doctrine restated as a compute model.** Nothing new is +laid out; the proposal is a way of *executing over* the existing register. That +is what makes it admissible under the STOP rule — no new tissue. + +The L0…L5 ladder (`region → basin → tract → bundle → hex → local state`) is the +nibble-tree: 1 hex digit = 1 level of the 16-ary tree, tier-of-level = `level >> 2` +(a shift, never a branch). The trie already exists; §11.1 asks that it be *read as +a mask* rather than *walked as a structure*. + +### 11.4 Where the tree currently VIOLATES the invariant — including my own §9 + +The loss condition is materializing IDs on the hot path. Census: + +| site | what it materializes | verdict | +|---|---|---| +| `lance-graph/…/blasgraph/heel_hip_twig_leaf.rs` — `heel_search` → `Vec` = `Vec<(usize, u32)>`, `k = 50` survivors **per tier**, sorted + truncated | a gathered row set, four times per query | **violates** — the semantic HHTL arm is the anti-pattern §11.1 names | +| `ndarray/src/hpc/splat3d/depth_cascade.rs` — `cascade_blocks` → per-block `BlockDepthDecision { block_index, tier_reached, action, … }` | an ID-carrying decision per block | **violates** (spatial HHTL arm) — though `HhtlAction::{Reject, KeepCoarse, Refine, ProjectExact, RenderExact}` is already a 5-valued *"lawful next refinement"*, exactly §11.1's surviving-bits semantics wearing an enum | +| `ndarray/src/hpc/splat3d/tile.rs` — packed `u64` key `(tile_id << 32) \| depth_bits`, sorted tile-major | **nothing** — routes by packed-key prefix | **conforms** — the renderer already does white-matter routing by address prefix | +| **this plan, §9 M1** — `next_panel_indices(...) -> ArrayVec` | a per-panel index list | **violates.** Smaller than v1's `Vec`, still an ID expansion on the hot path. | + +⊘ **D-GTM-5 is corrected a THIRD time.** v1: whole-board `Vec`. v1.1 (§9): +panel-ahead `ArrayVec`. v1.2: **the pack consumes mask words directly** — +iterate set bits with `tzcnt` inside the panel window (or `vpcompress` on +AVX-512), gathering rows straight into the packed panel buffer, **zero index +materialization**: + +```rust +// v1.2 — the pack reads the mask; no index list exists at any point +fn pack_a_masked_f32(a: &[f32], lda: usize, mask: &[u64], row_cursor: usize, + kc: usize, k_start: usize, buf: &mut [f32]) -> usize /* rows packed */ +``` + +Each revision removed one more layer of materialization; the operator's +invariant names the fixed point. + +### 11.5 K0…K7 — the eight "terniating" masks are the SPO 2³ triadic projections [H] + +A 3-input `ternlog` immediate is an 8-entry truth table indexed by +`(a<<2)|(b<<1)|c` (`simd.rs:559-563`). Eight is also **exactly** the query +grammar the workspace already carries: + +```rust +// lance-graph/.claude/knowledge/cam-codebook-resonance-projection.md §SPO 2^3 +pub enum TriadicProjection { Abc, AbAskC, AcAskB, BcAskA, AOnly, BOnly, COnly, Background } +``` + +*"For a triad (A, B, C), there are 2³ observation/query masks … The 2³ +structure is not decorative. It is the query grammar for the CAM field."* So +`K0…K7` reads as: the eight presence-patterns of `(S, P, O)`, each selecting +which of the triadic pressures (`S×P → O`, `S×O → P`, `P×O → S`) applies, and +"terniating" = iterating the ternlog truth table over them. **[H] pending one +operator word** — the mapping is exact in cardinality and in role, but the name +`K0..K7` itself is not in the tree. + +### 11.6 What amortizes, precisely + +§9 M1b said the cache key is `(mask generation, panel index)`. §11.1 generalizes +the *tile*: the reusable object is a **codebook entry** +`(context mask, relation atom, packed prefix delta) → permeability mask`. It is +laid once — by whichever traversal first needs it — and every later traversal +that hits the same `(prefix, relation)` crosses it for free. That is the +Mississippi Queen tile at the level of *learned transitions*, and it is what +keeps learned semantics low-entropy (§11.7 probe 6 measures exactly whether it +stays that way). + +### 11.7 The falsification program — six operator probes, plus the E-Q8 control + +Each is a W0 probe; none is production code. Numbering continues §5. + +| id | probe (operator's words) | the measurement | the E-Q8 control | +|---|---|---|---| +| D-GTM-0g | mask/trie propagation vs dense GEMM for an equivalent sparse learned transition | wall time + result equality, same transition matrix realized both ways | hex arm at degree 6 **and** 1 | +| D-GTM-0h | does chained VPTERNLOGQ stay register/cache resident as depth increases | `perf` L1/L2 miss rate vs chain depth 1…32; the knee is the finding | — | +| D-GTM-0i | sweep unknown density almost-empty → almost-full | density ∈ {1,5,10,25,50,75,90,99}% | — | +| D-GTM-0j | the crossover where GEMM wins | from 0g × 0i: a density-vs-size frontier, not a single number | — | +| D-GTM-0k | **bytes materialized per inference step** | count every heap/stack byte that is not the mask itself; the hex/trie path must approach **zero** | this is the invariant's own falsifier | +| D-GTM-0l | can packed-prefix routing express all required long-range transitions without exploding codebook entropy | entries needed vs transitions covered, on the R2IL/C64 ore; the C64 vocabulary-resolution rate is the reference | — | + +**Pre-registered kill condition (one, stated before any run):** if at every +density in 0i the GEMM arm is both faster AND materializes fewer bytes than the +hex/trie arm, §11.1's hypothesis is false *for this substrate* and is recorded +as such — the same way Q6 was. + +### 11.8 Restated non-goals + +- Not "TERNLOGQ replaces GEMM" (§11.1 pt 6 — the operator's own fence). +- Not a new layout, a new crate, or a second graph. The register is the V3 + facet; the trie is the nibble tree; both exist. +- Not a re-run of Q6/Q7/Q8's association task. Different claim, different + metrics, same degree-ablation discipline. + +### 11.9 The Panela primitive, and hex + diamond as the two lattices (operator infographic, 2026-09-05) + +**Panela** — the DDR construction toy: one flat piece with an E-E comb profile that +interlocks with its own kind. *"One shape. Many worlds. Positive shape = +connectivity. Negative space = admissibility."* That is the §11.1 invariant as a +physical object: the same piece IS the structure and IS the selection, and the +infographic draws the 96-bit `6×2×8` register (six rows L0…L5 × byte 0 / byte 1) +**as** the E-profile. Substrate S and mask M are the same silhouette; ternlog +combines them without either ever leaving the lane. + +**Two lattices, one per tissue** — "from planar to space-filling": + +| lattice | coordination | character | tissue | +|---|---|---|---| +| **hexagonal** (Bienenwaben) | 6, isotropic, planar | efficient packing, natural for local reasoning | **grey** — dense local recurrence | +| **diamond** (Bindungen) | 4, tetrahedral, 3-D | strong directional bonds, efficient long-range structure | **white** — cross-scale tracts | + +A hex sheet per level, diamond links between levels: the cortical sheet with its +tracts. This is the first *geometric* statement of white matter in this plan — +§11.1 gave it as an *interpretation of prefixes*; the diamond gives it a shape. + +**One mismatch to flag, not resolve.** A diamond lattice has coordination **4**; +the canon's trie is **16-ary** (1 hex digit = 1 level, `FAN_OUT=16`, tier-of-level += `level >> 2`). A tetrahedral tract therefore does not map one-to-one onto a +nibble level — it maps onto **two bits** of one. Either the diamond is a +*visualization* of the four sub-branches a nibble refines through (in which case +it is rhyme, and harmless), or it is a claim that white-matter routing is 4-ary +at each step (in which case it conflicts with the 3×4 vs 4×3 standing watch and +needs its own probe). **Graded [S] until the operator says which.** Nothing in +§11.7's six probes depends on it. + +**The neuron reading** (dendrites = incoming masks, membrane = permeability mask, +axon = outgoing mask, learning = mask update, state = the 96-bit substrate) and +**constraint soaking** (`U₁ = U₀ ⊗ M₁`, `U₂ = U₁ ⊗ M₂`, … until residual) are §11.1 +points 1 and 6 restated; recorded as vocabulary, not as new claims. From 8c9e80fb444519ec87e0deae995be70b08fe01a3 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 5 Sep 2026 22:40:10 +0000 Subject: [PATCH 03/16] =?UTF-8?q?plans:=20gemm=20consolidation=20v1.3=20?= =?UTF-8?q?=E2=80=94=20substrate=20=3D=3D=20mask=20geometry=20=3D=3D=20pro?= =?UTF-8?q?jection=20surface?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit §11.10 records the operator's strengthened invariant. The 3-D field (diamond tracts, the "cube") is never allocated: it is a mask-address projection of the 2-D 6x2x8 surface, recovered only when a question requires it. "Holographic" gets a falsifiable definition — the information to reconstruct the relevant 3-D relation is distributed through the 2-D representation — and the two probes that test it are already in §11.7: D-GTM-0l (recover the relation without codebook entropy exploding) paired with D-GTM-0k (bytes materialized per step approaching zero). VPTERNLOGQ stays dumb by design; meaning lives in how the fields are laid out. Resolves §11.9's diamond flag [S]->[H]: the tetrahedral bonds are implied by address+masks, not walked, so their coordination number never meets the 16-ary nibble trie. No arity probe needed. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01EfrUJH3UNnv5NpDH4jDGHq --- .claude/blackboard.md | 16 ++++ .../gemm-ternlog-mask-consolidation-v1.md | 73 ++++++++++++++++++- 2 files changed, 88 insertions(+), 1 deletion(-) diff --git a/.claude/blackboard.md b/.claude/blackboard.md index d594a7ef..86ee07c8 100644 --- a/.claude/blackboard.md +++ b/.claude/blackboard.md @@ -3,6 +3,22 @@ > **Read this first.** The "Polyglot Notebook" architecture below is a > separate/older program, not the current epoch. +## 2026-09-05 (v1.3) — the invariant strengthened: `substrate == mask geometry == projection surface` + +Operator: "make the 96-bit object holographic" was metaphor while the cube wanted +voxel-by-voxel states. The Panela/photolithographic layer fixes it — **the cube is +never stored.** Holographic now means: *the information to reconstruct/address the +relevant 3-D relation is distributed through the 2-D 6×2×8 surface*, and the +volume appears only when a question requires it. Depth ← packed location; local +curvature ← hex adjacency; scale ← trie prefixes; permeability ← masks; dumb +physics ← VPTERNLOGQ (meaning lives in layout, never in the instruction — +`membrane-tiers.md` T1 from the other side). Hardware inversion: don't flatten +3-D onto silicon; make the higher-dimensional object a mask-address projection of +the 2-D surface. **Resolves the §11.9 diamond flag [S]→[H]:** bonds are implied +by address+masks, not walked, so coordination-4 vs the 16-ary trie is not a +conflict. The hologram's test already sits in the program: recover the relation +(D-GTM-0l, codebook entropy) without allocating it (D-GTM-0k, bytes/step → 0). + ## 2026-09-05 (v1.2) — GEMM plan: grey/white matter over ONE packed register; D-GTM-5 corrected a third time **Operator statement folded in as §11.** Hex field = digital grey/white matter over diff --git a/.claude/plans/gemm-ternlog-mask-consolidation-v1.md b/.claude/plans/gemm-ternlog-mask-consolidation-v1.md index f6a598ca..b411d773 100644 --- a/.claude/plans/gemm-ternlog-mask-consolidation-v1.md +++ b/.claude/plans/gemm-ternlog-mask-consolidation-v1.md @@ -1,5 +1,12 @@ # gemm-ternlog-mask-consolidation-v1 — one GEMM entry per dtype, masks as the prefilter, ternlog as the mask ALU +> **Status:** DRAFT v1.3 (2026-09-05) — §11.10 strengthens the invariant to +> `substrate == mask geometry == projection surface`: the 3-D field (diamond tracts, the +> "cube") is never allocated — it is a mask-address projection of the 2-D 6×2×8 surface, +> recovered only when a question requires it. "Holographic" gets a falsifiable definition +> (recoverability, D-GTM-0l) paired with non-allocation (D-GTM-0k). §11.9's diamond flag +> resolved [S]→[H]: bonds are implied, not walked, so no arity conflict with the trie. +> > **Status:** DRAFT v1.2 (2026-09-05) — §11 folds in the operator's grey/white-matter > statement: hex (grey, 6×2×8 rails, local permeability) and trie (white, 3×2 path > prefixes, routing) are ONE 12-byte register read two ways; `substrate == selection == @@ -149,7 +156,7 @@ A 3-input `IMM` is the whole truth table (`simd.rs:561-563`), so any Boolean ove | D-GTM-F1 | `gemm_f32` default = hand-rolled F32x16 `sgemm_blocked` on `avx512f` hosts for square-ish shapes; `matrixmultiply` for skinny/wide rectangles (threshold from D-GTM-0c) and on other hosts. Exact on both. | §1.3: wins 256³–4096³ (up to 7%); LOSES 10–19% at 256×8192×256 and 64×2048×8192 | | D-GTM-F2 | AMX serves `gemm_bf16` and `gemm_i8` ONLY. Any f32 AMX path is a named opt-in carrying its measured table. | §1.3; ndarray#303 | | D-GTM-F3 | The facade is `backend::{gemm_f32, gemm_f64, gemm_bf16, gemm_i8}` (+ batched). Every other GEMM symbol is `pub(crate)`, a documented opt-in, or deleted. | §1.1 count 54→4 | -| D-GTM-F4 | ⊘ **AMENDED AGAIN by §11.4.** The pack CONSUMES MASK WORDS directly (tzcnt / vpcompress inside the panel window) — **no index list exists at any point**, not a `Vec` prologue (v1), not a panel-ahead `ArrayVec` (v1.1). Cache key stays `(mask generation, panel index)`; the reusable object generalizes to a permeability codebook entry (§11.6). | `substrate == selection == routing`; §11.4 | +| D-GTM-F4 | ⊘ **AMENDED AGAIN by §11.4.** The pack CONSUMES MASK WORDS directly (tzcnt / vpcompress inside the panel window) — **no index list exists at any point**, not a `Vec` prologue (v1), not a panel-ahead `ArrayVec` (v1.1). Cache key stays `(mask generation, panel index)`; the reusable object generalizes to a permeability codebook entry (§11.6). | `substrate == mask geometry == projection surface` (§11.10, strengthening §11.4) | | D-GTM-F5 | Every accuracy test in this surface uses inputs whose significands exceed 8 bits, and tolerances at f32 grade (1e-5) for f32 APIs. | the vacuous `(i+j)*0.5` test that hid the bf16 loss (#303) | | D-GTM-F6 | Every new kernel lands with a two-sided pin: the fast path must beat the reference by a stated factor AND the reference must still be measurably slower — so a regression in either direction fails. | `three_pass_split_beats_one_bf16_pass` pattern | @@ -616,3 +623,67 @@ needs its own probe). **Graded [S] until the operator says which.** Nothing in axon = outgoing mask, learning = mask update, state = the 96-bit substrate) and **constraint soaking** (`U₁ = U₀ ⊗ M₁`, `U₂ = U₁ ⊗ M₂`, … until residual) are §11.1 points 1 and 6 restated; recorded as vocabulary, not as new claims. + +### 11.10 The amended invariant: `substrate == mask geometry == projection surface` (operator, 2026-09-05) + +**What was missing, in the operator's words.** *"Make the 96-bit object +holographic"* risked being metaphor, because a volumetric cube wants a huge +number of independently addressable states — the 3-D geometry existed with no +economical way of specifying its interior. The Panela / photolithographic layer +closes that: **the cube is never stored voxel-by-voxel.** + +``` +apparent 3-D field (diamond bonds, tracts, the "cube") ← NOT materialized + ⇅ projection +planar bit geometry: EE EE EE … = 6×2×8 register, masks + packed location +``` + +**Holographic, defined falsifiably:** not "every voxel exists" but *the +information required to reconstruct / address the relevant 3-D relation is +distributed through the 2-D representation.* You don't fill the hologram with +bits; you fill the surface with enough invariants that the volume can be +recovered *when something asks a question that requires it.* + +**Assignment of parts** (replaces §11.1 pt 7): + +| supplies | from | +|---|---| +| depth | packed location | +| local curvature | hex adjacency | +| scale | trie prefixes | +| permeability | masks | +| the dumb physics | VPTERNLOGQ | + +**The invariant, strengthened.** §11.1's `substrate == selection geometry == +routing geometry` becomes + +> **`substrate == mask geometry == projection surface`** + +The hardware inversion is the point: not *3-D problem → flatten awkwardly onto +silicon*, but *embrace the 2-D silicon-like representation and make the +higher-dimensional object a mask-address projection of it.* VPTERNLOGQ stays +wonderfully boring — three bit fields, one Boolean function, no knowledge of +neurons, cubes, hexagons or ontology. **The meaning lives in how the fields are +laid out, not in the instruction executing them** — which is `membrane-tiers.md`'s +T1 rule (a primitive is dumb; the tier above it carries the meaning) stated from +the other side. + +⊘ **§11.9's diamond flag is RESOLVED, [S] → [H].** The mismatch I flagged — a +tetrahedral tract (coordination 4) against the 16-ary nibble trie — assumed the +diamond bonds were *walked*. They are not: they are *implied* by address + masks +and recovered on demand. Nothing is 4-ary per step because nothing steps. The +diamond is the shape of the projection, not a level of the trie. No conflict with +the 3×4 standing watch; nothing to probe about arity. + +**What IS now falsifiable, and where it already sits in §11.7:** + +- *Holographic recoverability* = **D-GTM-0l**: can packed-prefix routing + + masks + codebook express every required long-range relation without codebook + entropy exploding? If the surface's invariants are insufficient, the volume + cannot be recovered and the claim fails there — measured on the R2IL/C64 ore. +- *The volume is never allocated* = **D-GTM-0k**: bytes materialized per + inference step → 0. A "projection" that secretly allocates the cube shows up + as bytes. + +Those two probes together are the hologram's test: recover the relation (0l) +without allocating it (0k). Nothing else in the program changes. From 284b9995d6481aad3fc2e3572d047002bc780386 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 5 Sep 2026 22:48:25 +0000 Subject: [PATCH 04/16] =?UTF-8?q?probe:=20W0=20D-GTM-0g/0h/0i/0j/0k=20?= =?UTF-8?q?=E2=80=94=20mask/trie=20vs=20GEMM=20measured;=200j=20falsifies?= =?UTF-8?q?=20the=20density=20framing?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds examples/hex_trie_vs_gemm_probe.rs (--release, #[ignore]-free but example-gated) and records §12. 0k ANSWERED cleanly: the mask hot path materializes 0 bytes/step at every density and depth, both relation shapes, measured with a counting global allocator rather than asserted. GEMM materializes 73,728 B/step. 0j FALSIFIES the plan's own framing. §11.1 pt 6 said "GEMM is attractive when information is dense"; measured, there is no density crossover at all — masks win 745x at 0.02% relation density and 297x at 100%, because both costs are flat in density (GEMM O(N^2) FMAs, mask O(active*N/64) word ORs). The honest boundary is a TYPE boundary: masks win whenever the relation is Boolean; GEMM is required when the relation carries VALUES. §12.5 states plainly that the headline numbers are NOT evidence for the hypothesis: a dense f32 matrix for a 0/1 relation is a mis-specified baseline (a bitmask is 32x denser before any algorithm runs), and the missing arm is CSR SpMV, which would plausibly cross at low density. The one internally fair comparison is PREFIX vs RANDOM within the mask arm — 21-79 ns flat vs 305-35,522 ns scaling with active bits. The probe's correctness gate earned its place: it caught a real bug in the probe itself (the mask arm must union the TRANSPOSE; unioning srcs(i) agrees with the GEMM arm only for a symmetric relation, which bucket membership is and a random relation is not). A second flaw — mask timings at or below timer resolution making every early "speedup" a noise ratio — was caught by reading and fixed with a 50 ms floor. 0h is graded [S]: perf is unavailable in this sandbox, so residency is inferred from timing and that limit is stated, not hidden. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01EfrUJH3UNnv5NpDH4jDGHq --- .../gemm-ternlog-mask-consolidation-v1.md | 115 ++++++ Cargo.toml | 4 + examples/hex_trie_vs_gemm_probe.rs | 358 ++++++++++++++++++ 3 files changed, 477 insertions(+) create mode 100644 examples/hex_trie_vs_gemm_probe.rs diff --git a/.claude/plans/gemm-ternlog-mask-consolidation-v1.md b/.claude/plans/gemm-ternlog-mask-consolidation-v1.md index b411d773..49bc93b4 100644 --- a/.claude/plans/gemm-ternlog-mask-consolidation-v1.md +++ b/.claude/plans/gemm-ternlog-mask-consolidation-v1.md @@ -1,5 +1,13 @@ # gemm-ternlog-mask-consolidation-v1 — one GEMM entry per dtype, masks as the prefilter, ternlog as the mask ALU +> **Status:** DRAFT v1.4 (2026-09-05) — §12 RUNS D-GTM-0g/0h/0i/0j/0k. 0k passes +> cleanly (mask hot path = 0 bytes/step, measured by a counting allocator). **0j +> falsifies the framing:** there is no density crossover — masks win 745×→297× from +> 0.02% to 100% relation density — because "GEMM wins when dense" is wrong for a +> BOOLEAN relation; the real boundary is a TYPE one (Boolean ⇒ masks, valued ⇒ GEMM). +> The dense-f32 baseline is mis-specified and the headline numbers are explicitly NOT +> evidence (§12.5); the missing arm is CSR SpMV. D-GTM-0l is now the decisive probe. +> > **Status:** DRAFT v1.3 (2026-09-05) — §11.10 strengthens the invariant to > `substrate == mask geometry == projection surface`: the 3-D field (diamond tracts, the > "cube") is never allocated — it is a mask-address projection of the 2-D 6×2×8 surface, @@ -687,3 +695,110 @@ the 3×4 standing watch; nothing to probe about arity. Those two probes together are the hologram's test: recover the relation (0l) without allocating it (0k). Nothing else in the program changes. + +## §12 — WAVE 0 MEASURED: D-GTM-0g/0h/0i/0j/0k run, and the hypothesis's own framing is corrected + +Probe: `examples/hex_trie_vs_gemm_probe.rs` (`--release`, committed). N = 4096, +mask = 512 B, dense relation matrix = 64 MB. Task, identical for both arms: +`D` steps of `state = R(state) ∩ constraint`, with a **correctness gate** — +survivor counts must match or the run aborts. + +### 12.1 The gate fired, and it found a bug in the probe itself + +First run: PREFIX passed at every cell, RANDOM failed immediately +(`gemm 912 vs mask 930`). Cause: the GEMM arm computes +`{ i : srcs(i) ∩ active ≠ ∅ }` while the mask arm was unioning `srcs(i)` over +active `i`. **Those agree only for a SYMMETRIC relation** — true of bucket +membership, false of a random relation. The mask arm must union the TRANSPOSE +(`fwd[j]` = what `j` can activate). Fixed; recorded because the asymmetry is +easy to reintroduce and the gate is the only thing that catches it. + +A second flaw was caught by reading, not by any gate: the mask arm first timed +at 0.000–0.001 ms — **at or below timer resolution**, so every "speedup" in that +table (25,940× … 853,300×) was a ratio against quantization noise. Both arms now +run to a 50 ms floor and report ns/step. + +### 12.2 What was measured (all cells passed the equality gate) + +**State-density sweep**, depth ∈ {1, 8, 32}: + +| relation | mask ns/step | GEMM ns/step | mask B/step | GEMM B/step | +|---|---|---|---|---| +| PREFIX (structured) | **21–79**, flat in state density | 8.4–8.8 M | **0** | 73,728 | +| RANDOM (no structure) | 305 (1%) → 35,522 (99%), scales with active bits | 10.6–13.0 M | **0** | 73,728 | + +**Relation-density sweep** (RANDOM, state 50%, depth 8) — the axis the first +table missed: + +| edges/row | relation density | GEMM ns/step | mask ns/step | speedup | +|---|---|---|---|---| +| 1 | 0.02% | 8,758,129 | 11,743 | 745.8× | +| 16 | 0.39% | 12,050,934 | 42,601 | 282.9× | +| 64 | 1.56% | 12,309,149 | 41,855 | 294.1× | +| 256 | 6.25% | 13,112,322 | 42,667 | 307.3× | +| 1024 | 25.00% | 12,525,352 | 42,559 | 294.3× | +| **4096** | **100.00%** | 12,776,345 | 43,024 | **297.0×** | + +### 12.3 D-GTM-0k — ANSWERED, and it is the one clean result + +**Mask hot path: 0 bytes/step at every density, every depth, both relation +shapes.** GEMM: 73,728 B/step (a packing buffer inside `gemm_f32`). The +invariant's own falsifier passes — nothing is materialized on the mask path, +measured by a counting global allocator rather than asserted. + +### 12.4 D-GTM-0j — there is NO crossover, and that falsifies the framing rather than confirming it + +§11.1 pt 6 says *"GEMM is attractive when information is dense; a hex/trie field +may win when cognition is mostly successive elimination."* **Measured, the +density axis does not produce a crossover at all** — the mask arm wins by ~300× +at 0.02% and by 297× at 100%. Both costs are flat in relation density: GEMM pays +`O(N²)` FMAs regardless, the mask arm pays `O(active · N/64)` word-ORs +regardless. + +⊘ **So "GEMM wins when dense" is FALSE as stated for a Boolean relation.** The +honest correction, and it is a TYPE boundary rather than a density: + +> **Masks win whenever the relation is Boolean; GEMM is required when the +> relation carries VALUES.** A bitmask is 32× denser than f32 *before any +> algorithm runs*, so a Boolean relation in f32 was never the right +> representation. Where a weight must be accumulated (evidence strength, a +> learned probability, a distance), the mask arm cannot express the operation at +> all — that, not density, is where GEMM becomes mandatory. + +### 12.5 The baseline is mis-specified, and the headline numbers are NOT evidence + +Stated plainly so no future session cites 297× as support: + +1. **A dense f32 matrix for a 0/1 relation is an unfair baseline.** The mask arm + is not beating GEMM; it is beating a 32×-wasteful *representation* of a + Boolean relation. The number is real and the credit is misattributed. +2. **The missing arm is CSR SpMV.** A sparse f32 baseline costs `O(nnz)`, so at + `deg = 1` it is ~4,096 FMAs — the same order as the mask arm's 11.7 µs, and + it would plausibly cross. Until that arm is built, no claim about "mask beats + sparse GEMM" is available, and none is made here. +3. **The structured/unstructured separation is the one comparison that IS + internally fair** (same arm, same representation, same task): PREFIX 21–79 ns + flat vs RANDOM 305–35,522 ns scaling with active bits. ~3 orders of + magnitude, and it degrades exactly where the mask arm has no structure to + exploit — which is the E-Q8 discipline applied to compute rather than recall. + +### 12.6 D-GTM-0h — a limit of the sandbox, stated rather than papered over + +`perf` is unavailable here, so register/cache residency could not be measured +with counters. The timing proxy is suggestive only: PREFIX per-step cost *falls* +from ~70 ns at depth 1 to ~21–27 ns at depths 8 and 32 (loop-invariant setup +amortizing over more steps), with no knee up to depth 32 — consistent with +staying resident, and **not proof of it**. Graded [S] pending a machine with +counters. + +### 12.7 What W0 now leaves open + +- **D-GTM-0g** — partially answered; needs the CSR SpMV arm (12.5 pt 2) before + "mask/trie vs GEMM" means anything beyond "packed beats unpacked". +- **D-GTM-0l** — unrun. It is now the *decisive* probe: with the density axis + dead, the hypothesis stands or falls on whether packed-prefix routing can + express real long-range relations without codebook entropy exploding, on the + R2IL/C64 ore. +- **The weighted arm** — new, implied by 12.4: the type boundary needs its own + measurement (where does accumulating a value force GEMM?). +- 0c / 0d / 0e remain unrun. diff --git a/Cargo.toml b/Cargo.toml index ef7f9d9e..c1f0987b 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -51,6 +51,10 @@ required-features = ["splat3d"] # AMX examples import `ndarray::simd` / `ndarray::hpc`, both `#[cfg(feature = # "std")]`, so they must be skipped in `--no-default-features` CI jobs. +[[example]] +name = "hex_trie_vs_gemm_probe" +required-features = ["std"] + [[example]] name = "amx_gemm_bench" required-features = ["std"] diff --git a/examples/hex_trie_vs_gemm_probe.rs b/examples/hex_trie_vs_gemm_probe.rs new file mode 100644 index 00000000..dfcfe28c --- /dev/null +++ b/examples/hex_trie_vs_gemm_probe.rs @@ -0,0 +1,358 @@ +//! W0 probes D-GTM-0g/0h/0i/0j/0k — the mask/trie field against dense GEMM. +//! +//! Plan: `.claude/plans/gemm-ternlog-mask-consolidation-v1.md` §11.7. +//! Hypothesis under test (§11.1 pt 6, operator's own fence): **NOT** "TERNLOGQ +//! replaces GEMM". GEMM is attractive when information is dense; a learned +//! hex/trie field may win when cognition is mostly successive elimination of +//! possibility. The invariant (§11.10): `substrate == mask geometry == +//! projection surface` — expanding a mask into IDs or materializing a neighbour +//! list on the hot path is the loss condition. +//! +//! ## The task, identical for both arms +//! +//! `D` steps of `state = R(state) ∩ constraint_i` over `N` items — one hop +//! through a relation, then a filter. Both arms must return the same survivor +//! set; a mismatch aborts the run (a comparison of two different computations +//! measures nothing). +//! +//! ## Two relation shapes — this is what makes the probe two-sided +//! +//! * `Prefix` — `R(x)` = items sharing `x`'s prefix. The hypothesis's home +//! turf: successors of a SET are the union of touched prefix buckets, and a +//! bucket's member mask is a contiguous run of words, so propagation is a +//! word scan with no ID list and no allocation. +//! * `Random` — an arbitrary sparse relation with the SAME edge count and no +//! prefix structure. The mask arm has nothing to exploit and must OR one +//! successor row per active bit. If the mask arm does not lose here, the +//! probe is rigged. +//! +//! ## Metrics +//! +//! * time/step (0g, 0j) across density (0i) and chain depth (0h — a timing +//! proxy; `perf` is unavailable in this sandbox, so residency is inferred +//! from time/step vs depth, and that limit is stated rather than hidden). +//! * **bytes materialized per step (0k)** — a counting allocator, reported +//! separately for one-time setup (amortized per §11.6) and the hot path. The +//! invariant predicts the mask arm's hot path approaches zero. +//! +//! cargo run --release --example hex_trie_vs_gemm_probe --features std + +use std::alloc::{GlobalAlloc, Layout, System}; +use std::sync::atomic::{AtomicUsize, Ordering}; +use std::time::Instant; + +// ── counting allocator: the 0k instrument ──────────────────────────────────── +static ALLOCED: AtomicUsize = AtomicUsize::new(0); +static COUNTING: AtomicUsize = AtomicUsize::new(0); + +struct Counting; +unsafe impl GlobalAlloc for Counting { + unsafe fn alloc(&self, l: Layout) -> *mut u8 { + if COUNTING.load(Ordering::Relaxed) == 1 { + ALLOCED.fetch_add(l.size(), Ordering::Relaxed); + } + unsafe { System.alloc(l) } + } + unsafe fn dealloc(&self, p: *mut u8, l: Layout) { + unsafe { System.dealloc(p, l) } + } +} +#[global_allocator] +static A: Counting = Counting; + +fn count_on() { + ALLOCED.store(0, Ordering::Relaxed); + COUNTING.store(1, Ordering::Relaxed); +} +fn count_off() -> usize { + COUNTING.store(0, Ordering::Relaxed); + ALLOCED.load(Ordering::Relaxed) +} + +fn splitmix(s: &mut u64) -> u64 { + *s = s.wrapping_add(0x9E37_79B9_7F4A_7C15); + let mut z = *s; + z = (z ^ (z >> 30)).wrapping_mul(0xBF58_476D_1CE4_E5B9); + z = (z ^ (z >> 27)).wrapping_mul(0x94D0_49BB_1331_11EB); + z ^ (z >> 31) +} + +const N: usize = 4096; +const W: usize = N / 64; // 64 mask words = 512 bytes +const BUCKET: usize = 64; // prefix bucket = 1 word = 64 items + +#[derive(Clone, Copy, PartialEq)] +enum Rel { + Prefix, + Random, +} + +/// Dense 0/1 relation matrix, `N×N` f32 — what the GEMM arm needs materialized. +fn build_matrix_deg(rel: Rel, seed: u64, deg: usize) -> Vec { + let mut m = vec![0.0f32; N * N]; + let mut s = seed; + match rel { + Rel::Prefix => { + // successors of x = x's bucket. Edge count = N * BUCKET. + for i in 0..N { + let b = i / BUCKET; + for j in b * BUCKET..(b + 1) * BUCKET { + m[i * N + j] = 1.0; + } + } + } + Rel::Random => { + // SAME edge count, no prefix structure. + for i in 0..N { + for _ in 0..deg { + let j = (splitmix(&mut s) as usize) % N; + m[i * N + j] = 1.0; + } + } + } + } + m +} + +/// Forward masks — `fwd[j]` = the items `j` can activate, i.e. the TRANSPOSE of +/// the relation matrix's rows. +/// +/// The transpose is load-bearing and the probe's own correctness gate is what +/// found it: the GEMM arm computes `{ i : srcs(i) ∩ active ≠ ∅ }`, so the mask +/// arm must union `fwd[j]` over active `j`, never `srcs(i)` over active `i`. +/// Those two agree only for a SYMMETRIC relation — true of bucket membership, +/// false of a random relation, which is exactly where the gate fired. +fn build_fwd_masks(matrix: &[f32]) -> Vec { + let mut t = vec![0u64; N * W]; + for i in 0..N { + for j in 0..N { + if matrix[i * N + j] != 0.0 { + t[j * W + i / 64] |= 1u64 << (i % 64); + } + } + } + t +} + +fn dense_state(mask: &[u64]) -> Vec { + (0..N) + .map(|i| if mask[i / 64] >> (i % 64) & 1 == 1 { 1.0 } else { 0.0 }) + .collect() +} + +/// GEMM arm: one hop = matvec through the dense relation, threshold, filter. +fn step_gemm(matrix: &[f32], state: &mut [f32], scratch: &mut [f32], constraint: &[f32]) { + ndarray::backend::gemm_f32(N, 1, N, 1.0, matrix, N, state, 1, 0.0, scratch, 1); + for i in 0..N { + state[i] = if scratch[i] > 0.0 { constraint[i] } else { 0.0 }; + } +} + +/// Mask arm, `Prefix`: propagation is a word scan — a non-empty bucket becomes +/// full. No ID list, no allocation, no gather. +fn step_mask_prefix(state: &mut [u64], c1: &[u64], c2: &[u64]) { + for w in state.iter_mut() { + if *w != 0 { + *w = u64::MAX; // bucket == one word: the union of its member mask + } + } + // filter: state &= c1 & c2 — one VPTERNLOGQ per 512 bits + ndarray::simd::mask_ternlog_assign::<{ ndarray::simd::ternlog::AND3 }>(state, c1, c2); +} + +/// Mask arm, `Random`: no structure to exploit — OR one successor row per +/// active bit. This is the fallback the invariant calls the loss condition. +fn step_mask_random(state: &mut [u64], fwd: &[u64], out: &mut [u64], c1: &[u64], c2: &[u64]) { + out.fill(0); + for wi in 0..W { + let mut word = state[wi]; + while word != 0 { + let b = word.trailing_zeros() as usize; + word &= word - 1; + let row = (wi * 64 + b) * W; + for k in 0..W { + out[k] |= fwd[row + k]; + } + } + } + state.copy_from_slice(out); + ndarray::simd::mask_ternlog_assign::<{ ndarray::simd::ternlog::AND3 }>(state, c1, c2); +} + +fn popcnt(m: &[u64]) -> u32 { + m.iter().map(|w| w.count_ones()).sum() +} + +fn main() { + println!("hex/trie vs GEMM — N={N}, mask={} B, matrix={} MB\n", W * 8, N * N * 4 / 1_048_576); + + for rel in [Rel::Prefix, Rel::Random] { + let name = if rel == Rel::Prefix { + "PREFIX (structured)" + } else { + "RANDOM (no structure)" + }; + println!("═══ relation: {name} ═══"); + + count_on(); + let matrix = build_matrix_deg(rel, 0xC0FFEE, BUCKET); + let gemm_setup = count_off(); + count_on(); + let succ = build_fwd_masks(&matrix); + let mask_setup = count_off(); + + println!(" setup bytes (amortized, §11.6): GEMM {:>10} MASK {:>10}", gemm_setup, mask_setup); + println!( + " {:>6} {:>7} {:>11} {:>11} {:>9} {:>13} {:>13}", + "dens%", "depth", "gemm ns/st", "mask ns/st", "speedup", "gemm B/step", "mask B/step" + ); + + for &dens in &[1usize, 5, 10, 25, 50, 75, 90, 99] { + for &depth in &[1usize, 8, 32] { + // initial state at the requested density + let mut s = 0x5EEDu64 ^ (dens as u64) << 8; + let mut m0 = vec![0u64; W]; + for i in 0..N { + if (splitmix(&mut s) % 100) < dens as u64 { + m0[i / 64] |= 1u64 << (i % 64); + } + } + // constraints: two learned permeability masks per step + let mut c1 = vec![0u64; W]; + let mut c2 = vec![0u64; W]; + for w in 0..W { + c1[w] = splitmix(&mut s) | splitmix(&mut s); + c2[w] = splitmix(&mut s) | splitmix(&mut s); + } + let (c1f, c2f) = (dense_state(&c1), dense_state(&c2)); + let dense_m0 = dense_state(&m0); + + // ── GEMM arm ── (repeat to a 50 ms floor; a sub-resolution + // timing divided into a ratio is noise, not a speedup) + let mut gs = dense_state(&m0); + let mut scratch = vec![0.0f32; N]; + let mut reps = 0usize; + count_on(); + let t = Instant::now(); + while t.elapsed().as_secs_f64() < 0.05 { + gs.copy_from_slice(&dense_m0); + for _ in 0..depth { + step_gemm(&matrix, &mut gs, &mut scratch, &c1f); + for i in 0..N { + gs[i] *= c2f[i]; + } + } + reps += 1; + } + let el = t.elapsed().as_secs_f64(); + let gemm_b = count_off() / reps.max(1); + let gemm_ns = el * 1e9 / (reps * depth) as f64; + std::hint::black_box(&gs); + + // ── MASK arm ── + let mut ms_ = m0.clone(); + let mut out = vec![0u64; W]; + let mut mreps = 0usize; + count_on(); + let t = Instant::now(); + while t.elapsed().as_secs_f64() < 0.05 { + ms_.copy_from_slice(&m0); + for _ in 0..depth { + match rel { + Rel::Prefix => step_mask_prefix(&mut ms_, &c1, &c2), + Rel::Random => step_mask_random(&mut ms_, &succ, &mut out, &c1, &c2), + } + } + mreps += 1; + } + let el = t.elapsed().as_secs_f64(); + let mask_b = count_off() / mreps.max(1); + let mask_ns = el * 1e9 / (mreps * depth) as f64; + std::hint::black_box(&ms_); + + // ── correctness gate: same survivors, or the numbers mean nothing ── + let gemm_pop = gs.iter().filter(|&&v| v > 0.0).count() as u32; + let mask_pop = popcnt(&ms_); + assert_eq!( + gemm_pop, mask_pop, + "ARMS DISAGREE at dens={dens} depth={depth} ({name}): gemm {gemm_pop} vs mask {mask_pop}" + ); + + println!( + " {:>6} {:>7} {:>11.3} {:>11.3} {:>8.1}x {:>13} {:>13}", + dens, + depth, + gemm_ns, + mask_ns, + gemm_ns / mask_ns.max(1e-9), + gemm_b, + mask_b + ); + } + } + println!(); + } + + // ── D-GTM-0i, the axis the first table missed: RELATION density ────────── + // State density fixed at 50%, depth 8, RANDOM relation (no structure to + // exploit), sweeping edges-per-row from 1 to N. At deg = N the relation is + // fully dense — the regime §11.1 pt 6 concedes to GEMM. + println!("═══ D-GTM-0i: RELATION density sweep (RANDOM, state 50%, depth 8) ═══"); + println!(" {:>8} {:>9} {:>12} {:>12} {:>9}", "deg", "rel dens%", "gemm ns/st", "mask ns/st", "speedup"); + for ° in &[1usize, 16, 64, 256, 1024, 4096] { + let matrix = build_matrix_deg(Rel::Random, 0xC0FFEE, deg); + let fwd = build_fwd_masks(&matrix); + let mut s = 0xBEEFu64; + let mut m0 = vec![0u64; W]; + for i in 0..N { + if splitmix(&mut s) % 100 < 50 { + m0[i / 64] |= 1u64 << (i % 64); + } + } + let (mut c1, mut c2) = (vec![0u64; W], vec![0u64; W]); + for w in 0..W { + c1[w] = splitmix(&mut s) | splitmix(&mut s); + c2[w] = splitmix(&mut s) | splitmix(&mut s); + } + let (c1f, c2f) = (dense_state(&c1), dense_state(&c2)); + let dense_m0 = dense_state(&m0); + + let mut gs = dense_m0.clone(); + let mut scratch = vec![0.0f32; N]; + let (mut r, t) = (0usize, Instant::now()); + while t.elapsed().as_secs_f64() < 0.05 { + gs.copy_from_slice(&dense_m0); + for _ in 0..8 { + step_gemm(&matrix, &mut gs, &mut scratch, &c1f); + for i in 0..N { + gs[i] *= c2f[i]; + } + } + r += 1; + } + let gemm_ns = t.elapsed().as_secs_f64() * 1e9 / (r * 8) as f64; + + let mut ms_ = m0.clone(); + let mut out = vec![0u64; W]; + let (mut mr, t) = (0usize, Instant::now()); + while t.elapsed().as_secs_f64() < 0.05 { + ms_.copy_from_slice(&m0); + for _ in 0..8 { + step_mask_random(&mut ms_, &fwd, &mut out, &c1, &c2); + } + mr += 1; + } + let mask_ns = t.elapsed().as_secs_f64() * 1e9 / (mr * 8) as f64; + + let gp = gs.iter().filter(|&&v| v > 0.0).count() as u32; + assert_eq!(gp, popcnt(&ms_), "ARMS DISAGREE at deg={deg}"); + println!( + " {:>8} {:>8.2}% {:>12.0} {:>12.0} {:>8.1}x", + deg, + 100.0 * deg as f64 / N as f64, + gemm_ns, + mask_ns, + gemm_ns / mask_ns.max(1e-9) + ); + } +} From 701314a87ee8dc246ce17e32b67ca71c07abdad8 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 5 Sep 2026 22:49:21 +0000 Subject: [PATCH 05/16] probe: fix clippy needless_range_loop; blackboard entry for the W0 measured results MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The mask arm's random-relation step indexed `state` by loop variable; switched to `state.iter().enumerate()`. Re-verified end to end after the change: correctness gate still holds at every cell, relation-density sweep reproduces (802x at 0.02% -> 310x at 100%, same flat shape). Blackboard records what §12 found: 0k passes (0 bytes/step on the mask hot path), 0j falsifies "GEMM wins when dense" for Boolean relations and replaces it with a type boundary, the dense-f32 baseline is mis-specified so the headline numbers are not evidence, and the correctness gate caught a real transpose bug in the probe itself. clippy -D warnings clean, fmt clean. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01EfrUJH3UNnv5NpDH4jDGHq --- .claude/blackboard.md | 39 ++++++++++++++++++++++++++++++ examples/hex_trie_vs_gemm_probe.rs | 4 +-- 2 files changed, 41 insertions(+), 2 deletions(-) diff --git a/.claude/blackboard.md b/.claude/blackboard.md index 86ee07c8..608180cb 100644 --- a/.claude/blackboard.md +++ b/.claude/blackboard.md @@ -3,6 +3,45 @@ > **Read this first.** The "Polyglot Notebook" architecture below is a > separate/older program, not the current epoch. +## 2026-09-05 (W0 MEASURED) — mask/trie vs GEMM: 0k passes, 0j falsifies the density framing + +Probe `examples/hex_trie_vs_gemm_probe.rs` (committed, --release). N=4096, mask +512 B, dense relation 64 MB, correctness gate on survivor counts at every cell. + +**D-GTM-0k ANSWERED, cleanly.** Mask hot path = **0 bytes/step** at every +density, every depth, both relation shapes — measured with a counting global +allocator, not asserted. GEMM = 73,728 B/step (packing buffer inside gemm_f32). +The invariant's own falsifier passes. + +**D-GTM-0j FALSIFIES §11.1 pt 6.** "GEMM is attractive when information is +dense" — measured, there is NO density crossover: masks win 745x at 0.02% +relation density and 297x at 100%. Both costs are flat in density (GEMM O(N²) +FMAs; mask O(active·N/64) word ORs). Honest correction, a TYPE boundary not a +density one: **masks win whenever the relation is Boolean; GEMM is required when +the relation carries VALUES.** A bitmask is 32x denser than f32 before any +algorithm runs, so a 0/1 relation in f32 was never the right representation. + +**The headline numbers are explicitly NOT evidence** (§12.5): the dense-f32 +baseline is mis-specified, and the missing arm is CSR SpMV (O(nnz) — at deg=1 +that is ~4096 FMAs, same order as the mask arm's 11.7 µs, so it would plausibly +cross). The one internally fair comparison is PREFIX vs RANDOM inside the mask +arm: 21-79 ns flat vs 305-35,522 ns scaling with active bits — structure worth +~3 orders of magnitude, degrading exactly where there is nothing to exploit. + +**The gate caught a bug in my own probe.** RANDOM failed immediately (912 vs +930): the GEMM arm computes `{i : srcs(i) ∩ active ≠ ∅}` while the mask arm was +unioning `srcs(i)` over active i — those agree only for a SYMMETRIC relation +(bucket membership is, random is not). The mask arm must union the TRANSPOSE. +A second flaw was caught by reading, not by any gate: mask timings at/below +timer resolution made every early "speedup" (25,940x…853,300x) a noise ratio; +both arms now run to a 50 ms floor and report ns/step. + +**0h graded [S]:** no perf in this sandbox; residency inferred from timing +(PREFIX 70 ns at depth 1 → 21-27 ns at depths 8/32, no knee to 32) — consistent +with resident, not proof. **0l is now the decisive probe** — with the density +axis dead, the hypothesis stands or falls on prefix-routing coverage vs codebook +entropy on the R2IL/C64 ore. New probe implied by 12.4: the weighted arm. + ## 2026-09-05 (v1.3) — the invariant strengthened: `substrate == mask geometry == projection surface` Operator: "make the 96-bit object holographic" was metaphor while the cube wanted diff --git a/examples/hex_trie_vs_gemm_probe.rs b/examples/hex_trie_vs_gemm_probe.rs index dfcfe28c..f5ac502c 100644 --- a/examples/hex_trie_vs_gemm_probe.rs +++ b/examples/hex_trie_vs_gemm_probe.rs @@ -164,8 +164,8 @@ fn step_mask_prefix(state: &mut [u64], c1: &[u64], c2: &[u64]) { /// active bit. This is the fallback the invariant calls the loss condition. fn step_mask_random(state: &mut [u64], fwd: &[u64], out: &mut [u64], c1: &[u64], c2: &[u64]) { out.fill(0); - for wi in 0..W { - let mut word = state[wi]; + for (wi, &w0) in state.iter().enumerate() { + let mut word = w0; while word != 0 { let b = word.trailing_zeros() as usize; word &= word - 1; From 77718bc8a287170d51860a7ffc29de1b2998a7e1 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 5 Sep 2026 23:02:13 +0000 Subject: [PATCH 06/16] =?UTF-8?q?probe:=20D-GTM-0l=20=E2=80=94=20packed-pr?= =?UTF-8?q?efix=20tract=20coverage=20measured=20on=20the=20R2IL=206502=20o?= =?UTF-8?q?re?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The tract framing's success condition is that the codebook is much smaller than the edge set it routes. Measured over 427 real long-range call edges: prefix locality is genuine signal (k>=2 at 15.69% vs 1.52% under a degree-preserving null), but the codebook compresses only 1.66x at 256-byte tracts and 1.08x at 16-byte tracts, and the non-local residual is diffuse (entropy 6.36 of 6.82 uniform) so no small exception table recovers it. The ore is read by path and never vendored — its provenance file forbids redistributing the corpus, so the repository keeps the measurement, not the instruction stream. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01EfrUJH3UNnv5NpDH4jDGHq --- .claude/blackboard.md | 21 +++ .../gemm-ternlog-mask-consolidation-v1.md | 70 ++++++++ examples/prefix_tract_coverage_probe.rs | 160 ++++++++++++++++++ 3 files changed, 251 insertions(+) create mode 100644 examples/prefix_tract_coverage_probe.rs diff --git a/.claude/blackboard.md b/.claude/blackboard.md index 608180cb..3f0078ad 100644 --- a/.claude/blackboard.md +++ b/.claude/blackboard.md @@ -1,3 +1,24 @@ +## 2026-09-05 — D-GTM-0l MEASURED (prefix-tract coverage, R2IL 6502 ore) + +The probe I flagged as decisive ran. `examples/prefix_tract_coverage_probe.rs` +against the Elite-rs R2IL harvest (427 long-range call edges, 2,647 packed addresses; +ore passed by path, never vendored — its provenance forbids redistribution). + +- Prefix locality is REAL: k>=2 shared prefix is 15.69% of edges vs 1.52% under a + degree-preserving null (50 shuffles). ~11x enrichment. +- The tract codebook does NOT compress: 7.62x at k=1 (4 KiB buckets, 16 of them on a + 64 KiB image — no resolution left), 1.66x at k=2, 1.08x at k=3, 1.00x at k=4. +- The non-local residual is DIFFUSE, not a hub set: 262 edges over 113 targets, + entropy 6.36 bits vs 6.82 uniform. A k=1 tract plus explicit far edges is 318 + entries against 427 edges. + +Verdict [G] on this ore: "white matter as an interpretation of packed location +prefixes" fails its own success condition — one tract per edge is an edge list. +Scope leg: these are PHYSICAL addresses from a 1986 linker, not semantic addresses +minted so the prefix carries meaning. The falsification is of the physical case only. +Next probe is the same instrument against an OGAR-minted classid space; the plan's +new §13 carries the full tables and the pass/fail condition for it. + # Current epoch (2026-05-26) — splat / palette / pillar / 3DGS > **Read this first.** The "Polyglot Notebook" architecture below is a diff --git a/.claude/plans/gemm-ternlog-mask-consolidation-v1.md b/.claude/plans/gemm-ternlog-mask-consolidation-v1.md index 49bc93b4..8d720ec9 100644 --- a/.claude/plans/gemm-ternlog-mask-consolidation-v1.md +++ b/.claude/plans/gemm-ternlog-mask-consolidation-v1.md @@ -802,3 +802,73 @@ counters. - **The weighted arm** — new, implied by 12.4: the type boundary needs its own measurement (where does accumulating a value force GEMM?). - 0c / 0d / 0e remain unrun. + +--- + +## §13 — D-GTM-0l MEASURED: packed-prefix routing does not cover real long-range edges + +Probe: `examples/prefix_tract_coverage_probe.rs` (committed). Ore: the R2IL 6502 +harvest in `AdaWorldAPI/Elite-rs` `.claude/harvest/r2il-6502/` — 34,186 lifted +instruction facts over two images, 427 `CallSite` long-range edges across 2,647 +distinct packed addresses. The ore is passed by path and never vendored; its own +provenance file forbids redistributing the corpus, and this repo stores the +measurement rather than the instruction stream. + +**The claim under test** (operator, grey/white-matter framing): *"white matter should +not be another data structure. It should be an interpretation of packed location +prefixes."* A tract is `(prefix, mask, learned transition)`; routing is +`ADDRESS & PREFIX_MASK == PREFIX`. That is a routing mechanism only if the tract +codebook is much smaller than the edge set it covers. + +### Result 1 — prefix locality is real signal, ~11× over the null + +| shared prefix | real | degree-preserving null (50 shuffles) | +|---|---|---| +| k=0 | 61.36% | 84.90% | +| k=1 | 22.95% | 13.54% | +| k=2 | 13.35% | 1.46% | +| k=3 | 2.34% | 0.06% | +| k=4 | 0.00% | 0.04% | + +k≥2 is 15.69% real against 1.52% null. The mechanism is not measuring noise. + +### Result 2 — but the codebook does not compress at any usable width + +| k | tract width | tracts | edges | compression | +|---|---|---|---|---| +| 1 | 4 KiB | 56 | 427 | 7.62× | +| 2 | 256 B | 257 | 427 | 1.66× | +| 3 | 16 B | 396 | 427 | 1.08× | +| 4 | 1 B | 427 | 427 | 1.00× | + +k=1 is the only width that compresses, and on a 64 KiB image it is 16 buckets — the +routing decision has almost no resolution left. By k=2 the codebook is already +1.66× and by k=3 it is an edge list with extra steps. + +### Result 3 — the non-local residual is DIFFUSE, so no exception table rescues it + +262 non-local edges over 113 distinct targets; top-10 targets cover 32.1%, top-40 +cover 64.9%; target entropy 6.36 bits against 6.82 uniform — only 0.46 bits below +uniform. There is no small hub set. A k=1 tract plus explicit far edges is 318 +entries against 427 edges (1.34×). + +### Verdict — [G] on this ore, with the scope leg named + +**On physically laid-out addresses, packed-prefix routing is a real but partial +signal that cannot carry the transition set alone.** The hypothesis's own success +condition ("not another data structure") fails: to resolve a destination you need +essentially one tract per edge. + +**Scope leg, load-bearing.** This ore's addresses are PHYSICAL — a 1986 linker's +layout, with no reason to be prefix-organized by meaning. The hypothesis was about +packed SEMANTIC addresses (`classid | HEEL | HIP | TWIG`, minted so that the prefix +IS the meaning). Those are different claims, and this measurement falsifies only the +first. It is the hardest case and arguably the wrong one — but it was the decisive +one *offered*, so the result stands as recorded rather than explained away. + +**What this makes the next probe.** Re-run the identical instrument against an +OGAR-minted address space, where prefixes are minted from the concept hierarchy +rather than from a linker. If enrichment rises and k=2/k=3 compression goes with it, +the mechanism survives on the substrate it was actually proposed for. If the +compression column looks like the table above, the tract framing is dead generally +and white matter needs to be a structure after all. diff --git a/examples/prefix_tract_coverage_probe.rs b/examples/prefix_tract_coverage_probe.rs new file mode 100644 index 00000000..bf215b3e --- /dev/null +++ b/examples/prefix_tract_coverage_probe.rs @@ -0,0 +1,160 @@ +//! D-GTM-0l — does packed-prefix routing express real long-range transitions? +//! +//! The hypothesis under test (operator, grey/white-matter framing): *"white matter +//! should not be another data structure. It should be an interpretation of packed +//! location prefixes."* Concretely: a tract is `(prefix, mask, learned transition)`, +//! and a transition is routed by `ADDRESS & PREFIX_MASK == PREFIX`. That is only a +//! routing mechanism if the tract codebook is much smaller than the edge set it +//! covers. If one tract is needed per edge, the "interpretation" IS an edge list. +//! +//! Falsifier: measure, on a real long-range relation set, (a) whether edges are more +//! prefix-local than a degree-preserving null, (b) how far the tract codebook +//! compresses the edge set at each prefix width, and (c) whether the non-local +//! residual is concentrated (a small exception table keeps the mechanism) or diffuse +//! (it does not). +//! +//! Input is passed by path and never vendored: the corpus this was measured on is a +//! disassembly harvest that its own provenance file forbids redistributing. The probe +//! reads derived facts only, and the repository stores the measurement, not the ore. +//! +//! Usage: `cargo run --release --example prefix_tract_coverage_probe -- ` +//! +//! Ore schema (tab-separated, `#`-prefixed header lines skipped): field 1 = image, +//! field 4 = the 128-bit packed address as 32 hex nibbles, field 6 = fact kind, +//! field 8 = the call target as a decimal address. `CallSite` rows are the edges. + +use std::collections::{BTreeMap, BTreeSet}; + +/// Address nibbles 9..12 of the packed key hold the 16-bit program counter, +/// little-endian byte order — `..361d..` is `0x1d36`. +fn unpack_pc(hex: &str) -> Option { + if hex.len() < 12 { + return None; + } + let lo = u32::from_str_radix(&hex[8..10], 16).ok()?; + let hi = u32::from_str_radix(&hex[10..12], 16).ok()?; + Some((hi << 8) | lo) +} + +/// The leading `k` nibbles of a 16-bit address — the `PREFIX_MASK` reading. +fn prefix(addr: u32, k: u32) -> u32 { + addr >> (4 * (4 - k)) +} + +/// Longest shared prefix, in nibbles, of two addresses. +fn shared(a: u32, b: u32) -> u32 { + (0..=4) + .rev() + .find(|&k| prefix(a, k) == prefix(b, k)) + .unwrap_or(0) +} + +/// SplitMix64 — deterministic shuffling so the null is reproducible. +struct SplitMix(u64); +impl SplitMix { + fn next(&mut self) -> u64 { + self.0 = self.0.wrapping_add(0x9E37_79B9_7F4A_7C15); + let mut z = self.0; + z = (z ^ (z >> 30)).wrapping_mul(0xBF58_476D_1CE4_E5B9); + z = (z ^ (z >> 27)).wrapping_mul(0x94D0_49BB_1331_11EB); + z ^ (z >> 31) + } +} + +fn main() { + let path = match std::env::args().nth(1) { + Some(p) => p, + None => { + eprintln!("usage: prefix_tract_coverage_probe "); + std::process::exit(2); + } + }; + let text = std::fs::read_to_string(&path).expect("read ore"); + + let mut edges: Vec<(String, u32, u32)> = Vec::new(); + for line in text.lines() { + if line.starts_with('#') { + continue; + } + let f: Vec<&str> = line.split('\t').collect(); + if f.len() < 9 || f[5] != "CallSite" { + continue; + } + let (Some(src), Ok(dst)) = (unpack_pc(f[3]), f[7].parse::()) else { + continue; + }; + edges.push((f[0].to_string(), src, dst)); + } + let n = edges.len(); + assert!(n > 0, "no CallSite edges parsed — schema drift?"); + println!("long-range edges: {n}"); + + // (a) locality vs a degree-preserving null: same sources, destinations reshuffled. + let mut real = [0usize; 5]; + for (_, s, d) in &edges { + real[shared(*s, *d) as usize] += 1; + } + let mut null = [0usize; 5]; + let mut dsts: Vec = edges.iter().map(|e| e.2).collect(); + let mut rng = SplitMix(0x9E37_79B9_7F4A_7C15); + const ROUNDS: usize = 50; + for _ in 0..ROUNDS { + for i in (1..dsts.len()).rev() { + dsts.swap(i, (rng.next() % (i as u64 + 1)) as usize); + } + for (e, d) in edges.iter().zip(&dsts) { + null[shared(e.1, *d) as usize] += 1; + } + } + println!("\nshared prefix | real | null (k nibbles; higher = more local)"); + for k in 0..5 { + println!( + " k={k} | {:6.2}% | {:6.2}%", + 100.0 * real[k] as f64 / n as f64, + 100.0 * null[k] as f64 / (ROUNDS * n) as f64 + ); + } + + // (b) does the tract codebook actually compress the edge set? + let uniq: BTreeSet<_> = edges.iter().cloned().collect(); + println!("\nk | tracts | edges | compression"); + for k in 1..=4 { + let tracts: BTreeSet<_> = edges + .iter() + .map(|(b, s, d)| (b.clone(), prefix(*s, k), prefix(*d, k))) + .collect(); + println!("{k} | {:6} | {:5} | {:8.2}x", tracts.len(), uniq.len(), uniq.len() as f64 / tracts.len() as f64); + } + + // (c) is the non-local residual a small exception table, or diffuse? + let far: Vec<_> = edges + .iter() + .filter(|(_, s, d)| shared(*s, *d) == 0) + .collect(); + let mut hist: BTreeMap = BTreeMap::new(); + for (_, _, d) in &far { + *hist.entry(*d).or_default() += 1; + } + let tot = far.len() as f64; + let mut counts: Vec = hist.values().copied().collect(); + counts.sort_unstable_by(|a, b| b.cmp(a)); + let entropy: f64 = counts + .iter() + .map(|&c| { + let p = c as f64 / tot; + -p * p.log2() + }) + .sum(); + println!("\nnon-local edges {} over {} distinct targets", far.len(), hist.len()); + let mut cum = 0usize; + for (i, c) in counts.iter().enumerate() { + cum += c; + if matches!(i + 1, 1 | 5 | 10 | 20 | 40) { + println!(" top {:3} targets cover {:5.1}%", i + 1, 100.0 * cum as f64 / tot); + } + } + println!( + " target entropy {entropy:.2} bits vs {:.2} uniform — a small hub set would sit far below", + (hist.len() as f64).log2() + ); +} From 741e34b819227d1b112221576e95ec010d9cd5c4 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 5 Sep 2026 23:45:46 +0000 Subject: [PATCH 07/16] =?UTF-8?q?probe:=20D-GTM-0n=20=E2=80=94=20ternlog?= =?UTF-8?q?=20amortization,=20residency=20knee,=20and=20the=20mask/sparse?= =?UTF-8?q?=20density=20crossover?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three measurements the activation-substrate question needs before any learner is designed. Depth: folding two constraints into one VPTERNLOGQ saves instructions, not bandwidth. T3/T1 sits near 0.44-0.70 while the working set is L1-resident and returns to ~1.0 by K=32, where the masks no longer fit and the cost becomes memory-bound. Per-constraint cost is flat in K rather than falling. Residency, derived from achieved mask traffic rather than read off timings: L1 peaks near 139 GB/s, L2 holds 90-93, L3 settles at 29-30. Density: a bitset pays for every bit, so the mask arm is flat at ~3.2 us per constraint over a 2^20-bit substrate at every density, while a survivor-walking sparse arm scales with what survives. They cross between 0.1% and 0.8% active. Sections A and B ran at 97% density, which is a mis-specified baseline for their sparse columns in the way a dense f32 matrix was for a Boolean relation in 0j; those columns are marked void and section D replaces them. Chained steps allocate zero bytes, measured with a counting allocator. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01EfrUJH3UNnv5NpDH4jDGHq --- examples/behavioral_soak_probe.rs | 423 +++++++++++++++++++++++++ examples/ternlog_amortization_probe.rs | 363 +++++++++++++++++++++ 2 files changed, 786 insertions(+) create mode 100644 examples/behavioral_soak_probe.rs create mode 100644 examples/ternlog_amortization_probe.rs diff --git a/examples/behavioral_soak_probe.rs b/examples/behavioral_soak_probe.rs new file mode 100644 index 00000000..439dfd05 --- /dev/null +++ b/examples/behavioral_soak_probe.rs @@ -0,0 +1,423 @@ +//! D-GTM-0m — successive soaking: how much of a NEW corpus is expressible in a +//! behavioural vocabulary already minted from earlier ones? +//! +//! This replaces the address-prefix framing that D-GTM-0l falsified. The claim +//! now under test is not that semantically related things get nearby addresses, +//! but that *machine behaviour converges onto a small reusable basis*: soak the +//! codebook on corpus 1, feed corpus 2 through it without resetting, and measure +//! reuse against new mint. If reuse rises while the codebook grows sublinearly, +//! the substrate is a sponge. If the codebook grows with the corpus, it is a +//! dictionary of one entry per thing seen, which is not a basis. +//! +//! ## The vacuity trap this probe is built around +//! +//! Reuse is trivially ~100% at a coarse enough granularity — there are only a +//! dozen distinct p-code opcodes, so "opcode reuse" saturates on the second +//! basic block of the first corpus and says nothing. So the probe reports a +//! LADDER of granularities and a null, and a result only counts if reuse stays +//! high at a granularity whose vocabulary is still growing. +//! +//! | rung | atom | what it claims | +//! |---|---|---| +//! | G0 | opcode | control — must saturate; proves nothing | +//! | G1 | (kind, opcode) | still near-trivial | +//! | G2 | block opcode sequence | the behavioural BPE token | +//! | G3 | block (opcode, in-arity, out-arity) sequence | shape-sensitive | +//! | G4 | function block-token sequence | whole-routine shape | +//! +//! The null shuffles opcodes across blocks while preserving every block's LENGTH +//! and the corpus-wide opcode marginal. If measured reuse at G2/G3 is no better +//! than the null's, the "behavioural basis" is just the opcode frequency +//! distribution reappearing, not structure. +//! +//! Usage: `cargo run --release --example behavioral_soak_probe -- ...` +//! Corpora are soaked in argument order and the codebook is never reset. +//! Ore is read by path and never vendored. + +use std::collections::{BTreeMap, HashMap, HashSet}; + +/// One lifted instruction fact, reduced to what the atoms are built from. +struct Fact { + block: (String, String, String), + id: u64, + kind: String, + opcode: String, + ins: u32, + outs: u32, +} + +/// SplitMix64 — the null's shuffle must be reproducible. +struct SplitMix(u64); +impl SplitMix { + fn next(&mut self) -> u64 { + self.0 = self.0.wrapping_add(0x9E37_79B9_7F4A_7C15); + let mut z = self.0; + z = (z ^ (z >> 30)).wrapping_mul(0xBF58_476D_1CE4_E5B9); + z = (z ^ (z >> 27)).wrapping_mul(0x94D0_49BB_1331_11EB); + z ^ (z >> 31) + } +} + +fn parse(path: &str) -> Vec { + let text = std::fs::read_to_string(path).expect("read ore"); + let mut out = Vec::new(); + for line in text.lines() { + if line.starts_with('#') { + continue; + } + let f: Vec<&str> = line.split('\t').collect(); + if f.len() < 11 { + continue; + } + let Ok(id) = f[2].parse::() else { continue }; + out.push(Fact { + block: (f[0].to_string(), f[1].to_string(), f[10].to_string()), + id, + kind: f[5].to_string(), + opcode: f[6].to_string(), + ins: u32::from(f[5] == "OperandIn"), + outs: u32::from(f[5] == "OperandOut"), + }); + } + out +} + +/// Group facts into blocks, each block a fact list ordered by fact id. +fn blocks(facts: &[Fact]) -> Vec> { + let mut m: BTreeMap<&(String, String, String), Vec<&Fact>> = BTreeMap::new(); + for f in facts { + m.entry(&f.block).or_default().push(f); + } + let mut v: Vec> = m.into_values().collect(); + for b in &mut v { + b.sort_by_key(|f| f.id); + } + v +} + +/// The five rungs of the granularity ladder, as atom occurrence lists. +fn atoms(bs: &[Vec<&Fact>], rung: usize) -> Vec { + match rung { + 0 => bs.iter().flatten().map(|f| f.opcode.clone()).collect(), + 1 => bs + .iter() + .flatten() + .map(|f| format!("{}|{}", f.kind, f.opcode)) + .collect(), + 2 => bs + .iter() + .map(|b| { + b.iter() + .filter(|f| f.kind == "Op") + .map(|f| f.opcode.as_str()) + .collect::>() + .join(",") + }) + .filter(|s| !s.is_empty()) + .collect(), + 3 => bs + .iter() + .map(|b| { + // per-instruction shape: opcode plus how many values it consumed/produced + let mut per: BTreeMap<&str, (u32, u32)> = BTreeMap::new(); + for f in b { + let e = per.entry(f.opcode.as_str()).or_default(); + e.0 += f.ins; + e.1 += f.outs; + } + per.iter() + .map(|(o, (i, u))| format!("{o}:{i}:{u}")) + .collect::>() + .join(",") + }) + .filter(|s| !s.is_empty()) + .collect(), + _ => { + // whole-function shape: the sequence of its blocks' G2 tokens + let mut by_fn: BTreeMap<(&str, &str), Vec> = BTreeMap::new(); + for b in bs { + let tok = b + .iter() + .filter(|f| f.kind == "Op") + .map(|f| f.opcode.as_str()) + .collect::>() + .join(","); + if tok.is_empty() { + continue; + } + by_fn + .entry((b[0].block.0.as_str(), b[0].block.1.as_str())) + .or_default() + .push(tok); + } + by_fn.into_values().map(|v| v.join(";")).collect() + } + } +} + +/// Soak `corpora` in order through one never-reset codebook; report the curve. +fn soak(names: &[String], per_corpus: &[Vec], label: &str) { + println!("\n{label}"); + println!(" corpus | occurrences | reuse% | new atoms | codebook | bits/occ"); + let mut book: HashMap = HashMap::new(); + for (name, occ) in names.iter().zip(per_corpus) { + let before = book.len(); + let mut hits = 0usize; + for a in occ { + if book.contains_key(a) { + hits += 1; + } else { + let n = book.len(); + book.insert(a.clone(), n); + } + } + let minted = book.len() - before; + // cost of transmitting this corpus given the standing codebook: an index + // per occurrence, plus a literal description for each newly minted atom. + let idx_bits = if book.len() > 1 { + (book.len() as f64).log2() + } else { + 0.0 + }; + let new_bits: f64 = occ + .iter() + .collect::>() + .iter() + .filter(|a| book[**a] >= before) + .map(|a| a.len() as f64 * 8.0) + .sum(); + let total = occ.len() as f64 * idx_bits + new_bits; + println!( + " {:<25} | {:11} | {:5.1}% | {:9} | {:8} | {:8.2}", + name, + occ.len(), + 100.0 * hits as f64 / occ.len().max(1) as f64, + minted, + book.len(), + total / occ.len().max(1) as f64 + ); + } +} + +/// Reuse stratified by atom LENGTH — the control that decides whether transfer +/// is real. A one-op block is trivially shared by every architecture ever, so a +/// headline reuse figure dominated by short blocks says nothing. This reports, +/// for the corpora soaked AFTER the first, what fraction of blocks of each +/// length were already in the codebook, and how many blocks that is. +fn stratified(names: &[String], per_corpus: &[Vec], first_n: usize) { + println!("\nG2 reuse stratified by block length (transfer corpora only)"); + let mut book: HashSet = HashSet::new(); + for (i, occ) in per_corpus.iter().enumerate() { + if i >= first_n { + let mut hit: BTreeMap = BTreeMap::new(); + for a in occ { + let len = a.split(',').count(); + let bucket = if len >= 8 { 8 } else { len }; + let e = hit.entry(bucket).or_default(); + e.1 += 1; + if book.contains(a) { + e.0 += 1; + } + } + println!(" {}", names[i]); + for (len, (h, t)) in &hit { + let label = if *len == 8 { "8+".to_string() } else { len.to_string() }; + println!( + " len {:>2} ops | {:5} blocks | {:5.1}% already known", + label, + t, + 100.0 * *h as f64 / *t as f64 + ); + } + } + for a in occ { + book.insert(a.clone()); + } + } +} + +/// The composition test — the one exact-match cannot answer. +/// +/// A long block failing to match a soaked block EXACTLY does not mean it has no +/// reusable structure; it means one-token-per-block is the wrong atom for it. +/// That is what byte-pair encoding exists to fix: cover a long sequence by +/// concatenating short known tokens. So build the codebook from every contiguous +/// opcode n-gram (n = 1..=MAX_TOKEN) seen in the SOAK corpora, then greedily +/// longest-match-tokenize each TRANSFER block against it. +/// +/// The metric cannot be coverage: length-1 tokens make coverage trivially 100%. +/// What matters is (a) ops per token — how much of the sequence a single known +/// token accounts for — and (b) the share of ops covered by tokens of length ≥ 2, +/// which is the share that is genuinely structural rather than per-opcode. +fn composition(names: &[String], per_block_ops: &[Vec>], first_n: usize) { + const MAX_TOKEN: usize = 6; + let mut book: HashSet> = HashSet::new(); + for corpus in per_block_ops.iter().take(first_n) { + for b in corpus { + for n in 1..=MAX_TOKEN.min(b.len()) { + for w in b.windows(n) { + book.insert(w.to_vec()); + } + } + } + } + // Saturation control: with a ~12-symbol opcode alphabet a codebook can cover + // every short n-gram by combinatorics alone, at which case a high ops/token + // is arithmetic, not transfer. Report what fraction of the possible n-grams + // over the observed alphabet the codebook actually holds. + let alphabet: HashSet<&String> = per_block_ops + .iter() + .take(first_n) + .flatten() + .flatten() + .collect(); + let a = alphabet.len() as f64; + let mut have = vec![0usize; MAX_TOKEN + 1]; + for t in &book { + have[t.len()] += 1; + } + println!("\nBPE-style composition against a {}-token soaked codebook (n-grams n<={MAX_TOKEN})", book.len()); + print!(" alphabet {} opcodes; codebook holds", alphabet.len()); + for n in 1..=MAX_TOKEN { + print!(" n{n}:{}/{:.0}", have[n], a.powi(n as i32)); + } + println!(); + println!( + " corpus | blocks | ops | tokens | ops/token | ops in len>=2 tokens | unseen opcodes" + ); + for (i, corpus) in per_block_ops.iter().enumerate() { + if i < first_n { + continue; + } + let (mut ops, mut toks, mut structural, mut unseen) = (0usize, 0usize, 0usize, 0usize); + for b in corpus { + let mut p = 0usize; + while p < b.len() { + let mut best = 0usize; + for n in (1..=MAX_TOKEN.min(b.len() - p)).rev() { + if book.contains(&b[p..p + n]) { + best = n; + break; + } + } + if best == 0 { + // a genuinely novel opcode: no token of any length covers it + unseen += 1; + best = 1; + } else if best >= 2 { + structural += best; + } + toks += 1; + p += best; + } + ops += b.len(); + } + println!( + " {:<25} | {:6} | {:5} | {:6} | {:9.2} | {:19.1}% | {:14}", + names[i].rsplit('/').next().unwrap_or(&names[i]), + corpus.len(), + ops, + toks, + ops as f64 / toks.max(1) as f64, + 100.0 * structural as f64 / ops.max(1) as f64, + unseen + ); + } +} + +fn main() { + let paths: Vec = std::env::args().skip(1).collect(); + if paths.is_empty() { + eprintln!("usage: behavioral_soak_probe ..."); + std::process::exit(2); + } + + let mut names = Vec::new(); + let mut all: Vec> = Vec::new(); + for p in &paths { + let facts = parse(p); + // one ore file may hold several images; soak each image separately so a + // second image inside one file still counts as a second corpus. + let mut by_img: BTreeMap> = BTreeMap::new(); + for f in facts { + by_img.entry(f.block.0.clone()).or_default().push(f); + } + for (img, fs) in by_img { + names.push(img); + all.push(fs); + } + } + println!("corpora soaked in order: {}", names.join(" -> ")); + for (n, f) in names.iter().zip(&all) { + let b = blocks(f); + println!(" {n}: {} facts, {} blocks", f.len(), b.len()); + } + + for rung in 0..5 { + let label = [ + "G0 opcode (control)", + "G1 kind|opcode", + "G2 block opcode sequence", + "G3 block opcode:arity shape", + "G4 function block-token sequence", + ][rung]; + let per: Vec> = all.iter().map(|f| atoms(&blocks(f), rung)).collect(); + soak(&names, &per, label); + } + + // The length control: transfer restricted to blocks long enough to mean + // something. `first_n` is the count of corpora treated as the soak. + let g2: Vec> = all.iter().map(|f| atoms(&blocks(f), 2)).collect(); + let first_n: usize = std::env::var("SOAK_FIRST_N") + .ok() + .and_then(|v| v.parse().ok()) + .unwrap_or(1); + stratified(&names, &g2, first_n); + + let per_block_ops: Vec>> = all + .iter() + .map(|f| { + blocks(f) + .iter() + .map(|b| { + b.iter() + .filter(|x| x.kind == "Op") + .map(|x| x.opcode.clone()) + .collect::>() + }) + .filter(|b: &Vec| !b.is_empty()) + .collect() + }) + .collect(); + composition(&names, &per_block_ops, first_n); + + // Null: preserve every block's length and the corpus-wide opcode marginal, + // destroy which opcodes co-occur in a block. Reported at G2, the rung whose + // whole claim is that co-occurrence is structured. + let mut rng = SplitMix(0x9E37_79B9_7F4A_7C15); + let mut null_per: Vec> = Vec::new(); + for f in &all { + let bs = blocks(f); + let mut pool: Vec = bs + .iter() + .flatten() + .filter(|x| x.kind == "Op") + .map(|x| x.opcode.clone()) + .collect(); + for i in (1..pool.len()).rev() { + pool.swap(i, (rng.next() % (i as u64 + 1)) as usize); + } + let mut cur = 0usize; + let mut toks = Vec::new(); + for b in &bs { + let len = b.iter().filter(|x| x.kind == "Op").count(); + if len == 0 { + continue; + } + toks.push(pool[cur..cur + len].join(",")); + cur += len; + } + null_per.push(toks); + } + soak(&names, &null_per, "G2 NULL — block lengths and opcode marginal kept, co-occurrence destroyed"); +} diff --git a/examples/ternlog_amortization_probe.rs b/examples/ternlog_amortization_probe.rs new file mode 100644 index 00000000..7954dbe5 --- /dev/null +++ b/examples/ternlog_amortization_probe.rs @@ -0,0 +1,363 @@ +//! D-GTM-0n — is a *further* learned constraint cheap once the representation +//! is resident, and where does residency actually break? +//! +//! The architectural claim under test is NOT that ternary Boolean logic is +//! intelligent. It is narrower and mechanical: if learned state and active +//! state are the same shape, then stacking one more constraint should cost +//! roughly *one more resident load plus one instruction*, rather than another +//! pass of graph materialization. That is a claim about an amortization curve, +//! so this probe measures the curve rather than a headline ratio. +//! +//! Four arms compute the IDENTICAL logical result — `A ∧ M₁ ∧ … ∧ M_K` — and a +//! correctness gate aborts the run unless every arm agrees, both on the +//! survivor count and on the surviving set itself. +//! +//! | arm | representation | work per constraint | +//! |---|---|---| +//! | T0 | materialized `Vec` id list | rebuild the candidate list | +//! | T1 | bitset, `mask_and_assign` | one pass over the mask | +//! | T3 | bitset, `mask_ternlog_assign::` | one pass per TWO constraints | +//! | T5 | sorted index intersection | merge over the sparse sets | +//! +//! T3 is the arm the claim is about: `VPTERNLOGQ` folds two constraints into +//! one instruction, so if the cost model is "a pass over resident bytes", T3 +//! should sit near half of T1 and the per-constraint cost should be FLAT in K. +//! Flat is the honest success criterion — a per-constraint cost that *falls* +//! with K would mean something else is happening (and would need explaining, +//! not celebrating). +//! +//! Residency is derived, not asserted: the probe reports achieved mask-traffic +//! bandwidth, and the knee where GB/s collapses is where the working set left a +//! cache level. Reading residency off "pretty nanosecond numbers" is exactly +//! what D-GTM-0h was graded [S] for, so the bandwidth column is the evidence +//! and the ns column is not. +//! +//! Usage: `cargo run --release --example ternlog_amortization_probe` + +use std::alloc::{GlobalAlloc, Layout, System}; +use std::sync::atomic::{AtomicUsize, Ordering}; +use std::time::Instant; + +use ndarray::simd::ternlog::AND3; +use ndarray::simd::{mask_and_assign, mask_ternlog_assign}; + +/// Counting allocator — "materialized 0 bytes" is a measurement here, not a claim. +struct Counting; +static ALLOCATED: AtomicUsize = AtomicUsize::new(0); +unsafe impl GlobalAlloc for Counting { + unsafe fn alloc(&self, l: Layout) -> *mut u8 { + ALLOCATED.fetch_add(l.size(), Ordering::Relaxed); + unsafe { System.alloc(l) } + } + unsafe fn dealloc(&self, p: *mut u8, l: Layout) { + unsafe { System.dealloc(p, l) } + } +} +#[global_allocator] +static A: Counting = Counting; + +struct SplitMix(u64); +impl SplitMix { + fn next(&mut self) -> u64 { + self.0 = self.0.wrapping_add(0x9E37_79B9_7F4A_7C15); + let mut z = self.0; + z = (z ^ (z >> 30)).wrapping_mul(0xBF58_476D_1CE4_E5B9); + z = (z ^ (z >> 27)).wrapping_mul(0x94D0_49BB_1331_11EB); + z ^ (z >> 31) + } +} + +/// Bits set in a mask, as the arms' shared correctness currency. +fn popcnt(m: &[u64]) -> u32 { + m.iter().map(|w| w.count_ones()).sum() +} + +fn to_ids(m: &[u64]) -> Vec { + let mut v = Vec::new(); + for (w, word) in m.iter().enumerate() { + let mut b = *word; + while b != 0 { + v.push((w * 64 + b.trailing_zeros() as usize) as u32); + b &= b - 1; + } + } + v +} + +/// Run one (N, K) cell across all four arms. Returns per-arm ns/constraint. +fn cell(n_bits: usize, k: usize, seed: u64) -> [f64; 4] { + let words = n_bits / 64; + let mut rng = SplitMix(seed); + + // Constraint masks are dense enough that survivors persist to K=64: each + // keeps ~97% of bits, so the intersection stays non-empty and every arm + // does real work at every depth. A sparser mask would empty the set after + // a few constraints and measure nothing but early exit. + let masks: Vec> = (0..k) + .map(|_| (0..words).map(|_| !(1u64 << (rng.next() % 64))).collect()) + .collect(); + let base: Vec = (0..words).map(|_| rng.next() | rng.next()).collect(); + + // ---- correctness gate, before any timing ---- + let mut t1 = base.clone(); + for m in &masks { + mask_and_assign(&mut t1, m); + } + let mut t3 = base.clone(); + let mut i = 0; + while i + 1 < k { + mask_ternlog_assign::(&mut t3, &masks[i], &masks[i + 1]); + i += 2; + } + if i < k { + mask_and_assign(&mut t3, &masks[i]); + } + let gold = to_ids(&t1); + assert_eq!(gold, to_ids(&t3), "T3 disagrees with T1 at N={n_bits} K={k}"); + + // sparse arms share the same logical sets, expressed as sorted id lists + let sparse: Vec> = masks.iter().map(|m| to_ids(m)).collect(); + let base_ids = to_ids(&base); + let mut t5 = base_ids.clone(); + for s in &sparse { + let mut out = Vec::with_capacity(t5.len()); + let (mut a, mut b) = (0usize, 0usize); + while a < t5.len() && b < s.len() { + match t5[a].cmp(&s[b]) { + std::cmp::Ordering::Equal => { + out.push(t5[a]); + a += 1; + b += 1; + } + std::cmp::Ordering::Less => a += 1, + std::cmp::Ordering::Greater => b += 1, + } + } + t5 = out; + } + assert_eq!(gold, t5, "T5 disagrees with T1 at N={n_bits} K={k}"); + + // ---- timing: each arm runs to a 60 ms floor, well past timer resolution ---- + const FLOOR: f64 = 0.060; + let mut out = [0.0f64; 4]; + + // T1 — one pass per constraint + let mut scratch = base.clone(); + let (mut iters, t0) = (0u64, Instant::now()); + while t0.elapsed().as_secs_f64() < FLOOR { + scratch.copy_from_slice(&base); + for m in &masks { + mask_and_assign(&mut scratch, m); + } + iters += 1; + } + out[1] = t0.elapsed().as_secs_f64() * 1e9 / (iters as f64 * k as f64); + + // T3 — one pass per TWO constraints + let (mut iters3, t3c) = (0u64, Instant::now()); + while t3c.elapsed().as_secs_f64() < FLOOR { + scratch.copy_from_slice(&base); + let mut i = 0; + while i + 1 < k { + mask_ternlog_assign::(&mut scratch, &masks[i], &masks[i + 1]); + i += 2; + } + if i < k { + mask_and_assign(&mut scratch, &masks[i]); + } + iters3 += 1; + } + out[2] = t3c.elapsed().as_secs_f64() * 1e9 / (iters3 as f64 * k as f64); + + // T5 — sorted-index intersection (the honest sparse arm) + let (mut iters5, t5c) = (0u64, Instant::now()); + while t5c.elapsed().as_secs_f64() < FLOOR { + let mut cur = base_ids.clone(); + for s in &sparse { + let mut o = Vec::with_capacity(cur.len()); + let (mut a, mut b) = (0usize, 0usize); + while a < cur.len() && b < s.len() { + match cur[a].cmp(&s[b]) { + std::cmp::Ordering::Equal => { + o.push(cur[a]); + a += 1; + b += 1; + } + std::cmp::Ordering::Less => a += 1, + std::cmp::Ordering::Greater => b += 1, + } + } + cur = o; + } + std::hint::black_box(&cur); + iters5 += 1; + } + out[3] = t5c.elapsed().as_secs_f64() * 1e9 / (iters5 as f64 * k as f64); + + // T0 — materialize the surviving id list after every constraint + let (mut iters0, t0c) = (0u64, Instant::now()); + while t0c.elapsed().as_secs_f64() < FLOOR { + let mut cur = base.clone(); + for m in &masks { + mask_and_assign(&mut cur, m); + std::hint::black_box(to_ids(&cur)); + } + iters0 += 1; + } + out[0] = t0c.elapsed().as_secs_f64() * 1e9 / (iters0 as f64 * k as f64); + + out +} + +fn main() { + println!("D-GTM-0n — ternlog amortization + residency boundary"); + println!("host L1d 48 KiB/core, L2 2 MiB/core, L3 260 MiB shared\n"); + + // Depth sweep at a fixed L1-resident mask, isolating K from working-set size. + let n = 1 << 15; // 32768 bits = 4 KiB per mask + println!("A. DEPTH SWEEP — mask 4 KiB, working set = (K+1)x4 KiB"); + println!(" K | working set | T0 mat'd | T1 and | T3 ternlog | T5 sparse | T3/T1 | T3 GB/s"); + for k in [1usize, 2, 4, 8, 16, 32, 64] { + let r = cell(n, k, 0x1234 + k as u64); + let ws = (k + 1) * n / 8; + // each constraint moves one mask in and the accumulator through + let gbs = (n as f64 / 8.0 * 2.0) / r[2]; + println!( + " {k:2} | {:8} B | {:8.1} | {:6.1} | {:10.1} | {:9.1} | {:5.2} | {:7.1}", + ws, + r[0], + r[1], + r[2], + r[3], + r[2] / r[1], + gbs + ); + } + + // Working-set sweep at fixed depth: the knee locates the cache boundary. + println!("\nB. RESIDENCY SWEEP — K=8 constraints, mask size grows"); + println!(" mask | working set | T1 and | T3 ternlog | T3/T1 | T3 GB/s | level"); + for lg in [12usize, 14, 16, 18, 20, 22, 24, 26] { + let nb = 1usize << lg; + let mask_b = nb / 8; + let ws = 9 * mask_b; + let r = cell(nb, 8, 0x99 + lg as u64); + let gbs = (mask_b as f64 * 2.0) / r[2]; + let level = if ws <= 48 * 1024 { + "L1" + } else if ws <= 2 * 1024 * 1024 { + "L2" + } else if ws <= 260 * 1024 * 1024 { + "L3" + } else { + "DRAM" + }; + println!( + " {:6} B | {:9} B | {:6.1} | {:10.1} | {:5.2} | {:7.1} | {level}", + mask_b, + ws, + r[1], + r[2], + r[2] / r[1], + gbs + ); + } + + // D. DENSITY SWEEP — the arm that decides whether masks are the right + // currency at all. A focus field over a huge substrate is SPARSE, and a + // bitset pays for every bit whether set or not, while an id list pays only + // for survivors. Sections A and B ran at ~97% density, which is a + // mis-specified baseline for the sparse arms in exactly the way a dense f32 + // matrix was a mis-specified baseline for a Boolean relation in D-GTM-0j — + // so their T0/T5 columns are void as evidence and this sweep replaces them. + println!("\nD. DENSITY SWEEP — N=2^20 bits (128 KiB mask), K=8 constraints"); + println!(" active | survivors | T1 and | T3 ternlog | T5 sparse | winner"); + let nb = 1usize << 20; + let words = nb / 64; + for (label, keep_shift) in [("100%", 0u32), ("50%", 1), ("6%", 4), ("0.8%", 7), ("0.1%", 10), ("0.012%", 13)] { + let mut rng = SplitMix(0xDEAD + keep_shift as u64); + // base: keep roughly 1 bit in 2^keep_shift + let base: Vec = (0..words) + .map(|_| { + let mut w = u64::MAX; + for _ in 0..keep_shift { + w &= rng.next(); + } + w + }) + .collect(); + // constraints stay near-total so survivors track the base density + let masks: Vec> = (0..8) + .map(|_| (0..words).map(|_| !(1u64 << (rng.next() % 64))).collect()) + .collect(); + + let mut t1v = base.clone(); + for m in &masks { + mask_and_assign(&mut t1v, m); + } + let survivors = popcnt(&t1v); + let sparse: Vec> = masks.iter().map(|m| to_ids(m)).collect(); + let base_ids = to_ids(&base); + + const FLOOR: f64 = 0.060; + let mut scratch = base.clone(); + let (mut i1, c1) = (0u64, Instant::now()); + while c1.elapsed().as_secs_f64() < FLOOR { + scratch.copy_from_slice(&base); + for m in &masks { + mask_and_assign(&mut scratch, m); + } + i1 += 1; + } + let ns1 = c1.elapsed().as_secs_f64() * 1e9 / (i1 as f64 * 8.0); + + let (mut i3, c3) = (0u64, Instant::now()); + while c3.elapsed().as_secs_f64() < FLOOR { + scratch.copy_from_slice(&base); + let mut i = 0; + while i + 1 < 8 { + mask_ternlog_assign::(&mut scratch, &masks[i], &masks[i + 1]); + i += 2; + } + i3 += 1; + } + let ns3 = c3.elapsed().as_secs_f64() * 1e9 / (i3 as f64 * 8.0); + + // The sparse arm gets its BEST case: it walks only survivors, and the + // constraint sets are checked by direct bit test rather than by merging + // two long lists — a list merge against a near-total set is the sparse + // representation's worst case and would not be how anyone builds this. + let (mut i5, c5) = (0u64, Instant::now()); + while c5.elapsed().as_secs_f64() < FLOOR { + let mut cur = base_ids.clone(); + for m in &masks { + cur.retain(|&id| m[id as usize / 64] >> (id % 64) & 1 == 1); + } + std::hint::black_box(&cur); + i5 += 1; + } + let ns5 = c5.elapsed().as_secs_f64() * 1e9 / (i5 as f64 * 8.0); + let _ = &sparse; + + let winner = if ns5 < ns3 { "sparse" } else { "mask" }; + println!(" {:8} | {:9} | {:6.0} | {:10.0} | {:9.0} | {winner}", label, survivors, ns1, ns3, ns5); + } + + // Materialization: the claim is that the Boolean arms never build a set. + let before = ALLOCATED.load(Ordering::Relaxed); + let words = (1usize << 16) / 64; + let mut acc: Vec = vec![u64::MAX; words]; + let m1: Vec = vec![0xF0F0_F0F0_F0F0_F0F0; words]; + let m2: Vec = vec![0xFF00_FF00_FF00_FF00; words]; + let setup = ALLOCATED.load(Ordering::Relaxed); + for _ in 0..1000 { + mask_ternlog_assign::(&mut acc, &m1, &m2); + } + let after = ALLOCATED.load(Ordering::Relaxed); + println!( + "\nC. MATERIALIZATION — setup {} B, then {} B over 1000 chained steps (popcount {})", + setup - before, + after - setup, + popcnt(&acc) + ); +} From f1f40236ec0d3139384b340e18aedb9d0ed40157 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 5 Sep 2026 23:48:15 +0000 Subject: [PATCH 08/16] =?UTF-8?q?probe:=20D-GTM-0m=20=E2=80=94=20behaviour?= =?UTF-8?q?al=20soak=20across=20ISAs,=20and=20the=20null=20that=20deflates?= =?UTF-8?q?=20its=20headline?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Soaks one never-reset codebook across corpora in order and measures reuse against new mint, over a granularity ladder chosen so the trivial rung is visible: opcode and kind|opcode saturate at ~100% and prove nothing, which is the control. Cross-ISA, a codebook minted from 1986 6502 game code covers 50.4% and 56.8% of two x86-64 C binaries' basic blocks by exact match, against a marginal-preserving null of 17.1% and 37.5%. Stratifying by block length shows where that lives: blocks of 1-4 ops transfer at 56-100%, blocks of 8+ ops at 0.0% and 1.9%. Exact sequence identity is the wrong atom for a long block. Composing long blocks from short known tokens is the BPE question that answers, and it is where the headline dies. Real soak covers 98.3% and 97.1% of x86 ops with tokens of length two or more; a shuffled soak that keeps every block length and the opcode marginal exactly reaches 97.6% and 95.6%. Coverage is therefore not evidence. What survives is the state budget: the shuffled codebook needs 4526 tokens where the real one needs 681, so the structure shows up as a 6.6x smaller codebook at matched coverage, not as coverage. The alphabet control explains why coverage saturates: seven opcodes, and the real codebook holds 44.9% of possible bigrams and 15.2% of trigrams. This workspace has caught the same shape before, where held-out coverage looked tautological over a seven-symbol alphabet. Ore is read by path and never vendored. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01EfrUJH3UNnv5NpDH4jDGHq --- examples/behavioral_soak_probe.rs | 52 ++++++++++++++++++++++++++----- 1 file changed, 45 insertions(+), 7 deletions(-) diff --git a/examples/behavioral_soak_probe.rs b/examples/behavioral_soak_probe.rs index 439dfd05..a463c3c6 100644 --- a/examples/behavioral_soak_probe.rs +++ b/examples/behavioral_soak_probe.rs @@ -249,7 +249,7 @@ fn stratified(names: &[String], per_corpus: &[Vec], first_n: usize) { /// What matters is (a) ops per token — how much of the sequence a single known /// token accounts for — and (b) the share of ops covered by tokens of length ≥ 2, /// which is the share that is genuinely structural rather than per-opcode. -fn composition(names: &[String], per_block_ops: &[Vec>], first_n: usize) { +fn composition(label: &str, names: &[String], per_block_ops: &[Vec>], first_n: usize) { const MAX_TOKEN: usize = 6; let mut book: HashSet> = HashSet::new(); for corpus in per_block_ops.iter().take(first_n) { @@ -271,17 +271,33 @@ fn composition(names: &[String], per_block_ops: &[Vec>], first_n: us .flatten() .flatten() .collect(); - let a = alphabet.len() as f64; + let a = alphabet.len(); let mut have = vec![0usize; MAX_TOKEN + 1]; for t in &book { have[t.len()] += 1; } - println!("\nBPE-style composition against a {}-token soaked codebook (n-grams n<={MAX_TOKEN})", book.len()); - print!(" alphabet {} opcodes; codebook holds", alphabet.len()); + // Alphabet-saturation control. With a small opcode alphabet, an n-gram + // codebook can cover almost any sequence by combinatorics alone, which is + // the boring explanation this workspace has already caught once: a 7-symbol + // alphabet made held-out coverage look tautological, and made a degree-6 + // neighbourhood the complete graph minus self. So report what fraction of + // the POSSIBLE short n-grams the codebook holds; if that is near 1, high + // composition coverage is arithmetic rather than structure. + let alphabet: HashSet<&String> = per_block_ops + .iter() + .take(first_n) + .flatten() + .flatten() + .collect(); + let a = alphabet.len(); + println!("\n{label} — {}-token codebook (n-grams n<={MAX_TOKEN})", book.len()); + print!(" alphabet {a} symbols; codebook holds"); for n in 1..=MAX_TOKEN { - print!(" n{n}:{}/{:.0}", have[n], a.powi(n as i32)); + let possible = (a as f64).powi(n as i32); + let held = book.iter().filter(|t| t.len() == n).count(); + print!(" {n}-gram {:.1}%", 100.0 * held as f64 / possible); } - println!(); + println!(" <- near 100% would make coverage arithmetic, not structure"); println!( " corpus | blocks | ops | tokens | ops/token | ops in len>=2 tokens | unseen opcodes" ); @@ -389,7 +405,29 @@ fn main() { .collect() }) .collect(); - composition(&names, &per_block_ops, first_n); + composition("BPE-style composition, REAL soak", &names, &per_block_ops, first_n); + + // The null that decides whether composition coverage is structure. Shuffle + // the SOAK corpora's opcodes so every block keeps its length and the corpus + // keeps its opcode marginal exactly, but co-occurrence order is destroyed; + // mint the codebook from that and tokenize the REAL transfer corpora. If + // coverage and ops/token survive, the result was the opcode distribution + // reappearing, not a behavioural basis. + let mut nrng = SplitMix(0x5EED_1234_5678_9ABC); + let mut shuffled = per_block_ops.clone(); + for corpus in shuffled.iter_mut().take(first_n) { + let mut pool: Vec = corpus.iter().flatten().cloned().collect(); + for i in (1..pool.len()).rev() { + pool.swap(i, (nrng.next() % (i as u64 + 1)) as usize); + } + let mut cur = 0usize; + for b in corpus.iter_mut() { + let n = b.len(); + b.clone_from_slice(&pool[cur..cur + n]); + cur += n; + } + } + composition("COMPOSITION NULL, shuffled soak", &names, &shuffled, first_n); // Null: preserve every block's length and the corpus-wide opcode marginal, // destroy which opcodes co-occur in a block. Reported at G2, the rung whose From ab3359f41a9b19ad99f04e1bab4e992e8ffb24e0 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 5 Sep 2026 23:48:36 +0000 Subject: [PATCH 09/16] probe: drop two dead locals left by deduplicating the alphabet control Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01EfrUJH3UNnv5NpDH4jDGHq --- examples/behavioral_soak_probe.rs | 4 ---- 1 file changed, 4 deletions(-) diff --git a/examples/behavioral_soak_probe.rs b/examples/behavioral_soak_probe.rs index a463c3c6..39312de2 100644 --- a/examples/behavioral_soak_probe.rs +++ b/examples/behavioral_soak_probe.rs @@ -272,10 +272,6 @@ fn composition(label: &str, names: &[String], per_block_ops: &[Vec>] .flatten() .collect(); let a = alphabet.len(); - let mut have = vec![0usize; MAX_TOKEN + 1]; - for t in &book { - have[t.len()] += 1; - } // Alphabet-saturation control. With a small opcode alphabet, an n-gram // codebook can cover almost any sequence by combinatorics alone, which is // the boring explanation this workspace has already caught once: a 7-symbol From b86189200ef188bea56569a36265cb46babe7b74 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 5 Sep 2026 23:48:58 +0000 Subject: [PATCH 10/16] probe: remove the duplicate alphabet binding Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01EfrUJH3UNnv5NpDH4jDGHq --- examples/behavioral_soak_probe.rs | 1 - 1 file changed, 1 deletion(-) diff --git a/examples/behavioral_soak_probe.rs b/examples/behavioral_soak_probe.rs index 39312de2..786d455c 100644 --- a/examples/behavioral_soak_probe.rs +++ b/examples/behavioral_soak_probe.rs @@ -271,7 +271,6 @@ fn composition(label: &str, names: &[String], per_block_ops: &[Vec>] .flatten() .flatten() .collect(); - let a = alphabet.len(); // Alphabet-saturation control. With a small opcode alphabet, an n-gram // codebook can cover almost any sequence by combinatorics alone, which is // the boring explanation this workspace has already caught once: a 7-symbol From ed75c44992aca7a149c699881cb633ea7ca1c00a Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 5 Sep 2026 23:49:17 +0000 Subject: [PATCH 11/16] probe: drop the duplicated alphabet-saturation block Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01EfrUJH3UNnv5NpDH4jDGHq --- examples/behavioral_soak_probe.rs | 10 ---------- 1 file changed, 10 deletions(-) diff --git a/examples/behavioral_soak_probe.rs b/examples/behavioral_soak_probe.rs index 786d455c..9245f7f6 100644 --- a/examples/behavioral_soak_probe.rs +++ b/examples/behavioral_soak_probe.rs @@ -261,16 +261,6 @@ fn composition(label: &str, names: &[String], per_block_ops: &[Vec>] } } } - // Saturation control: with a ~12-symbol opcode alphabet a codebook can cover - // every short n-gram by combinatorics alone, at which case a high ops/token - // is arithmetic, not transfer. Report what fraction of the possible n-grams - // over the observed alphabet the codebook actually holds. - let alphabet: HashSet<&String> = per_block_ops - .iter() - .take(first_n) - .flatten() - .flatten() - .collect(); // Alphabet-saturation control. With a small opcode alphabet, an n-gram // codebook can cover almost any sequence by combinatorics alone, which is // the boring explanation this workspace has already caught once: a 7-symbol From e8927e1e02047b68a2ce7b3dd5057c50fedc1fcb Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 5 Sep 2026 23:52:39 +0000 Subject: [PATCH 12/16] plan: plastic-activation-seam research map v1 (sections B-L; A pending the physics audit) A discovery map, not an implementation plan. Records what this session measured about whether a small plastic mechanism could make a large dormant substrate activate usefully, and grades every claim F/M/I/S. The load-bearing findings are negative or deflationary. Zero of fifty-four MedCare capabilities are outcome-driven. The composition headline from the behavioural soak dies against its own null, leaving only a 6.6x state-budget advantage. Ternlog saves instructions rather than bandwidth and its advantage is gone by depth 32. Codebook growth is linear and accelerates at a domain boundary, so the scale-inversion hope is unsupported on that carrier. The hex line is killed again with a reason that generalizes: degree-6 over a seven-symbol alphabet is the complete graph minus self. The one constructive finding is that the activation seam already exists and has no callers. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01EfrUJH3UNnv5NpDH4jDGHq --- .claude/plans/plastic-activation-seam-v1.md | 413 ++++++++++++++++++++ 1 file changed, 413 insertions(+) create mode 100644 .claude/plans/plastic-activation-seam-v1.md diff --git a/.claude/plans/plastic-activation-seam-v1.md b/.claude/plans/plastic-activation-seam-v1.md new file mode 100644 index 00000000..71fd9cd2 --- /dev/null +++ b/.claude/plans/plastic-activation-seam-v1.md @@ -0,0 +1,413 @@ +# Plastic activation seam — research map v1 + +**Status:** DISCOVERY MAP. No learner proposed for implementation. Every number +below is either a MEASUREMENT taken this session, a FACT read from source, or is +marked UNRECOVERED. Probes are committed; nothing production changed. + +Grading used throughout: **[F]** fact read from source · **[M]** measured this +session · **[I]** inference from F/M · **[S]** speculation. + +--- + +## C. The old 95% finding — what it proves and what it does not + +### C.1 What was actually recovered + +The pipeline is real and its shape matches the reconstruction **[F]**: ELF → +r2sleigh/libsla lift → `ruff_r2il::behavior::FunctionBehavior` (a lossless ore +carrier that "invents no vocabulary, and it decides nothing") → +`furnace::smelt` against a seven-opcode convention (`convention.rs:104-110`: +`Copy, IntAdd, Load, Store, CBranch, Call, Return`) → ore TSV → forward def-use +chains → greedy BPE merges. + +`ogar-loco` exists as a crate and **contains no learner** — one doc-comment +mentions R2IL and nothing else **[F]**. The only BPE code in the four roots is +a throwaway example, `lance-graph-planner/examples/probe_bpe_r2il_loco_microcode.rs`, +which hardcodes loco's constants and asks whether learned merges would fit its +geometry. "The ogar-loco experiment" is that fit question, not a learner in +the crate. + +### C.2 The numbers, classified + +| number | status | +|---|---| +| 1,872 chains · 33 macros | live-derivable from the committed pass-1 corpus; **never asserted in any test or output file** | +| 8,659 / 8,652 / 2.409 / 2.529 / 0.994 | **prose only.** The instrumented harness was reverted; the entry itself says "no source change landed" | +| `ore_all.tsv` (94,536 rows), the 72,567-row serde_json ore | **not on disk anywhere** | +| "99.6% hexagon op-decode, 3 training programs, muscle-memory arc" | **UNRECOVERED.** One operator sentence, propagated into three derived docs, and graded *artifact not located* by a prior session on the same day it was written | + +### C.3 The boring explanations, and which one bites + +The board already deflated its own headline **[F]**: "608/608 looked +tautological for a 7-symbol alphabet"; and the PAL 0.99–1.00 transfer was +corrected the same day as "a **needle** test, not a distribution test … exact +addresses transfer when the address space is shared." + +Reported controls were two shuffle nulls. **No opcode-frequency baseline, no +bigram baseline, and no occurrence-matched control was ever run** **[F]**. + +**The decisive structural fact: the carrier cannot see a loop.** The def-use +chain extractor is forward-only by construction — +`probe_r2il_defuse_macros.rs:234` calls it "the **back-edge exclusion fence**" +**[F]**. So no experiment on this carrier could ever have discovered +WHILE-like structure, however it scored. The "child learns the grammar without +being handed the grammar book" hypothesis is not merely unproven on this +substrate; it is **inexpressible** on it. + +### C.4 This session's independent replication attempt + +Rather than trust the numbers, the same *shape* of experiment was rebuilt on +ore that does exist (`examples/behavioral_soak_probe.rs`): soak one never-reset +codebook across corpora and measure reuse against new mint, with a granularity +ladder so the trivial rung is visible. + +**[M] Cross-ISA exact-match transfer.** A codebook minted from 1986 6502 game +code covers 50.4% and 56.8% of two x86-64 C binaries' basic blocks; the +marginal-preserving null reaches 17.1% and 37.5%. + +**[M] Where that transfer lives.** Stratified by block length: + +| block length | transfer, binary 1 | transfer, binary 2 | +|---|---|---| +| 1–4 ops | 83–100% | 56–96% | +| 8+ ops | **0.0%** (135 blocks) | **1.9%** (103 blocks) | + +Exact sequence identity is the wrong atom for a long block, and the largest +bucket in both binaries transfers essentially not at all. + +**[M] The composition question, and the null that kills it.** Covering long +blocks by concatenating short known tokens — the actual BPE question — gives +98.3% and 97.1% of x86 ops covered by tokens of length ≥ 2. **A shuffled soak +that preserves every block length and the opcode marginal exactly reaches 97.6% +and 95.6%.** Coverage is therefore not evidence. + +**[M] What survives the null: the state budget.** The shuffled codebook needs +**4,526 tokens** where the real one needs **681** — a 6.6× smaller codebook at +matched coverage. The alphabet control explains why coverage saturates: seven +symbols, and the real codebook holds 44.9% of possible bigrams, 15.2% of +trigrams, 0.3% of 6-grams. + +**Ruling on F2.** The cross-language regularity is real but it is **not** a +coverage phenomenon and **not** evidence of a behavioural basis in the strong +sense. Restated honestly: *lowering to a seven-opcode IR makes coverage +saturate for anyone, and the learned vocabulary's only measured advantage is +that it does the same job with 6.6× less state.* Compression, not +comprehension. + +--- + +## E. Ternlog amortization — measured, and the claim is half right + +`examples/ternlog_amortization_probe.rs`. All arms compute `A ∧ M₁ ∧ … ∧ M_K` +with a correctness gate on the surviving set, and every arm runs to a 60 ms +floor. + +**[M] Depth.** Folding two constraints into one `VPTERNLOGQ` saves +instructions, not bandwidth. T3/T1 sits at 0.44–0.70 while the working set is +L1-resident and **returns to ~1.0 by K=32**, where the masks stop fitting and +cost becomes memory-bound. Per-constraint cost is **flat** in K, not falling. + +**[M] Residency, derived from achieved mask traffic rather than read off +timings.** L1 peaks near 139 GB/s, L2 holds 90–93 GB/s, L3 settles at 29–30 +GB/s. Host is 48 KiB L1d, 2 MiB L2 per core, 260 MiB shared L3. + +**[M] Materialization.** 1,000 chained ternlog steps allocate **0 bytes**, +counted with a global allocator. + +**[M] The density crossover — the result that actually matters.** A bitset pays +for every bit, so the mask arm is **flat at ~3.2 µs per constraint at every +density** over a 2²⁰-bit substrate, while a survivor-walking sparse arm scales +with what survives. They cross **between 0.1% and 0.8% active** (roughly 2,000–7,000 +survivors out of 1,048,576). + +**[I] Consequence for an attention field.** A focus mask's cost is independent +of how few things are lit. For attention — whose whole point is that few things +are active — that is the wrong cost curve **unless the substrate is partitioned +so a mask only ever covers a resident region**. The honest architecture is +therefore hierarchical: sparse index across the substrate, mask within a tier. + +--- + +## F. Membrane — which half is mechanism + +**[F] The mechanical half is already shipped and is not a metaphor.** +`ndarray::simd::ternlog` exposes eight truth tables, and one of them is exactly +the membrane update: `AND2_ANDNOT = 0x40` computes `a & b & !c` over three +same-shaped masks in one instruction. Read as *activation ∧ permeability ∧ +¬inhibition*, that is a local gate deciding what propagates, what is blocked, +and it composes with `OR2_AND` (`(a|b) & c`, either prerequisite gated by a +third) and `MAJ3` (two-of-three majority). Three states — structural, +instantaneous, learned-permeability — can share one geometry with no conversion +and, per E, no allocation. + +**[S] The metaphor half.** Nothing here models calcium channels, and no +measurement supports calling a mask a synapse. The useful content is the +computational role only. + +**[I] What learning would have to be.** Under the membrane reading, learning is +`P[i] ↑ / P[i] ↓` on a permeability mask — a bit flip, or a small counter that +thresholds into a bit. That is the cheapest possible plastic state and it is +ternlog-native. Whether it is *sufficient* is exactly what probe H1 below asks. + +--- + +## G. Hex ruling + +**Killed again, and this time the reason generalizes.** Beyond E-Q6 (failed G1, +G2, G3) and Q8's degree-1 ablation ("identical to four decimals, at 5.5× less +memory … B is a bigram successor table; the topology is decoration"), Q8b +recorded the structural reason **[F]**: *"The atom alphabet is 7. A degree-6 +neighbourhood is therefore the complete graph minus self."* + +**[I]** A six-neighbour topology cannot carry information over an alphabet of +seven. The hex experiments were not unlucky; they were degenerate by +construction. Any future spatial-locality claim must first show its alphabet is +large enough for degree-6 to be a restriction rather than a tautology — that is +now the entry gate, and it is cheap to check. + +An audit also found **no hexagonal or axial adjacency exists anywhere** in +lance-graph, ndarray, or OGAR **[F]**. There was never a hex substrate to have +measured 99.6% on. + +--- + +## J. Scale law — measured, and it says no + +**[M]** From the soak curve, codebook size against cumulative corpus: + +| corpus | cumulative occurrences | codebook | codebook/occ | marginal atoms per occ | +|---|---|---|---|---| +| 6502 image 1 | 1,430 | 299 | 0.209 | 0.209 | +| 6502 image 2 | 1,787 | 370 | 0.207 | 0.199 | +| x86-64 binary 1 | 2,015 | 483 | 0.240 | **0.496** | +| x86-64 binary 2 | 2,316 | 613 | 0.265 | 0.432 | + +**No sublinear growth.** The ratio *rises*, and the marginal cost per new +occurrence more than doubles at the ISA boundary. On this carrier the plastic +state grows at least linearly and accelerates when the domain changes. + +**[I]** The scale-inversion hope is not supported by any measurement in this +workspace. It may still hold for a different atom — this measures block-opcode +sequences, not conductance over a fixed channel set, and a fixed channel set is +bounded by construction in a way a vocabulary is not. That distinction is the +one thing that keeps the hypothesis alive, and it is testable. + +--- + +## K. Kill conditions + +| claim | already dead | would die if | +|---|---|---| +| hex locality | **YES** — degree-6 over a 7-symbol alphabet is the complete graph; three failed gates | — | +| tiny-plastic-state / scale inversion | wounded — [M] shows linear-and-accelerating growth | a fixed-channel conductance field also grows with corpus | +| ternlog amortization | **half dead** — [M] flat, not falling; saves instructions not bandwidth | already answered; do not re-ask | +| membrane masks | alive | a permeability bit cannot beat a per-channel counter at equal bytes | +| learned activation | alive, untested | it cannot beat a bigram successor table at equal state — the Q8 outcome, one level up | +| behavioural basis (F2 strong form) | **dead** — [M] the null reaches 97.6% | — | +| behavioural basis (F2 weak form: 6.6× less state at matched coverage) | alive | an occurrence-matched control closes the gap | + +--- + +## B. Dormancy map — the number is zero + +Measured over MedCare-rs, the richest consumer of the substrate **[F]**: + +| count | what | +|---|---| +| 54 | distinct callable capabilities enumerated | +| 26 | reached only transitively from an HTTP handler, request-local | +| 6 | run once at boot | +| 22 | **no production caller at all** — tests, examples and bake binaries only | +| 1 | a caller that is neither route, test, nor boot (a cohort *generator*), feeding nothing back | +| **0** | **whose output is consumed by another adapter across requests, by a scheduler, or by any persisted outcome** | + +The five-level distinction resolves sharply. Capabilities *exist* (54) and are +mostly *reachable* (32). Selection is by URL: `patient.icd` → one of fifteen +curated enum values → an `is_a` ancestor walk to a depth the query string +supplies. Nothing is *selected* by a policy, and **nothing's activation changes +a later selection**. Every reasoning call recomputes from a boot-time synthetic +cohort and renders. The only per-request mutable state is memoization of +identical bytes, so latency changes and content cannot. + +Dormant-by-design capabilities include the abductive frontier (zero consumers), +a reinforcement lane whose own doc calls persistence "deferred", the cognitive +cycle driver (built only under a feature production never enables), lab trends, +vital stats, the Cypher engine, and audit-event emission — an audit sink is +constructed at boot and **no route ever emits an event**, so its merkle chain +stays at zero. + +**[F] The seam already exists and has zero callers.** `bake_hydrate::append_witness` +plus a `patient_nodes` Lance table are fully implemented, with the module doc +stating that appends "are the normal, expected operation". The table is seeded +once and never appended. Every overlay the reasoners produce is explicitly +ephemeral — one module says its overlay "is transient and is discarded after the +merge"; another pins its cycle counter to a constant so two requests are +byte-identical. + +**[I] So the missing thing is not knowledge and not an adapter.** It is a +*write*. The substrate has an outcome-recording seam, it is finished, and it is +not wired. Everything in §D is a proposal for what to write through it. + +--- + +## D. Plasticity candidates, ranked smallest first + +Ranking is by *state bytes and mechanism size*, not by appeal. Each is judged +against the same standard the workspace has already applied and failed things +by: it must beat a bigram successor table at equal state. + +**D1 — Visit-count conductance on a permeability mask.** *State:* one bit per +channel, flipped when a saturating counter crosses a threshold. *Update:* on a +useful outcome, increment channels on the successful path; decrement on +failure. *Readout:* `ternlog(active, permeable, ¬inhibited)`, one instruction. +*Bytes:* channels/8. At 4,096 channels, **512 bytes**. *Ternlog-native:* yes, +by construction. *Locality:* none required. *Biggest falsifier:* a per-channel +frequency count at the same bytes reaches the same recall — i.e. the Q8 outcome +one level up. + +**D2 — Successor table over channel pairs.** *State:* sparse `(from, to) → +count`. *Bytes:* grows with observed pairs; at 4,096 channels the dense bound +is 2 MB and the sparse reality far less. *Ternlog-native:* no — it is a lookup, +not a mask. *Falsifier:* it is the baseline everything else must beat; if it +wins, the answer is "a transition matrix" and the romance is over. **Rank it +first to run, not first to hope for.** + +**D3 — Eligibility trace over the mask.** *State:* D1 plus a decaying trace of +recently active channels, so credit reaches a path rather than an endpoint. +*Bytes:* one small counter per channel; 4,096 channels × u8 = **4 KB**. +*Ternlog-native:* the readout is, the decay is not. *Falsifier:* a trace of +length 1 (i.e. D1) matches it. + +**D4 — Learned inhibition only.** *State:* one *suppression* mask; nothing is +ever excited, only ruled out. *Bytes:* channels/8. *Rationale:* the substrate's +measured strength is Boolean elimination, and `AND2_ANDNOT` makes inhibition +one instruction. *Falsifier:* inhibition alone cannot bring a useful target +*earlier*, only cheaper — which section H1 measures directly. + +**D5 — Per-class conductance keyed by classid.** *State:* one counter per +`classid`, not per node — so state is bounded by the codebook (98 concepts +today), not by the graph. *Bytes:* ~100 counters, **under 1 KB, and constant in +graph size**. *This is the only candidate whose state provably cannot grow with +the substrate*, which is exactly what §J found the vocabulary carrier could not +promise. *Falsifier:* class granularity is too coarse to discriminate useful +from useless targets within a class. + +**D6 — Bloom-shaped "seen-useful" filter.** *State:* one bitset, hashed +membership of channel sets that preceded a good outcome. *Bytes:* tunable. +*Falsifier:* false-positive rate makes precision worse than D1 at equal bytes. + +**D7 — Logistic regression over channel features.** *State:* one weight per +channel, f32. *Bytes:* 16 KB at 4,096 channels. *Ternlog-native:* no — this is +the numeric rail. *Included deliberately* as the conventional baseline §14 of +the brief demands; if it wins decisively the Boolean framing is wrong. + +**D8 — Do nothing plastic; cache the frontier.** *State:* memoized BFS results. +*Rationale:* the honest null for the whole programme. If cached traversal at +equal bytes matches every learner, there is no plasticity finding. + +--- + +## H. Two minimal probes + +### H1 — focus propagation (arm A, smallest honest form) + +*Question:* at an identical expansion budget, does any plastic state reach a +useful target earlier than the shipped controller? + +*Substrate:* the largest edge set that can be fetched — SNOMED at 2,053,329 +edges over 484,036 concepts, bridged to MONDO by 9,161 xref pairs. Note this is +2M, not 10M; the 10M figure is reachable only as transitive closure (4,273,203 +ancestor pairs under `is_a ∪ part_of`), and closure is not the same object. + +*Seeds:* per patient, `labs(id)` plus the grounded anamnese entries; the oracle +is `SoaPatient.icd` resolved through `crosswalk::resolve_icd10`. + +*Arms:* A0 shipped controller (curated enum + `is_a` walk) · A1 candidate from +§D · A2 shuffled A1 · A3 bigram successor · A4 random state at identical bytes · +A5 cached BFS. + +*Gates, all mandatory:* the candidate space must be the open-set 5,750-key axis, +never the fifteen-value enum — the repo's own note calls the latter "15× +hindsight contamination". Activation sparsity is reported beside recall, so a +learner that lights everything is visibly disqualified. State bytes are equal +across arms by construction. + +### H2 — persistence (arm B, the actual plasticity question) + +*Question:* does episode *n+1* get behaviourally easier without the ontology +changing? + +*Mechanism:* wire `append_witness` — the seam that exists and has no caller — +so an episode's outcome lands in `patient_nodes`. Run E1…En updating `P`, freeze, +measure on held-out. + +*Curves:* expansions-to-target vs episode · precision vs episode · **state bytes +vs episode** (the §J question, re-asked on a fixed channel set where it may +answer differently) · recovery after unrelated episodes. + +*Kill:* the curve is flat, or state bytes track episodes linearly. + +**Neither probe is authorized here.** H2 in particular is a *write* into a +private clinical repo and is the operator's call, not an agent's. + +--- + +## I. Surprise criterion, quantitatively + +An event counts as unprogrammed-but-useful when **all four** hold: + +1. A capability becomes active that the shipped controller would **not** have + selected for this seed — verified by running A0 on the same seed and + confirming absence from its expansion set. +2. The activation is **useful**: it lies on the oracle path, or it strictly + reduces expansions-to-target versus A0. +3. It is **attributable to learned state**: the shuffled arm A2, at identical + bytes, does not produce it. This is the clause that makes the criterion + falsifiable rather than decorative. +4. The full causal trace is logged — seed, channel sequence, the plastic entries + consulted, and the outcome. + +Reported as a rate, not an anecdote: *surprises per 100 held-out episodes*, with +its A2 rate beside it. A single spectacular trace is not evidence. + +--- + +## L. Epiphanies + +**[F] The activation seam is already built and unwired.** `append_witness` plus +`patient_nodes` are complete, documented as the normal path, and have zero +callers. The gap between "has knowledge" and "has behaviour" is currently one +missing write, not a missing subsystem. + +**[M] Zero of fifty-four capabilities are outcome-driven.** Nothing in the +richest consumer becomes easier or harder to reach because of anything that +happened before. + +**[M] Coverage saturates; only the state budget carries signal.** Real and +shuffled soaks reach 98.3% and 97.6%. The real codebook does it with 6.6× less +state. Report budgets, never coverage. + +**[M] Ternlog saves instructions, not bandwidth.** Its advantage is real while +L1-resident and gone by K=32. Chaining is not a way to make more constraints +free; it is a way to make each one cheaper until the masks stop fitting. + +**[M] A bitset's cost is independent of sparsity, and attention is sparse.** The +mask/sparse crossover sits between 0.1% and 0.8% active. This is the strongest +architectural constraint found today: masks belong *within* a resident tier, +sparse indices *across* the substrate. + +**[F] The def-use carrier excludes back-edges by construction.** Loop discovery +was never expressible on the substrate every macro experiment used. + +**[F] Degree-6 over a seven-symbol alphabet is the complete graph minus self.** +The hex experiments were degenerate, not unlucky. + +**[I] The one candidate with a bounded state promise is per-class conductance.** +Everything keyed per node or per vocabulary item grew with the corpus when +measured; state keyed by classid cannot, because the codebook is 98 entries and +minted by hand. + +**[S] Focus as an emergent observable rather than a subsystem.** Untested. The +substrate has no salience, LOD, or label-budget layer — its own UX contract says +"layer 2 does not exist" — so there is currently nothing for focus to emerge +*into*, and that gap is a prerequisite, not a consequence. From 236b849c58186072c44a6bfed5ef84d83f39ef15 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 5 Sep 2026 23:53:16 +0000 Subject: [PATCH 13/16] =?UTF-8?q?plan:=20add=20the=20direction=20control?= =?UTF-8?q?=20=E2=80=94=20cross-ISA=20transfer=20is=20asymmetric?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Soaking x86-64 first and testing on 6502 gives 81% exact-match reuse where the forward direction gives 50-57%. A shared basis would transfer roughly symmetrically; a 30-point gap is what containment of a simpler repertoire by a richer one looks like. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01EfrUJH3UNnv5NpDH4jDGHq --- .claude/plans/plastic-activation-seam-v1.md | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/.claude/plans/plastic-activation-seam-v1.md b/.claude/plans/plastic-activation-seam-v1.md index 71fd9cd2..eb3148c6 100644 --- a/.claude/plans/plastic-activation-seam-v1.md +++ b/.claude/plans/plastic-activation-seam-v1.md @@ -87,10 +87,27 @@ matched coverage. The alphabet control explains why coverage saturates: seven symbols, and the real codebook holds 44.9% of possible bigrams, 15.2% of trigrams, 0.3% of 6-grams. +**[M] Direction control — transfer is strongly asymmetric.** Soaking the other +way round, an x86-64 codebook of 275 atoms covers **81.1% and 81.0%** of the two +6502 images, against 50.4% and 56.8% in the forward direction. + +| direction | soak atoms | exact-match reuse on the other ISA | +|---|---|---| +| 6502 → x86-64 | 370 | 50.4% · 56.8% | +| x86-64 → 6502 | 275 | **81.1% · 81.0%** | + +If two architectures had genuinely converged on a shared behavioural basis, +transfer would be roughly symmetric. A 30-point asymmetry says instead that one +repertoire largely *contains* the other: the 6502 corpus is repetitive (299 +atoms over 1,430 blocks) while the x86 corpus is varied (275 atoms over 529 +blocks). Containment of a simpler repertoire by a richer one is the ordinary +expectation, not evidence of convergence. + **Ruling on F2.** The cross-language regularity is real but it is **not** a coverage phenomenon and **not** evidence of a behavioural basis in the strong sense. Restated honestly: *lowering to a seven-opcode IR makes coverage -saturate for anyone, and the learned vocabulary's only measured advantage is +saturate for anyone, the direction of transfer is asymmetric in the way +containment predicts, and the learned vocabulary's only measured advantage is that it does the same job with 6.6× less state.* Compression, not comprehension. From babf19dc3dde46af10d9462190030ff57d17bdc0 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 5 Sep 2026 23:54:53 +0000 Subject: [PATCH 14/16] =?UTF-8?q?plan:=20section=20A=20=E2=80=94=20shipped?= =?UTF-8?q?=20activation=20physics,=20from=20source?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two of fifteen capabilities become active in a shipped path; zero change a later selection, matching MedCare's independently measured zero of fifty-four. The uncomfortable finding is that the mechanism is not missing. A threshold-and-accumulate activation field exists in MailboxSoA and has zero non-test callers; four in-memory closed loops exist and are equally unreached; no mask cache exists at all; mask_ternlog_assign has no callers; and edges cannot be walked because the ref-to-row convention was never written. Sharpest of all, the one live dispatch path clones the seed at every rung, so its wave schedule models a dependency chain the runtime does not execute. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01EfrUJH3UNnv5NpDH4jDGHq --- .claude/plans/plastic-activation-seam-v1.md | 94 +++++++++++++++++++++ 1 file changed, 94 insertions(+) diff --git a/.claude/plans/plastic-activation-seam-v1.md b/.claude/plans/plastic-activation-seam-v1.md index eb3148c6..0796a970 100644 --- a/.claude/plans/plastic-activation-seam-v1.md +++ b/.claude/plans/plastic-activation-seam-v1.md @@ -7,6 +7,88 @@ marked UNRECOVERED. Probes are committed; nothing production changed. Grading used throughout: **[F]** fact read from source · **[M]** measured this session · **[I]** inference from F/M · **[S]** speculation. +--- + +## A. Shipped activation physics — what lance-graph can actually do today + +Read from source, callers counted outside `#[cfg(test)]`, `tests/`, `examples/`. + +### A.1 The five-level table + +| capability | exists | reachable | selected by a policy | active in a shipped path | changes a LATER selection | +|---|---|---|---|---|---| +| `PremiseBundle` → `BeliefArena` → `rcr_abduce` differential | Y | Y | Y | **Y** (MedCare open-set, non-default feature) | N — the arena is rebuilt per call | +| `dispatch_thought` wave schedule + recipe kernels | Y | Y | Y | **Y** (MedCare frontier dispatch) | N — see A.3 | +| `ShaderDriver` cycle: free energy → MUL gate → awareness revise → rung elevator | Y | Y | Y | N — lab/serve/grpc bins only | (Y in-process, never reached) | +| `AutocompleteCache` turn loop | Y | Y | Y | N — serve bin only | (Y in-memory, never reached) | +| `MetaOrchestrator` topology revise → `select_next` | Y | Y | Y | N — zero callers | (Y, never reached) | +| `cycle_driver::run_cycle`, held-owner re-poll | Y | Y | Y | N — zero non-test callers | N | +| `MailboxSoA` energy / threshold / `consume_firing` | Y | Y | Y | **N — zero non-test callers** | N | +| Planner 17-strategy affinity selection | Y | Y | Y | N — `LanceNativePlanner` has zero callers | N | +| `select_tactic` / `dispatch_mode::route` | Y | Y | Y | N — examples only | N | +| HHTL `RouteAction` skip/attend table | Y | Y | Y (percentile rule, at build) | N — `graph/audio` has no callers | N — static table | +| `NestedBands` ternlog buckets | Y | Y | N | N — test/probe only | N | +| `EdgeBlock` slot decode | Y | Y | N | N — and **no slot→row resolution exists anywhere** | N | +| BlasGraph `traverse` | Y | Y | N | N — the query path returns `Err` unconditionally | N | +| Gremlin `Traversal.step` | Y | Y | N | N — zero callers | N | +| `OgarAuthority::activate`, `ReinforcementLane` | Y | Y | N | N — zero callers | N | + +**Reaches "active": 2 of 15. Reaches "changes a later selection": 0 of 15.** +That matches MedCare's independently measured 0 of 54. + +### A.2 The plastic field is already built + +`mailbox_soa.rs:348/380` is the literal activation physics: energy accumulates +as `mantissa × confidence`, the row fires when `|energy| ≥ threshold`, once per +cycle. **`apply_edges` and `consume_firing` have zero non-test callers**, in +lance-graph or in MedCare, which touches only the setters. + +So the question "what is the smallest plastic mechanism" has an uncomfortable +answer: a threshold-and-accumulate field **exists, is shipped, and is not +called**. Four further closed loops exist and are equally unreached — an +awareness vector revised each cycle and read back into the gate, a turn cache +that mutates and reads its own state, a topology that records outcomes and +selects on them, and a driver that re-polls held owners. + +### A.3 The one live dispatch path discards its own propagation + +`wave_dispatch.rs:62-87` runs a static dataflow schedule over recipe +`requires`/`writes` masks. Inside the per-wave closure it does +`let mut ctx = seed.clone();` — **every rung restarts from the seed**, so wave +*n+1* never observes wave *n*'s writes. The audit's phrasing: *the plan models a +dependency chain the runtime does not execute.* Its output is a scanpath of +claimed addresses, rendered to strings and dropped. + +This is the sharpest finding in the whole map. The single reached activation +loop is prevented from propagating by one clone, not by a missing subsystem. + +### A.4 Masks, ternlog, edges, traversal + +- **No mask cache exists.** Not implemented, not stubbed, not planned in code. +- **`mask_ternlog_assign` has zero callers in lance-graph.** The only ternlog + use is `nested_bands.rs`, one chain at bucket-build time, nothing in a request + path — and `NestedBands` is constructed only by a test and probes. +- **Edges cannot be walked.** `edge_slots_coarse` returns raw refs and its own + doc says resolving a ref to a neighbour row "needs the basin-local-index→row + convention"; that convention does not exist. Every `EdgeCodecFlavor` resolves + to `CoarseOnly` because the trait default ignores the class. +- **Eight traversal engines exist; none is reachable from MedCare.** The + GraphBLAS path returns `Err("not yet wired to input datasets")` unconditionally + with the traversal below it marked unreachable. +- **The 96-bit facet register is a format, not a selector.** Encode/decode is + used inside the contract crate; nothing in a shipped path selects on a rail + value, and `walk_rails` has zero callers. +- **Skip-vs-attend is decided by nothing today.** `SigmaBandScan::plan` is + literally `Ok(input)`; `TruthPropagation::plan` is the same shape. + +### A.5 Learned and persistent state + +Nothing is loaded at startup and modified by outcomes. The awareness vector is +bootstrapped fresh per process and never saved. `TripleModel` plasticity and the +reinforcement lane are in-memory with persistence documented as deferred. +`EpisodicMemory` is constructed only in tests. + + --- ## C. The old 95% finding — what it proves and what it does not @@ -391,6 +473,18 @@ its A2 rate beside it. A single spectacular trace is not evidence. ## L. Epiphanies +**[F] The plastic field, the closed loops, and the write seam are all already +built — and none is called.** A threshold-and-accumulate activation substrate +(`MailboxSoA` energy/`consume_firing`), four in-memory closed loops, and an +outcome-recording write (`append_witness` + `patient_nodes`) exist in shipped +source with zero non-test callers between them. The missing thing is not a +mechanism. + +**[F] The one live dispatch path cannot propagate, by one line.** +`dispatch_thought` clones the seed at every rung, so its own wave schedule never +carries wave *n*'s writes into wave *n+1*. Before any learner is designed, this +is the cheapest possible experiment in the entire map. + **[F] The activation seam is already built and unwired.** `append_witness` plus `patient_nodes` are complete, documented as the normal path, and have zero callers. The gap between "has knowledge" and "has behaviour" is currently one From a1880fa297cd67344bcf0cffc1d7d82a2af4499b Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 5 Sep 2026 23:56:13 +0000 Subject: [PATCH 15/16] fix: gate the ternlog probe behind the std feature CI's --no-default-features job builds examples, and this probe imports ndarray::simd, which is std-gated. Every sibling example that reaches simd or hpc already carries required-features = ["std"] for exactly this reason; the new probe was added without one, so tests/beta and tests/1.98.1 both failed to compile it. Reproduced the CI command locally: cargo test --no-run -p ndarray --no-default-features now succeeds with the example skipped, and the example still builds under default features. The two other probes added in this arc use only std and need no gate. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01EfrUJH3UNnv5NpDH4jDGHq --- Cargo.toml | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/Cargo.toml b/Cargo.toml index c1f0987b..11ba2ddb 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -55,6 +55,10 @@ required-features = ["splat3d"] name = "hex_trie_vs_gemm_probe" required-features = ["std"] +[[example]] +name = "ternlog_amortization_probe" +required-features = ["std"] + [[example]] name = "amx_gemm_bench" required-features = ["std"] From 3e598a1594f385ae42e3049f6f4ea68caf833be3 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 6 Sep 2026 00:08:06 +0000 Subject: [PATCH 16/16] review: address CodeRabbit findings on the Wave-0 probes and plan Code: - both counting allocators gain the SAFETY comments the workspace requires - the hex probe's correctness gate compares survivor SETS, not popcounts, at both sites; two different sets of equal size can no longer pass - the hex probe reports realized relation density counted from the matrix; the nominal deg/N overstated it (deg=4096 is 63.22%, not 100%) - the ternlog probe drops an unused id-list expansion in the density sweep - the soak probe documents G3 as what it is: an order-blind opcode:arity profile, a different cut from G2 rather than a finer rung Plan and blackboard: - D-GTM-F3 reopened as conditional after 0b found the BlasLevel3 trait facade - D-GTM-6 payload is mask words or the codebook entry; the stale index-list wording contradicted F4 - 11.1 pt 6, 12.4 and the blackboard no longer claim GEMM is required for value-carrying relations; 0j establishes the Boolean/valued type boundary only, and the weighted arm decides the algorithm - 12 gains the realized-density correction with the re-run numbers Gate re-run after the change: sets agree at every cell. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01EfrUJH3UNnv5NpDH4jDGHq --- .claude/blackboard.md | 8 ++-- .../gemm-ternlog-mask-consolidation-v1.md | 32 ++++++++++----- examples/behavioral_soak_probe.rs | 8 +++- examples/hex_trie_vs_gemm_probe.rs | 40 ++++++++++++++----- examples/ternlog_amortization_probe.rs | 8 +++- 5 files changed, 71 insertions(+), 25 deletions(-) diff --git a/.claude/blackboard.md b/.claude/blackboard.md index 3f0078ad..ebe3d534 100644 --- a/.claude/blackboard.md +++ b/.claude/blackboard.md @@ -38,9 +38,11 @@ The invariant's own falsifier passes. dense" — measured, there is NO density crossover: masks win 745x at 0.02% relation density and 297x at 100%. Both costs are flat in density (GEMM O(N²) FMAs; mask O(active·N/64) word ORs). Honest correction, a TYPE boundary not a -density one: **masks win whenever the relation is Boolean; GEMM is required when -the relation carries VALUES.** A bitmask is 32x denser than f32 before any -algorithm runs, so a 0/1 relation in f32 was never the right representation. +density one: **masks win whenever the relation is Boolean; a relation that +carries VALUES needs a value-aware algorithm** — which one (GEMM, CSR SpMV, other) +is the still-unrun weighted arm, not a conclusion of 0j. A bitmask is 32x denser +than f32 before any algorithm runs, so a 0/1 relation in f32 was never the right +representation. **The headline numbers are explicitly NOT evidence** (§12.5): the dense-f32 baseline is mis-specified, and the missing arm is CSR SpMV (O(nnz) — at deg=1 diff --git a/.claude/plans/gemm-ternlog-mask-consolidation-v1.md b/.claude/plans/gemm-ternlog-mask-consolidation-v1.md index 8d720ec9..f23c72a6 100644 --- a/.claude/plans/gemm-ternlog-mask-consolidation-v1.md +++ b/.claude/plans/gemm-ternlog-mask-consolidation-v1.md @@ -163,7 +163,7 @@ A 3-input `IMM` is the whole truth table (`simd.rs:561-563`), so any Boolean ove |---|---|---| | D-GTM-F1 | `gemm_f32` default = hand-rolled F32x16 `sgemm_blocked` on `avx512f` hosts for square-ish shapes; `matrixmultiply` for skinny/wide rectangles (threshold from D-GTM-0c) and on other hosts. Exact on both. | §1.3: wins 256³–4096³ (up to 7%); LOSES 10–19% at 256×8192×256 and 64×2048×8192 | | D-GTM-F2 | AMX serves `gemm_bf16` and `gemm_i8` ONLY. Any f32 AMX path is a named opt-in carrying its measured table. | §1.3; ndarray#303 | -| D-GTM-F3 | The facade is `backend::{gemm_f32, gemm_f64, gemm_bf16, gemm_i8}` (+ batched). Every other GEMM symbol is `pub(crate)`, a documented opt-in, or deleted. | §1.1 count 54→4 | +| D-GTM-F3 | ⊘ **REOPENED by D-GTM-0b (§10).** v1 froze `backend::{gemm_f32, gemm_f64, gemm_bf16, gemm_i8}` (+ batched) as THE facade. 0b then found a second shipped public contract, the `BlasLevel3` trait → `BlasFloat::backend_gemm`. Until one is named canonical (and the other made a documented delegate), F3 is CONDITIONAL: **W1's D-GTM-2 may not demote or bypass either contract.** Candidate resolution: the free functions delegate to the trait (trait = contract, functions = ergonomic facade); decided by a probe of external callers of each, not by preference. | §1.1 count 54→4 was blind to the trait; §10 0b | | D-GTM-F4 | ⊘ **AMENDED AGAIN by §11.4.** The pack CONSUMES MASK WORDS directly (tzcnt / vpcompress inside the panel window) — **no index list exists at any point**, not a `Vec` prologue (v1), not a panel-ahead `ArrayVec` (v1.1). Cache key stays `(mask generation, panel index)`; the reusable object generalizes to a permeability codebook entry (§11.6). | `substrate == mask geometry == projection surface` (§11.10, strengthening §11.4) | | D-GTM-F5 | Every accuracy test in this surface uses inputs whose significands exceed 8 bits, and tolerances at f32 grade (1e-5) for f32 APIs. | the vacuous `(i+j)*0.5` test that hid the bf16 loss (#303) | | D-GTM-F6 | Every new kernel lands with a two-sided pin: the fast path must beat the reference by a stated factor AND the reference must still be measurably slower — so a regression in either direction fails. | `three_pass_split_beats_one_bf16_pass` pattern | @@ -191,7 +191,7 @@ A 3-input `IMM` is the whole truth table (`simd.rs:561-563`), so any Boolean ove ### Wave 2 — the mask→GEMM seam (ndarray + lgj-abi, one PR each) - D-GTM-5 (ndarray): ⊘ corrected three times, see §11.4 — `pack_a_masked_f32(a, lda, mask, row_cursor, kc, k_start, buf) -> rows_packed` consumes mask words directly; **no index list, no `Vec`, no `ArrayVec`**. `pruned_gemm_rows` has zero callers (§10 0f) so this is a first writer. Ladder per D-GTM-0e; bytes-materialized per D-GTM-0k must be ≈ 0. -- D-GTM-6 (lgj-abi, **filed against mask-risc-lowering-v1, not built here**): a carving kind whose payload is that index list, keyed by mask generation, invalidated with the mask. This is the single ask of the other plan. +- D-GTM-6 (lgj-abi, **filed against mask-risc-lowering-v1, not built here**): ⊘ corrected to match F4/D-GTM-5 — a carving kind whose payload is the **mask words themselves (or the §11.6 permeability codebook entry)**, keyed by mask generation, invalidated with the mask. **No index list**: the earlier "payload is that index list" wording predated the §11.4 amendment and contradicted F4. If a consumer ever needs an index list it is a separate, non-GEMM artifact with its own named producer and consumer, never this carving. This is the single ask of the other plan. ### Wave 3 — consumers (last, by the STOP rule) @@ -450,10 +450,13 @@ production behaviour. 5. **The invariant: `substrate == selection geometry == routing geometry`.** Expanding a mask into IDs, materializing a neighbour list, or converting the trie into an edge table *for the hot path* is the loss condition. -6. **The hypothesis is NOT "TERNLOGQ replaces GEMM."** GEMM is attractive when - information is dense; a hex/trie field may win when cognition is mostly - *successive elimination of possibility*. Grey squeezes locally; white moves - the constraint field cheaply across distance. +6. **The hypothesis is NOT "TERNLOGQ replaces GEMM."** ⊘ *The density half of + this point is FALSIFIED by D-GTM-0j (§12.4): there is no density crossover for + a Boolean relation.* What survives: the boundary is the relation's TYPE. + Masks carry Boolean elimination; a value-carrying relation needs a + value-aware algorithm (which one — GEMM, CSR SpMV, or another — is the open + weighted probe, §12.5). Grey squeezes locally; white moves the constraint + field cheaply across distance. 7. **Underlined:** white matter is not another data structure — it is an *interpretation of packed location prefixes*. Hexagon supplies neighbourhood; trie supplies scale; TERNLOGQ supplies permeability. @@ -758,12 +761,23 @@ regardless. ⊘ **So "GEMM wins when dense" is FALSE as stated for a Boolean relation.** The honest correction, and it is a TYPE boundary rather than a density: -> **Masks win whenever the relation is Boolean; GEMM is required when the -> relation carries VALUES.** A bitmask is 32× denser than f32 *before any +> **Masks win whenever the relation is Boolean; a relation that carries VALUES +> needs a value-aware algorithm.** A bitmask is 32× denser than f32 *before any > algorithm runs*, so a Boolean relation in f32 was never the right > representation. Where a weight must be accumulated (evidence strength, a > learned probability, a distance), the mask arm cannot express the operation at -> all — that, not density, is where GEMM becomes mandatory. +> all — that, not density, is the boundary. **Which value-aware algorithm wins +> there (dense GEMM, CSR SpMV, something else) is NOT established by this probe**; +> it is the weighted arm (§12.5 pt 2) and nothing here should be read as +> "GEMM is required". + +**Density column correction (post-review).** The §12.4 table's "relation density" +column was the NOMINAL `deg / N`. The Random relation draws `deg` columns per row +with replacement, so collisions make the realized density lower; the probe now +counts non-zero cells. Re-run after the fix: 0.02 / 0.39 / 1.55 / 6.06 / 22.12 / +**63.22 %** for deg = 1 … 4096 (the nominal "100 %" row is really 63 %). The +conclusion is unchanged — both costs stay flat across the whole realized range — +and the correctness gate now compares survivor SETS, not counts, at every cell. ### 12.5 The baseline is mis-specified, and the headline numbers are NOT evidence diff --git a/examples/behavioral_soak_probe.rs b/examples/behavioral_soak_probe.rs index 9245f7f6..b122912e 100644 --- a/examples/behavioral_soak_probe.rs +++ b/examples/behavioral_soak_probe.rs @@ -22,7 +22,7 @@ //! | G0 | opcode | control — must saturate; proves nothing | //! | G1 | (kind, opcode) | still near-trivial | //! | G2 | block opcode sequence | the behavioural BPE token | -//! | G3 | block (opcode, in-arity, out-arity) sequence | shape-sensitive | +//! | G3 | block UNORDERED opcode:in-arity:out-arity profile | dataflow-shape, order-blind | //! | G4 | function block-token sequence | whole-routine shape | //! //! The null shuffles opcodes across blocks while preserving every block's LENGTH @@ -118,7 +118,11 @@ fn atoms(bs: &[Vec<&Fact>], rung: usize) -> Vec { 3 => bs .iter() .map(|b| { - // per-instruction shape: opcode plus how many values it consumed/produced + // Per-block dataflow profile: for each opcode, how many values it + // consumed/produced across the block. Deliberately ORDER-BLIND and + // occurrence-merged, so G3 is not strictly finer than G2 — a block + // `load,store,load` and `store,load,load` share one G3 atom. It is + // a different cut (dataflow shape), not a deeper rung of G2. let mut per: BTreeMap<&str, (u32, u32)> = BTreeMap::new(); for f in b { let e = per.entry(f.opcode.as_str()).or_default(); diff --git a/examples/hex_trie_vs_gemm_probe.rs b/examples/hex_trie_vs_gemm_probe.rs index f5ac502c..33c28655 100644 --- a/examples/hex_trie_vs_gemm_probe.rs +++ b/examples/hex_trie_vs_gemm_probe.rs @@ -46,14 +46,21 @@ static ALLOCED: AtomicUsize = AtomicUsize::new(0); static COUNTING: AtomicUsize = AtomicUsize::new(0); struct Counting; +// SAFETY: a pure pass-through allocator. Every call forwards the caller's own +// `Layout` (and, for `dealloc`, the pointer that `alloc` returned for that +// layout) to `System` unchanged, so `System` upholds the `GlobalAlloc` +// contract on our behalf; the only added work is a relaxed atomic counter +// that never touches the allocation. unsafe impl GlobalAlloc for Counting { unsafe fn alloc(&self, l: Layout) -> *mut u8 { if COUNTING.load(Ordering::Relaxed) == 1 { ALLOCED.fetch_add(l.size(), Ordering::Relaxed); } + // SAFETY: `l` is the layout the caller passed, forwarded verbatim. unsafe { System.alloc(l) } } unsafe fn dealloc(&self, p: *mut u8, l: Layout) { + // SAFETY: `p` was returned by `System.alloc(l)` above for this same `l`. unsafe { System.dealloc(p, l) } } } @@ -102,7 +109,9 @@ fn build_matrix_deg(rel: Rel, seed: u64, deg: usize) -> Vec { } } Rel::Random => { - // SAME edge count, no prefix structure. + // Same NOMINAL degree, no prefix structure. Random column draws + // collide, so the realized edge count is below N*deg; the table + // reports the realized density, counted from the matrix. for i in 0..N { for _ in 0..deg { let j = (splitmix(&mut s) as usize) % N; @@ -183,6 +192,18 @@ fn popcnt(m: &[u64]) -> u32 { m.iter().map(|w| w.count_ones()).sum() } +/// The GEMM arm's survivor vector as a bitset, so the gate compares SETS. +/// Two different survivor sets of equal size must not pass as agreement. +fn gemm_bits(gs: &[f32]) -> Vec { + let mut m = vec![0u64; W]; + for (i, &v) in gs.iter().enumerate() { + if v > 0.0 { + m[i / 64] |= 1u64 << (i % 64); + } + } + m +} + fn main() { println!("hex/trie vs GEMM — N={N}, mask={} B, matrix={} MB\n", W * 8, N * N * 4 / 1_048_576); @@ -271,11 +292,12 @@ fn main() { std::hint::black_box(&ms_); // ── correctness gate: same survivors, or the numbers mean nothing ── - let gemm_pop = gs.iter().filter(|&&v| v > 0.0).count() as u32; - let mask_pop = popcnt(&ms_); - assert_eq!( - gemm_pop, mask_pop, - "ARMS DISAGREE at dens={dens} depth={depth} ({name}): gemm {gemm_pop} vs mask {mask_pop}" + let gemm_set = gemm_bits(&gs); + assert!( + gemm_set == ms_, + "ARMS DISAGREE at dens={dens} depth={depth} ({name}): gemm {} vs mask {} survivors (sets differ)", + popcnt(&gemm_set), + popcnt(&ms_) ); println!( @@ -344,12 +366,12 @@ fn main() { } let mask_ns = t.elapsed().as_secs_f64() * 1e9 / (mr * 8) as f64; - let gp = gs.iter().filter(|&&v| v > 0.0).count() as u32; - assert_eq!(gp, popcnt(&ms_), "ARMS DISAGREE at deg={deg}"); + let gemm_set = gemm_bits(&gs); + assert!(gemm_set == ms_, "ARMS DISAGREE at deg={deg}: survivor sets differ"); println!( " {:>8} {:>8.2}% {:>12.0} {:>12.0} {:>8.1}x", deg, - 100.0 * deg as f64 / N as f64, + 100.0 * matrix.iter().filter(|&&v| v != 0.0).count() as f64 / (N * N) as f64, gemm_ns, mask_ns, gemm_ns / mask_ns.max(1e-9) diff --git a/examples/ternlog_amortization_probe.rs b/examples/ternlog_amortization_probe.rs index 7954dbe5..dc2f1589 100644 --- a/examples/ternlog_amortization_probe.rs +++ b/examples/ternlog_amortization_probe.rs @@ -44,12 +44,18 @@ use ndarray::simd::{mask_and_assign, mask_ternlog_assign}; /// Counting allocator — "materialized 0 bytes" is a measurement here, not a claim. struct Counting; static ALLOCATED: AtomicUsize = AtomicUsize::new(0); +// SAFETY: a pure pass-through allocator. Every call forwards the caller's own +// `Layout` (and, for `dealloc`, the pointer that `alloc` returned for that +// layout) to `System` unchanged, so `System` upholds the `GlobalAlloc` +// contract on our behalf; the only added work is a relaxed atomic counter. unsafe impl GlobalAlloc for Counting { unsafe fn alloc(&self, l: Layout) -> *mut u8 { ALLOCATED.fetch_add(l.size(), Ordering::Relaxed); + // SAFETY: `l` is the layout the caller passed, forwarded verbatim. unsafe { System.alloc(l) } } unsafe fn dealloc(&self, p: *mut u8, l: Layout) { + // SAFETY: `p` was returned by `System.alloc(l)` above for this same `l`. unsafe { System.dealloc(p, l) } } } @@ -296,7 +302,6 @@ fn main() { mask_and_assign(&mut t1v, m); } let survivors = popcnt(&t1v); - let sparse: Vec> = masks.iter().map(|m| to_ids(m)).collect(); let base_ids = to_ids(&base); const FLOOR: f64 = 0.060; @@ -337,7 +342,6 @@ fn main() { i5 += 1; } let ns5 = c5.elapsed().as_secs_f64() * 1e9 / (i5 as f64 * 8.0); - let _ = &sparse; let winner = if ns5 < ns3 { "sparse" } else { "mask" }; println!(" {:8} | {:9} | {:6.0} | {:10.0} | {:9.0} | {winner}", label, survivors, ns1, ns3, ns5);