diff --git a/.claude/blackboard.md b/.claude/blackboard.md index c1a5c3cc..ebe3d534 100644 --- a/.claude/blackboard.md +++ b/.claude/blackboard.md @@ -1,8 +1,181 @@ +## 2026-09-05 — D-GTM-0l MEASURED (prefix-tract coverage, R2IL 6502 ore) + +The probe I flagged as decisive ran. `examples/prefix_tract_coverage_probe.rs` +against the Elite-rs R2IL harvest (427 long-range call edges, 2,647 packed addresses; +ore passed by path, never vendored — its provenance forbids redistribution). + +- Prefix locality is REAL: k>=2 shared prefix is 15.69% of edges vs 1.52% under a + degree-preserving null (50 shuffles). ~11x enrichment. +- The tract codebook does NOT compress: 7.62x at k=1 (4 KiB buckets, 16 of them on a + 64 KiB image — no resolution left), 1.66x at k=2, 1.08x at k=3, 1.00x at k=4. +- The non-local residual is DIFFUSE, not a hub set: 262 edges over 113 targets, + entropy 6.36 bits vs 6.82 uniform. A k=1 tract plus explicit far edges is 318 + entries against 427 edges. + +Verdict [G] on this ore: "white matter as an interpretation of packed location +prefixes" fails its own success condition — one tract per edge is an edge list. +Scope leg: these are PHYSICAL addresses from a 1986 linker, not semantic addresses +minted so the prefix carries meaning. The falsification is of the physical case only. +Next probe is the same instrument against an OGAR-minted classid space; the plan's +new §13 carries the full tables and the pass/fail condition for it. + # Current epoch (2026-05-26) — splat / palette / pillar / 3DGS > **Read this first.** The "Polyglot Notebook" architecture below is a > separate/older program, not the current epoch. +## 2026-09-05 (W0 MEASURED) — mask/trie vs GEMM: 0k passes, 0j falsifies the density framing + +Probe `examples/hex_trie_vs_gemm_probe.rs` (committed, --release). N=4096, mask +512 B, dense relation 64 MB, correctness gate on survivor counts at every cell. + +**D-GTM-0k ANSWERED, cleanly.** Mask hot path = **0 bytes/step** at every +density, every depth, both relation shapes — measured with a counting global +allocator, not asserted. GEMM = 73,728 B/step (packing buffer inside gemm_f32). +The invariant's own falsifier passes. + +**D-GTM-0j FALSIFIES §11.1 pt 6.** "GEMM is attractive when information is +dense" — measured, there is NO density crossover: masks win 745x at 0.02% +relation density and 297x at 100%. Both costs are flat in density (GEMM O(N²) +FMAs; mask O(active·N/64) word ORs). Honest correction, a TYPE boundary not a +density one: **masks win whenever the relation is Boolean; a relation that +carries VALUES needs a value-aware algorithm** — which one (GEMM, CSR SpMV, other) +is the still-unrun weighted arm, not a conclusion of 0j. A bitmask is 32x denser +than f32 before any algorithm runs, so a 0/1 relation in f32 was never the right +representation. + +**The headline numbers are explicitly NOT evidence** (§12.5): the dense-f32 +baseline is mis-specified, and the missing arm is CSR SpMV (O(nnz) — at deg=1 +that is ~4096 FMAs, same order as the mask arm's 11.7 µs, so it would plausibly +cross). The one internally fair comparison is PREFIX vs RANDOM inside the mask +arm: 21-79 ns flat vs 305-35,522 ns scaling with active bits — structure worth +~3 orders of magnitude, degrading exactly where there is nothing to exploit. + +**The gate caught a bug in my own probe.** RANDOM failed immediately (912 vs +930): the GEMM arm computes `{i : srcs(i) ∩ active ≠ ∅}` while the mask arm was +unioning `srcs(i)` over active i — those agree only for a SYMMETRIC relation +(bucket membership is, random is not). The mask arm must union the TRANSPOSE. +A second flaw was caught by reading, not by any gate: mask timings at/below +timer resolution made every early "speedup" (25,940x…853,300x) a noise ratio; +both arms now run to a 50 ms floor and report ns/step. + +**0h graded [S]:** no perf in this sandbox; residency inferred from timing +(PREFIX 70 ns at depth 1 → 21-27 ns at depths 8/32, no knee to 32) — consistent +with resident, not proof. **0l is now the decisive probe** — with the density +axis dead, the hypothesis stands or falls on prefix-routing coverage vs codebook +entropy on the R2IL/C64 ore. New probe implied by 12.4: the weighted arm. + +## 2026-09-05 (v1.3) — the invariant strengthened: `substrate == mask geometry == projection surface` + +Operator: "make the 96-bit object holographic" was metaphor while the cube wanted +voxel-by-voxel states. The Panela/photolithographic layer fixes it — **the cube is +never stored.** Holographic now means: *the information to reconstruct/address the +relevant 3-D relation is distributed through the 2-D 6×2×8 surface*, and the +volume appears only when a question requires it. Depth ← packed location; local +curvature ← hex adjacency; scale ← trie prefixes; permeability ← masks; dumb +physics ← VPTERNLOGQ (meaning lives in layout, never in the instruction — +`membrane-tiers.md` T1 from the other side). Hardware inversion: don't flatten +3-D onto silicon; make the higher-dimensional object a mask-address projection of +the 2-D surface. **Resolves the §11.9 diamond flag [S]→[H]:** bonds are implied +by address+masks, not walked, so coordination-4 vs the 16-ary trie is not a +conflict. The hologram's test already sits in the program: recover the relation +(D-GTM-0l, codebook entropy) without allocating it (D-GTM-0k, bytes/step → 0). + +## 2026-09-05 (v1.2) — GEMM plan: grey/white matter over ONE packed register; D-GTM-5 corrected a third time + +**Operator statement folded in as §11.** Hex field = digital grey/white matter over +one packed-address substrate, not two graphs. Grey = local hex state in the +existing 96-bit `6×2×8` register (same geometry is substrate AND mask; learning +changes permeability masks, never pointers). White = trie routing through packed +location (`ADDRESS & PREFIX_MASK == PREFIX`; a tract is `(prefix, mask, learned +transition)`, never a materialized path). TERNLOGQ is the membrane algebra. +**Invariant: `substrate == selection == routing`** — expanding a mask into IDs, +materializing a neighbour list, or converting the trie to an edge table on the hot +path is the loss condition. Hypothesis is NOT "ternlog replaces GEMM": GEMM wins +when dense; hex/trie may win when cognition is successive elimination. + +**Reconciled with the measured hex record, not against it.** Q6/Q7/Q8 tested a +learned *association overlay* (recall/interference); §11 is a *compute + bytes* +claim — the bar r2il §7.2 already sharpened ("wins only as a COMPUTE topology"). +E-Q8's degree-ablation lesson is now a MANDATORY control on every new probe. §9 R1 +re-graded [S]→[H]: adjacency (grey, six neighbours) and carving (white, six rails) +are two readings of the same 12 bytes — `le-contract.md` §3 already says the +register "holds every sanctioned reading at once". + +**The census bites both existing HHTL arms AND my own §9:** blasgraph +`heel_search → Vec` (k=50 per tier) and splat3d `Vec` +both materialize IDs; `splat3d/tile.rs`'s packed `(tile_id<<32 | depth)` key already +conforms. **D-GTM-5 corrected a THIRD time:** v1 `Vec` → v1.1 `ArrayVec` +→ v1.2 `pack_a_masked_f32` consumes mask words directly (tzcnt / vpcompress), +zero index materialization. Each revision removed one layer; the invariant is the +fixed point. + +**K0..K7 [H]** = the SPO 2³ `TriadicProjection` masks (`cam-codebook-resonance- +projection.md`: 8 observation/query masks, "not decorative — the query grammar") += the 8 rows of a ternlog truth table. **Six operator falsifiers** D-GTM-0g..0l +(mask/trie vs GEMM; VPTERNLOGQ residency vs depth; density sweep; crossover; +bytes-materialized/step → 0; prefix-routing codebook entropy) with one +pre-registered kill condition. **Panela + diamond (§11.9):** the toy's +positive/negative shape IS the invariant; diamond lattice (coord 4, tetrahedral) +as white-matter geometry is [S] — it conflicts with the 16-ary nibble trie unless +read as two bits of a level; flagged for one operator word. + +## 2026-09-05 (later) — GEMM consolidation plan v1.1: Mississippi Queen amendment + Wave 0 static results + +**Operator metaphor folded in (§9).** Mississippi Queen: the river board is laid a +few hex tiles ahead of the lead boat, speed changes ±1 and is committed before the +move, extra maneuvers cost from a fixed coal budget, and a tile laid for the leader +is free for every boat behind. Graded per the mechanism-vs-rhyme rule: + +- **M1 [G] CORRECTS D-GTM-F4/D-GTM-5.** v1 said the compacted row-index list is + "built once per mask generation" — laying the whole river before any boat moves. + Wrong shape: `pack_a_f32` (`kernels_avx512.rs:552`) already walks a panel cursor, + so mask→index expansion belongs ONE PANEL AHEAD of that cursor, in that loop. + Signature changes `mask_to_row_indices(&[u64]) -> Vec` → + `next_panel_indices(&[u64], cursor, mr) -> ArrayVec` (stack, no + hot-loop alloc — which also fixes a quiet `data-flow.md` §1 violation in v1). +- **M1b [G] — the amortization itself.** Many boats, one river: the cache key is + `(mask generation, panel index)`, NOT the call. Per-call caching amortizes nothing. +- **M2 [H]** lookahead depth adapts ±1 and commits before the panel → D-GTM-0e + becomes a LADDER (lookahead 1/2/4/8 × density 10/50/90%), not one crossover. +- **M3 [H]** coal = a bounded budget for mid-stream re-chains; replaces the + T2→T1 prohibition (`membrane-tiers.md:105`) with a budget. +- **R1 [S] the hexagon is rhyme** pending one operator word: the game's six is + ADJACENCY, the substrate's six (`6×(u8:u8)` facet rails, 6-byte HHTL path = + CAM-PQ 6×256) is FIELD CARVING. Same cardinality, different mechanism. Unbuilt. + +**Wave 0 static probes run (§10) — each corrected the inventory:** + +- **0a:** both duplicate names DIVERGENT, only one a defect. `bf16_tile_gemm_16x16` + = polyfill (`simd_ops.rs`, F32x16 decode) vs dispatcher (`hpc/`, AMX/VNNI) — + legitimate, and `simd.rs:714` already renames the dispatcher `_amx`. + `simd_avx2.rs:462 sgemm_blocked` is a **naive scalar triple loop** — neither AVX2 + nor blocked; file, name and body disagree three ways. +- **0b:** `blas_level3.rs` is NOT empty — 393 lines, zero `pub fn` because it is a + **trait** (`BlasLevel3`: gemm/gemm_into/syrk/symm/trmm/trsm, blanket impl, + re-exported `simd.rs:656`) dispatching to `BlasFloat::backend_gemm` + (`backend/mod.rs:75`, impl'd **f32/f64 only**). CLAUDE.md was right; my `pub fn` + grep was blind. **Re-frames D-GTM-F3: TWO facades already exist** — the generic + trait method and the four free functions — and `BlasFloat`'s `num_traits::Float` + bound structurally excludes i8/bf16 from the generic one. W1's first question is + which is canonical, not how to build one. +- **0f caller census:** `pruned_gemm_rows` **0 callers**, `mixed_precision_gemm` + **0**, `blas_gemm` 0 external. §2.3 called `pruned_gemm_rows` "the ONLY existing + mask→GEMM bridge" — it is dead code, so D-GTM-5 is a FIRST WRITER, not a + migration. Only `bf16_tile_gemm_16x16` has real external consumers (5). + +**Incidental find, reported not fixed (lance-graph call sites):** the 5 external +references reach `bf16_tile_gemm_16x16` by two paths that resolve to two different +bodies. `symbiont/src/domino.rs:27` imports it from `ndarray::simd` alongside +`amx_available` and its doc mentions tile ops — but that name is the POLYFILL; it +wants `bf16_tile_gemm_16x16_amx`. `thinking-engine/examples/amx_bf16_probe.rs:15` +imports from `ndarray::hpc::bf16_tile_gemm::*`, reaching past the facade — the exact +form the "all SIMD from `ndarray::simd`, never `hpc::*`" iron rule forbids. ndarray's +own facade is correct; both defects are consumer-side (and symbiont is deprecated). + +**Not run:** 0c (f64/tail bench), 0d (MKL-ternlog tail hunch), 0e (the M2 ladder — +now known to measure a zero-caller kernel, fine for a probe, not production evidence). + ## 2026-09-05 — AMX f32 GEMM was silently bf16; `matmul_f32` made exact; consolidation plan filed **Finding (measured, PR #303):** `hpc::amx_matmul::matmul_f32` downcast both operands diff --git a/.claude/plans/gemm-ternlog-mask-consolidation-v1.md b/.claude/plans/gemm-ternlog-mask-consolidation-v1.md index abd7cb86..f23c72a6 100644 --- a/.claude/plans/gemm-ternlog-mask-consolidation-v1.md +++ b/.claude/plans/gemm-ternlog-mask-consolidation-v1.md @@ -1,5 +1,33 @@ # gemm-ternlog-mask-consolidation-v1 — one GEMM entry per dtype, masks as the prefilter, ternlog as the mask ALU +> **Status:** DRAFT v1.4 (2026-09-05) — §12 RUNS D-GTM-0g/0h/0i/0j/0k. 0k passes +> cleanly (mask hot path = 0 bytes/step, measured by a counting allocator). **0j +> falsifies the framing:** there is no density crossover — masks win 745×→297× from +> 0.02% to 100% relation density — because "GEMM wins when dense" is wrong for a +> BOOLEAN relation; the real boundary is a TYPE one (Boolean ⇒ masks, valued ⇒ GEMM). +> The dense-f32 baseline is mis-specified and the headline numbers are explicitly NOT +> evidence (§12.5); the missing arm is CSR SpMV. D-GTM-0l is now the decisive probe. +> +> **Status:** DRAFT v1.3 (2026-09-05) — §11.10 strengthens the invariant to +> `substrate == mask geometry == projection surface`: the 3-D field (diamond tracts, the +> "cube") is never allocated — it is a mask-address projection of the 2-D 6×2×8 surface, +> recovered only when a question requires it. "Holographic" gets a falsifiable definition +> (recoverability, D-GTM-0l) paired with non-allocation (D-GTM-0k). §11.9's diamond flag +> resolved [S]→[H]: bonds are implied, not walked, so no arity conflict with the trie. +> +> **Status:** DRAFT v1.2 (2026-09-05) — §11 folds in the operator's grey/white-matter +> statement: hex (grey, 6×2×8 rails, local permeability) and trie (white, 3×2 path +> prefixes, routing) are ONE 12-byte register read two ways; `substrate == selection == +> routing` is the V3 4+12 facet doctrine as a compute model. §9 R1 re-graded [S]→[H]; +> D-GTM-5 corrected a THIRD time (pack consumes mask words, zero index materialization); +> K0..K7 = the SPO 2³ triadic projections [H]; six operator falsifiers D-GTM-0g..0l with +> the E-Q8 degree-ablation control mandatory. +> +> **Status:** DRAFT v1.1 (2026-09-05) — §9 folds in the operator's Mississippi Queen +> metaphor: M1 CORRECTS D-GTM-F4/D-GTM-5 (panel-ahead expansion, not a whole-board +> prologue), M1b names where the cache lives, M2 re-shapes D-GTM-0e into a ladder, +> M3 turns the T2→T1 prohibition into a budget. R1 (the hexagon) is marked rhyme. +> > **Status:** DRAFT v1 (2026-09-05). Source-first: every "exists" row cites `file:line` > at ndarray `claude/great-curie-d2ufyl` HEAD (PR #303 + this doc). Every "proposed" > row carries a falsifier. Nothing in §5 is built. No kernel, no ABI symbol. @@ -64,7 +92,7 @@ Three claims, each falsifiable in §6: | Runtime re-exports | `matmul_f32`, `matmul_bf16_to_f32`, `matmul_i8_to_i32` (×2), `gemm_u8_i8` | `simd_runtime/matmul.rs:43,33,80,193,227` | feature `runtime-dispatch` mirror of the AMX API | | Misc | `matmul_vec`, `matmul_i8_to_i32_wasm` | `hpc/models/layers.rs:174`, `simd_wasm.rs:1474` | model layer GEMV; wasm arm | -**Counted, not estimated:** 54 entry points, 12 files, **4 unified.** `hpc/blas_level3.rs` (named in CLAUDE.md as "BLAS L3 gemm/syrk/trsm/symm") returned **zero** `pub fn` hits in this grep — W0 must resolve whether that module is empty, macro-generated, or misnamed. +**Counted, not estimated:** 54 entry points, 12 files, **4 unified.** ⊘ **§10 D-GTM-0b corrects this count's blind spot:** `hpc/blas_level3.rs` returned zero `pub fn` hits because its surface is a **trait** (`BlasLevel3`, six methods, blanket impl, re-exported at `simd.rs:656`) dispatching through `BlasFloat::backend_gemm` — a dtype-generic facade that already exists beside the four free functions. A `pub fn` grep cannot see a method; any re-inventory must grep `fn`. ### §1.2 — ternlog: already a T1 primitive, already chained once @@ -135,8 +163,8 @@ A 3-input `IMM` is the whole truth table (`simd.rs:561-563`), so any Boolean ove |---|---|---| | D-GTM-F1 | `gemm_f32` default = hand-rolled F32x16 `sgemm_blocked` on `avx512f` hosts for square-ish shapes; `matrixmultiply` for skinny/wide rectangles (threshold from D-GTM-0c) and on other hosts. Exact on both. | §1.3: wins 256³–4096³ (up to 7%); LOSES 10–19% at 256×8192×256 and 64×2048×8192 | | D-GTM-F2 | AMX serves `gemm_bf16` and `gemm_i8` ONLY. Any f32 AMX path is a named opt-in carrying its measured table. | §1.3; ndarray#303 | -| D-GTM-F3 | The facade is `backend::{gemm_f32, gemm_f64, gemm_bf16, gemm_i8}` (+ batched). Every other GEMM symbol is `pub(crate)`, a documented opt-in, or deleted. | §1.1 count 54→4 | -| D-GTM-F4 | Mask→GEMM prefilter consumes a compacted row-index list built once per mask generation, stored as a mask carving. Never bit-tests inside the micro-kernel. | §2.3; mask-risc §14.7 | +| D-GTM-F3 | ⊘ **REOPENED by D-GTM-0b (§10).** v1 froze `backend::{gemm_f32, gemm_f64, gemm_bf16, gemm_i8}` (+ batched) as THE facade. 0b then found a second shipped public contract, the `BlasLevel3` trait → `BlasFloat::backend_gemm`. Until one is named canonical (and the other made a documented delegate), F3 is CONDITIONAL: **W1's D-GTM-2 may not demote or bypass either contract.** Candidate resolution: the free functions delegate to the trait (trait = contract, functions = ergonomic facade); decided by a probe of external callers of each, not by preference. | §1.1 count 54→4 was blind to the trait; §10 0b | +| D-GTM-F4 | ⊘ **AMENDED AGAIN by §11.4.** The pack CONSUMES MASK WORDS directly (tzcnt / vpcompress inside the panel window) — **no index list exists at any point**, not a `Vec` prologue (v1), not a panel-ahead `ArrayVec` (v1.1). Cache key stays `(mask generation, panel index)`; the reusable object generalizes to a permeability codebook entry (§11.6). | `substrate == mask geometry == projection surface` (§11.10, strengthening §11.4) | | D-GTM-F5 | Every accuracy test in this surface uses inputs whose significands exceed 8 bits, and tolerances at f32 grade (1e-5) for f32 APIs. | the vacuous `(i+j)*0.5` test that hid the bf16 loss (#303) | | D-GTM-F6 | Every new kernel lands with a two-sided pin: the fast path must beat the reference by a stated factor AND the reference must still be measurably slower — so a regression in either direction fails. | `three_pass_split_beats_one_bf16_pass` pattern | @@ -146,12 +174,12 @@ A 3-input `IMM` is the whole truth table (`simd.rs:561-563`), so any Boolean ove | id | probe | falsifier / gate | |---|---|---| -| D-GTM-0a | Diff `simd_ops.rs:587 bf16_tile_gemm_16x16` vs `hpc/bf16_tile_gemm.rs:45`; diff `simd_avx2.rs:462 sgemm_blocked` vs `kernels_avx512.rs:665`. | byte-identical bodies → delete one; divergent → record which is canonical and why | -| D-GTM-0b | Resolve `hpc/blas_level3.rs`: what does it export, if anything? | zero `pub fn` → either delete the CLAUDE.md claim or find the macro | +| D-GTM-0a | ✅ **RUN — §10.** Both pairs divergent. `bf16_tile_gemm_16x16` = polyfill vs dispatcher, legitimate, facade already renames one `_amx`. `sgemm_blocked` in `simd_avx2.rs` = a naive scalar triple loop (neither AVX2 nor blocked) — a naming defect. | v1 expected "byte-identical → delete"; neither pair is | +| D-GTM-0b | ✅ **RUN — §10.** A `BlasLevel3` trait, 6 methods, blanket impl, → `BlasFloat::backend_gemm` (f32/f64 only). CLAUDE.md was right; the grep was blind. **Re-frames D-GTM-F3: two facades already exist.** | found the surface; the macro hypothesis was wrong | | D-GTM-0c | Extend `gemm_paths_bench` to f64 (`dgemm_blocked` vs `gemm_f64_tiled_fma` vs matrixmultiply) and to non-square / K-tail shapes (e.g. 1000×1000×1000, 17×33×15 scaled). | if `sgemm_blocked` loses on any tail shape, D-GTM-F1 gains a shape guard, not a revert | | D-GTM-0d | **The MKL-ternlog hunch:** in `backend/mkl.rs` `sgemm` (`:384`), time the tail-lane handling with the current idiom vs one `mask_ternlog` select, K∈{255,257,1023,1025}. | < 3% end-to-end → record as shape-only win, no speed claim; ≥ 3% → W1 item | -| D-GTM-0e | `pruned_gemm_rows` (`prefilter.rs:189`): measure per-row bit-test vs compacted-index pack at 10%/50%/90% mask density. | the crossover density decides D-GTM-F4's threshold, or proves compaction always wins | -| D-GTM-0f | Count real call sites of every §1.1 symbol across ndarray, lance-graph, lance-graph-java, burn (`grep -rn`). | symbols with zero external callers are `pub(crate)` candidates for W1 with no consumer wave | +| D-GTM-0e | ⊘ **RE-SHAPED by §9 M2 into a LADDER:** `pruned_gemm_rows` (`prefilter.rs:189`) measured over lookahead ∈ {1,2,4,8} panels × density ∈ {10%,50%,90%}, per-row bit-test as the floor. | a single crossover cannot express a ±1-adaptive depth; the ladder decides the step size, or proves depth inert | +| D-GTM-0f | ✅ **RUN — §10.** `pruned_gemm_rows` and `mixed_precision_gemm` have **0 callers anywhere**; `blas_gemm` 0 external; only `bf16_tile_gemm_16x16` has real external consumers (5), reached by two paths that resolve to two different bodies (one an iron-rule violation, reported). | D-GTM-5 is a FIRST WRITER, not a migration | ### Wave 1 — ndarray (the facade and the backends) @@ -162,8 +190,8 @@ A 3-input `IMM` is the whole truth table (`simd.rs:561-563`), so any Boolean ove ### Wave 2 — the mask→GEMM seam (ndarray + lgj-abi, one PR each) -- D-GTM-5 (ndarray): `pruned_gemm_rows` takes a `&[u32]` compacted index list, not a mask; a `mask_to_row_indices(&[u64]) -> Vec` T1 primitive builds it (popcount + expand, one pass). Density crossover per D-GTM-0e. -- D-GTM-6 (lgj-abi, **filed against mask-risc-lowering-v1, not built here**): a carving kind whose payload is that index list, keyed by mask generation, invalidated with the mask. This is the single ask of the other plan. +- D-GTM-5 (ndarray): ⊘ corrected three times, see §11.4 — `pack_a_masked_f32(a, lda, mask, row_cursor, kc, k_start, buf) -> rows_packed` consumes mask words directly; **no index list, no `Vec`, no `ArrayVec`**. `pruned_gemm_rows` has zero callers (§10 0f) so this is a first writer. Ladder per D-GTM-0e; bytes-materialized per D-GTM-0k must be ≈ 0. +- D-GTM-6 (lgj-abi, **filed against mask-risc-lowering-v1, not built here**): ⊘ corrected to match F4/D-GTM-5 — a carving kind whose payload is the **mask words themselves (or the §11.6 permeability codebook entry)**, keyed by mask generation, invalidated with the mask. **No index list**: the earlier "payload is that index list" wording predated the §11.4 amendment and contradicted F4. If a consumer ever needs an index list it is a separate, non-GEMM artifact with its own named producer and consumer, never this carving. This is the single ask of the other plan. ### Wave 3 — consumers (last, by the STOP rule) @@ -190,3 +218,671 @@ A 3-input `IMM` is the whole truth table (`simd.rs:561-563`), so any Boolean ove 2. Does the reverse-engineered MKL path have any caller today, or is it a second facade with zero consumers? (D-GTM-0f decides whether W1 demotes it or deletes it) 3. At what mask density does compacted-index packing beat per-row bit tests? (D-GTM-0e) 4. Is there a real shape where an f32 AMX path wins? Nothing measured says yes; the opt-ins exist so the question stays cheap to re-ask. + +## §9 — The Mississippi Queen shape (operator metaphor, 2026-09-05) — v1.1 AMENDMENT + +**The game.** A paddle-steamer race whose river board does not exist in advance: +hex tiles are laid a few ahead of the lead boat, never the whole course. Each turn +you set speed by **±1 only** and commit it *before* moving that many hexes. One +direction change is free; extras are paid from a fixed **coal** budget. Several +boats race the same river, and a tile laid for the leader is free for everyone +behind. + +Graded per the workspace rule (`cross-domain-synthesizer`: shared MECHANISM is +transferable, mere rhyme is decorative and must be labelled). Three mechanisms, +one rhyme. + +### M1 — Reveal ahead of the cursor; never lay the whole river [G] — **CORRECTS D-GTM-F4 / D-GTM-5** + +D-GTM-F4 as written says the compacted row-index list is *"built once per mask +generation"*. That is laying the entire river before any boat moves: `O(n_rows)` +memory, and it pays for rows the GEMM may never reach — a pruned GEMM can stop +early, and a blocked GEMM only ever needs the panel under its cursor. + +The premise that makes the correction concrete is already in the code: +`pack_a_f32` (`backend/kernels_avx512.rs:552`) walks +`while ii + SGEMM_MR <= mc`, addressing `a[(i_start + ii + ir) * lda + …]`. +**Packing already has a cursor.** The mask→index expansion belongs at that same +cursor, in that same loop, one panel ahead — not in a prologue. + +Consequence, replacing D-GTM-5's signature: + +```rust +// WAS (v1): whole-board prologue, heap, pays for unreached rows +fn mask_to_row_indices(mask: &[u64]) -> Vec + +// IS (v1.1): one panel ahead of the pack cursor, stack, no hot-loop alloc +fn next_panel_indices(mask: &[u64], cursor: usize, mr: usize) -> ArrayVec +``` + +This also satisfies `.claude/rules/data-flow.md` §1 ("never allocate inside a hot +loop — slice into pre-allocated storage"), which the `Vec` version quietly +violated. + +### M1b — A tile serves every boat behind it [G] — this is *the* amortization + +Several boats race one river. The leader pays to reveal a tile; everyone behind +crosses it free. That is the amortization the operator named, and the game says +**where the cache lives**: on the *tile*, not on the *boat*. + +So the cache key is `(mask generation, panel index)` — **not** the call. Two GEMMs +against the same mask generation reuse the same expanded panels; a new mask +generation invalidates them wholesale (the registry already does exactly this to +`cached_carving`, `lgj-abi/registry.rs:842-849`). A per-call cache would re-lay the +river for every boat and amortize nothing. + +### M2 — Speed changes by ±1 and is committed before the move [H] + +Lookahead depth (how many panels ahead the expansion runs) is a state variable +that moves **one step at a time** and is **committed before the panel is entered**. +Two properties, both load-bearing: + +- *Hysteresis* — it cannot be re-derived per row, which is what stops a + per-call heuristic from thrashing between depths on adjacent panels. +- *Commit-ahead* — you cannot discover mid-panel that you needed a deeper + lookahead; by then the pack loop is already running. + +**Changes D-GTM-0e:** it measured a single crossover density. It now measures a +**ladder** — lookahead ∈ {1, 2, 4, 8} panels × density ∈ {10%, 50%, 90%} — because +a single crossover cannot express a ±1-adaptive depth. Graded [H]: the ladder shape +is argued, the step size is not yet measured. + +### M3 — Coal: a bounded budget for extra maneuvers [H] + +Re-chaining the ternlog predicate mid-stream (the mask gains a conjunct, or changes +shape) is a maneuver paid from a fixed budget. When the budget is spent you commit +to the mask you hold rather than re-deriving it. + +This is what keeps *chaining* from degenerating into *re-evaluate the predicate per +panel* — the exact T2→T1 violation `membrane-tiers.md:105` already forbids as a +prohibition. The game supplies the better form: **a budget rather than a ban**, so +the legitimate mid-stream re-chain stays possible and the pathological one runs out +of coal. + +### R1 — The hexagon itself is rhyme, pending one operator word [S] + +Six is conspicuous on both sides: the game moves on 6-neighbour hexes; this +substrate carves the 12-byte V3 facet as `6×(u8:u8)` and the HHTL path as 6 bytes += CAM-PQ `6×256`. **They are not obviously the same six.** The game's six is +*adjacency* (which cell may I move to next); the substrate's six is *field carving* +(which byte pair means what). Adjacency and carving are different mechanisms that +happen to share a cardinality — the textbook rhyme signature. + +Left [S] and unbuilt. If the operator meant the **rails** specifically, this is +promoted and gets its own section; nothing in M1-M3 depends on it either way. + +### What this amendment does NOT touch + +The AMX verdict (§1.3, measured), the facade consolidation (D-GTM-F1/F3), and the +W0 static probes (0a, 0b, 0f) are unaffected — the metaphor is about *when work is +done and who pays for it*, not about which kernel is fastest or how many entry +points exist. + +## §10 — WAVE 0 RESULTS (run 2026-09-05, static probes 0a / 0b / 0f) + +Three of the six W0 probes are measurement-free (greps and body diffs) and are run +here. Each corrected something in §1.1's inventory — which is the point of running +them before building anything. + +### D-GTM-0a — the two duplicate names: BOTH divergent, only ONE is a defect + +| pair | `simd_*` body | `hpc/` or `backend/` body | verdict | +|---|---|---|---| +| `bf16_tile_gemm_16x16` | `simd_ops.rs`, 37 lines — decode BF16→f32, then F32x16 + FMA. The **polyfill**. | `hpc/bf16_tile_gemm.rs`, 17 lines — `amx_available() \|\| avx512bf16` → VNNI-pack → tile tiers. The **dispatcher**. | **NOT a defect.** Backend vs dispatcher, legitimately distinct. The facade already disambiguates: `simd.rs:714` re-exports the hpc one **renamed** `bf16_tile_gemm_16x16_amx`; `simd.rs:744` exports the polyfill under the plain name. | +| `sgemm_blocked` | `simd_avx2.rs:462`, 14 lines — a **naive scalar triple loop** (`for i / for j / for p { sum += … }`). | `backend/kernels_avx512.rs:665`, 52 lines — the real packed-panel MR=6/NR=16 kernel. | **DEFECT, naming.** The `simd_avx2.rs` body is neither AVX2 nor blocked; the file name, the function name, and the body disagree three ways. | + +⊘ **Corrects v1's D-GTM-0a**, which anticipated "byte-identical bodies → delete +one". Neither pair is byte-identical and neither should be deleted. The real +finding is narrower and different: one legitimate tier pair (already handled by a +facade rename) and one mislabelled scalar fallback. + +### D-GTM-0b — `blas_level3.rs` is not empty; the inventory grep was blind + +393 lines, **zero `pub fn`** — because the surface is method-shaped: + +```rust +pub trait BlasLevel3 { + fn blas_gemm(&self, alpha: A, b: &Self, beta: A) -> Array; + fn blas_gemm_into(&self, alpha: A, b: &Self, beta: A, c: &mut Array); + fn blas_syrk (&self, uplo: Uplo, alpha: A, beta: A, c_init: Option<&Self>) -> Array; + fn blas_symm (&self, side: Side, uplo: Uplo, alpha: A, b: &Self, beta: A, c_init: Option<&Self>) -> …; + fn blas_trmm (&self, side: Side, uplo: Uplo, alpha: A, a_tri: &Self) -> Array; + fn blas_trsm (&self, side: Side, uplo: Uplo, alpha: A, b: &Self) -> Array; +} +impl BlasLevel3 for ArrayBase where A: BlasFloat + Float + AddAssign, S: Data +``` + +Re-exported at `simd.rs:656`. CLAUDE.md's "BLAS L3 (gemm, syrk, trsm, symm)" claim +was **accurate all along**; §1.1's `grep 'pub fn …gemm'` simply could not see a +trait. **Any future inventory of this crate must grep `fn`, not `pub fn`.** + +**And it dispatches through a facade that already exists.** `blas_gemm`'s body is +`A::backend_gemm(m, n, k, alpha, …)` — a method on `BlasFloat` (`backend/mod.rs:75`), +implemented for **f32 and f64 only** (`:83`, `:110`), with `f32::backend_gemm` → +`gemm_f32`. + +⊘ **This materially re-frames D-GTM-F3.** v1 said "the facade is +`backend::{gemm_f32, gemm_f64, gemm_bf16, gemm_i8}`; consolidate everything else +under it." In fact **two facades already exist side by side**: + +| | shape | dtypes | callers (0f) | +|---|---|---|---| +| `BlasFloat::backend_gemm` | dtype-**generic** trait method | f32, f64 | reached only via `BlasLevel3` | +| `backend::{cblas_sgemm, cblas_dgemm, gemm_i8, gemm_bf16}` | four **free functions** | f32, f64, i8, bf16 | 10 / 2 / 6 in-crate | + +The consolidation question is therefore **not** "build one facade" but "**which of +the two existing facades is canonical**". The constraint that decides it is already +in the source: `BlasFloat`'s impl bounds require `num_traits::Float`, so **i8 and +bf16 cannot join it** without a second trait or a bound change. A generic facade +that structurally excludes half the dtypes is not the canonical one. Recorded as +the first thing W1 must settle; no verdict claimed here. + +### D-GTM-0f — caller census: two of the plan's own load-bearing symbols are DEAD + +| symbol | in-crate | external | note | +|---|---|---|---| +| `bf16_tile_gemm_16x16` | 21 | **5** | the only symbol with real external consumers | +| `int8_gemm_vnni` | 12 | 0 | | +| `batched_gemm_f32` | 11 | 0 | | +| `cblas_sgemm` | 10 | 0 | | +| `gemm_f64_tiled_fma` | 9 | 0 | | +| `gemm_bf16` | 6 | 0 | | +| `sgemm_blocked` | 3 | 0 | `pub(crate)`-reachable only | +| `gemm_i8` | 2 | 0 | | +| `blas_gemm` | 2 | 0 | decl + impl; the trait facade is **unused** | +| **`pruned_gemm_rows`** | **0** | **0** | ⚠ | +| **`mixed_precision_gemm`** | **0** | **0** | ⚠ | + +**`pruned_gemm_rows` has zero callers.** §2.3 called it "the seed" and "the ONLY +existing mask→GEMM bridge", and §9 M1 rewrote its signature — all of that was +reasoning about **dead code**. It is still the right *shape* to build on, but the +plan must stop describing it as an existing integration: nothing integrates it. +D-GTM-5 is therefore a **first writer**, not a migration. + +### A live iron-rule violation, found incidentally by 0f + +The five external `bf16_tile_gemm_16x16` references reach it by **two different +paths**, and because 0a proved the bodies divergent, they are calling **different +functions**: + +- `lance-graph/crates/symbiont/src/domino.rs:27` — `use ndarray::simd::{amx_available, amx_report, bf16_tile_gemm_16x16}` → resolves to the **polyfill** (F32x16 decode), *not* a tile op. Its own module doc at `:8` reads *"which `bf16_tile_gemm_16x16` calls before any tile op"*, and it imports `amx_available` alongside — so the call site appears to expect the AMX dispatcher and receives the polyfill. `..._amx` is the name it wants. +- `lance-graph/crates/thinking-engine/examples/amx_bf16_probe.rs:15` — `use ndarray::hpc::bf16_tile_gemm::bf16_tile_gemm_16x16`, reaching **past the facade into `hpc::`**. That is the exact form the workspace iron rule forbids (*"all SIMD from `ndarray::simd`, never `ndarray::hpc::*`, never raw intrinsics"* — lance-graph-java `CLAUDE.md`, `abi.md` §8). It gets the dispatcher, correctly, by an illegal route. + +Both are **lance-graph** call sites, not ndarray's, and `symbiont` is +⊘ DEPRECATED (operator no-go, 2026-08-18) — so this is **reported, not fixed**, and +belongs to whoever next touches those files. ndarray's own facade is correct: the +rename at `simd.rs:714` is precisely the disambiguation these call sites failed to +use. + +### W0 remaining + +`0c` (f64 + tail-shape bench), `0d` (the MKL-ternlog tail hunch), `0e` (the M2 +lookahead × density **ladder**) are measurement probes and are not run here. Note +0e's target is now known to be dead code — the ladder measures a kernel with no +consumers, which is fine for a probe and must not be described as measuring +production behaviour. + +## §11 — v1.2: the hex/trie field is ONE packed-address substrate read two ways (operator statement, 2026-09-05) + +### 11.1 The statement, compressed to its load-bearing claims + +> *Treat the hexagon field as digital grey/white matter over one packed-address +> substrate, not as two separately materialized graphs.* + +1. **Grey = local hex state.** Each cell is a fixed-width state carrier — the + existing **96-bit 6×2×8** geometry. The same geometry is substrate AND mask. + Local learning changes *admissibility/permeability masks*, never an external + pointer structure. +2. **White = trie routing through packed location.** Long-range edges are never + object lists. Location is hierarchical *in the address bits*; each prefix is a + trie level/region; navigation is successive prefix refinement. Every prefix is + itself a mask: `ADDRESS & PREFIX_MASK == PREFIX`. A tract is + `(prefix, mask, learned transition)`, not a materialized path. +3. **TERNLOGQ is the membrane algebra.** `U' = ternlog(U, local_membrane_mask, + trie_route_mask)`; the surviving bits ARE the lawful next hex / prefix + refinement. Cached masks (`current_state`, `local_hex_mask`, + `trie_prefix_mask`, `learned_relation_mask`, `attention/focus_mask`) chain in + the same native bit geometry. +4. **Learning is a codebook, not addresses.** `context mask + relation atom + + packed prefix delta → permeability`. After exposure, most structure resolves + through learned vocabulary; only residual novelty alters the membrane. The + R2IL/C64 experiment is the model. +5. **The invariant: `substrate == selection geometry == routing geometry`.** + Expanding a mask into IDs, materializing a neighbour list, or converting the + trie into an edge table *for the hot path* is the loss condition. +6. **The hypothesis is NOT "TERNLOGQ replaces GEMM."** ⊘ *The density half of + this point is FALSIFIED by D-GTM-0j (§12.4): there is no density crossover for + a Boolean relation.* What survives: the boundary is the relation's TYPE. + Masks carry Boolean elimination; a value-carrying relation needs a + value-aware algorithm (which one — GEMM, CSR SpMV, or another — is the open + weighted probe, §12.5). Grey squeezes locally; white moves the constraint + field cheaply across distance. +7. **Underlined:** white matter is not another data structure — it is an + *interpretation of packed location prefixes*. Hexagon supplies neighbourhood; + trie supplies scale; TERNLOGQ supplies permeability. + +### 11.2 Why the measured hex record (Q6 / Q7 / Q8) does NOT close this — and what it DOES bind + +Three board entries measured "hex" and found it wanting: +`E-Q6-HEX-FAILS-CONTENT-ADDRESSING-…-1` (learns less *and* interferes more), +`E-Q7-…-COMPLEMENTARY-NOT-COMPETING-1` (frequency sizing rescues learning, not +interference), `E-Q8-THE-SIX-DOES-NO-WORK-…-1` (degree-1 ablation: identical +completion at 5.5× less memory — *"the information is in the PAIR, not in the +neighbourhood's shape"*). The #1023 audit found **zero** hex adjacency by grep +anywhere in lance-graph / ndarray / OGAR. + +**Those experiments tested a different claim.** Their B-arm was a *learned +association overlay* — a co-occurrence neighbourhood graph for macro recall, +scored on completion / false resonance / interference. §11.1 is a *compute and +memory* claim: propagation cost and bytes materialized under successive +elimination, against GEMM. That is precisely the bar `r2il-machine-semantic- +contract-v1` §7.2 already sharpened: *"if hex wins, it wins as a COMPUTE +topology — never retroactively as an explanation of the 96-bit register."* +§11.1 claims exactly and only that. + +**What the record binds, non-negotiably:** +- **E-Q8's lesson becomes a mandatory control.** *"A locality claim needs a + DEGREE ablation, not only a wiring null."* Every probe in §11.7 runs the hex + arm at degree 6 **and** degree 1; any advantage that survives degree 1 is not + hexagonal and must not be reported as such. +- The ratified demarcation is retained verbatim: *"White Matter ist Wahrheit und + Zwang. Grey Matter ist Hypothese und Plastizität. … Hexagon ist noch gar + nichts außer einem Kandidaten für lokale Rechengeometrie."* Grey is + hypothesis; a learned permeability mask is never a second truth + (`E-*-a-macro-never-becomes-a-second-truth`, the B4 invariant). + +⊘ **§9 R1 is therefore RE-GRADED, [S] → [H]-with-falsifier.** R1 said the game's +six (adjacency) and the substrate's six (carving) were different mechanisms. +That distinction stands — and §11.1 resolves it by assigning them to the two +tissues: adjacency is *grey* (the six neighbours, an unproven compute topology), +carving is *white* (the six rails, the packed address). Not rhyme, not the same +six — two readings of one register (11.3). + +### 11.3 6×2×8 and 3×2 are the SAME twelve bytes — this is already canon + +| reading | what the 12-byte V3 payload means | tissue | +|---|---|---| +| **rails** — `6×(u8:u8)` = 6×2×8 = 96 bits | six `palette256:palette256` pairs; the colon carries the distribution (`E-PALETTE256-IS-A-NEEDLE-THE-COLON-IS-THE-DISTRIBUTION-1`) | **grey** — local state | +| **path** — `HEEL:HIP:TWIG` = 3 tiers × 2 axes = 6 bytes | the CAM-PQ `6×256` code; `path distance = 3 tier-table lookups, O(1)`; longest-prefix binding; `is_ancestor_of` = centroid-tree containment (OGAR `CLAUDE.md` §Tier interpretation) | **white** — routing | + +`le-contract.md` §3 already says it: *"the 12B is a dumb byte register the +ClassView projects — it holds every sanctioned reading at once."* So the +operator's invariant `substrate == selection == routing` is **the V3 +content-blind 4+12 facet doctrine restated as a compute model.** Nothing new is +laid out; the proposal is a way of *executing over* the existing register. That +is what makes it admissible under the STOP rule — no new tissue. + +The L0…L5 ladder (`region → basin → tract → bundle → hex → local state`) is the +nibble-tree: 1 hex digit = 1 level of the 16-ary tree, tier-of-level = `level >> 2` +(a shift, never a branch). The trie already exists; §11.1 asks that it be *read as +a mask* rather than *walked as a structure*. + +### 11.4 Where the tree currently VIOLATES the invariant — including my own §9 + +The loss condition is materializing IDs on the hot path. Census: + +| site | what it materializes | verdict | +|---|---|---| +| `lance-graph/…/blasgraph/heel_hip_twig_leaf.rs` — `heel_search` → `Vec` = `Vec<(usize, u32)>`, `k = 50` survivors **per tier**, sorted + truncated | a gathered row set, four times per query | **violates** — the semantic HHTL arm is the anti-pattern §11.1 names | +| `ndarray/src/hpc/splat3d/depth_cascade.rs` — `cascade_blocks` → per-block `BlockDepthDecision { block_index, tier_reached, action, … }` | an ID-carrying decision per block | **violates** (spatial HHTL arm) — though `HhtlAction::{Reject, KeepCoarse, Refine, ProjectExact, RenderExact}` is already a 5-valued *"lawful next refinement"*, exactly §11.1's surviving-bits semantics wearing an enum | +| `ndarray/src/hpc/splat3d/tile.rs` — packed `u64` key `(tile_id << 32) \| depth_bits`, sorted tile-major | **nothing** — routes by packed-key prefix | **conforms** — the renderer already does white-matter routing by address prefix | +| **this plan, §9 M1** — `next_panel_indices(...) -> ArrayVec` | a per-panel index list | **violates.** Smaller than v1's `Vec`, still an ID expansion on the hot path. | + +⊘ **D-GTM-5 is corrected a THIRD time.** v1: whole-board `Vec`. v1.1 (§9): +panel-ahead `ArrayVec`. v1.2: **the pack consumes mask words directly** — +iterate set bits with `tzcnt` inside the panel window (or `vpcompress` on +AVX-512), gathering rows straight into the packed panel buffer, **zero index +materialization**: + +```rust +// v1.2 — the pack reads the mask; no index list exists at any point +fn pack_a_masked_f32(a: &[f32], lda: usize, mask: &[u64], row_cursor: usize, + kc: usize, k_start: usize, buf: &mut [f32]) -> usize /* rows packed */ +``` + +Each revision removed one more layer of materialization; the operator's +invariant names the fixed point. + +### 11.5 K0…K7 — the eight "terniating" masks are the SPO 2³ triadic projections [H] + +A 3-input `ternlog` immediate is an 8-entry truth table indexed by +`(a<<2)|(b<<1)|c` (`simd.rs:559-563`). Eight is also **exactly** the query +grammar the workspace already carries: + +```rust +// lance-graph/.claude/knowledge/cam-codebook-resonance-projection.md §SPO 2^3 +pub enum TriadicProjection { Abc, AbAskC, AcAskB, BcAskA, AOnly, BOnly, COnly, Background } +``` + +*"For a triad (A, B, C), there are 2³ observation/query masks … The 2³ +structure is not decorative. It is the query grammar for the CAM field."* So +`K0…K7` reads as: the eight presence-patterns of `(S, P, O)`, each selecting +which of the triadic pressures (`S×P → O`, `S×O → P`, `P×O → S`) applies, and +"terniating" = iterating the ternlog truth table over them. **[H] pending one +operator word** — the mapping is exact in cardinality and in role, but the name +`K0..K7` itself is not in the tree. + +### 11.6 What amortizes, precisely + +§9 M1b said the cache key is `(mask generation, panel index)`. §11.1 generalizes +the *tile*: the reusable object is a **codebook entry** +`(context mask, relation atom, packed prefix delta) → permeability mask`. It is +laid once — by whichever traversal first needs it — and every later traversal +that hits the same `(prefix, relation)` crosses it for free. That is the +Mississippi Queen tile at the level of *learned transitions*, and it is what +keeps learned semantics low-entropy (§11.7 probe 6 measures exactly whether it +stays that way). + +### 11.7 The falsification program — six operator probes, plus the E-Q8 control + +Each is a W0 probe; none is production code. Numbering continues §5. + +| id | probe (operator's words) | the measurement | the E-Q8 control | +|---|---|---|---| +| D-GTM-0g | mask/trie propagation vs dense GEMM for an equivalent sparse learned transition | wall time + result equality, same transition matrix realized both ways | hex arm at degree 6 **and** 1 | +| D-GTM-0h | does chained VPTERNLOGQ stay register/cache resident as depth increases | `perf` L1/L2 miss rate vs chain depth 1…32; the knee is the finding | — | +| D-GTM-0i | sweep unknown density almost-empty → almost-full | density ∈ {1,5,10,25,50,75,90,99}% | — | +| D-GTM-0j | the crossover where GEMM wins | from 0g × 0i: a density-vs-size frontier, not a single number | — | +| D-GTM-0k | **bytes materialized per inference step** | count every heap/stack byte that is not the mask itself; the hex/trie path must approach **zero** | this is the invariant's own falsifier | +| D-GTM-0l | can packed-prefix routing express all required long-range transitions without exploding codebook entropy | entries needed vs transitions covered, on the R2IL/C64 ore; the C64 vocabulary-resolution rate is the reference | — | + +**Pre-registered kill condition (one, stated before any run):** if at every +density in 0i the GEMM arm is both faster AND materializes fewer bytes than the +hex/trie arm, §11.1's hypothesis is false *for this substrate* and is recorded +as such — the same way Q6 was. + +### 11.8 Restated non-goals + +- Not "TERNLOGQ replaces GEMM" (§11.1 pt 6 — the operator's own fence). +- Not a new layout, a new crate, or a second graph. The register is the V3 + facet; the trie is the nibble tree; both exist. +- Not a re-run of Q6/Q7/Q8's association task. Different claim, different + metrics, same degree-ablation discipline. + +### 11.9 The Panela primitive, and hex + diamond as the two lattices (operator infographic, 2026-09-05) + +**Panela** — the DDR construction toy: one flat piece with an E-E comb profile that +interlocks with its own kind. *"One shape. Many worlds. Positive shape = +connectivity. Negative space = admissibility."* That is the §11.1 invariant as a +physical object: the same piece IS the structure and IS the selection, and the +infographic draws the 96-bit `6×2×8` register (six rows L0…L5 × byte 0 / byte 1) +**as** the E-profile. Substrate S and mask M are the same silhouette; ternlog +combines them without either ever leaving the lane. + +**Two lattices, one per tissue** — "from planar to space-filling": + +| lattice | coordination | character | tissue | +|---|---|---|---| +| **hexagonal** (Bienenwaben) | 6, isotropic, planar | efficient packing, natural for local reasoning | **grey** — dense local recurrence | +| **diamond** (Bindungen) | 4, tetrahedral, 3-D | strong directional bonds, efficient long-range structure | **white** — cross-scale tracts | + +A hex sheet per level, diamond links between levels: the cortical sheet with its +tracts. This is the first *geometric* statement of white matter in this plan — +§11.1 gave it as an *interpretation of prefixes*; the diamond gives it a shape. + +**One mismatch to flag, not resolve.** A diamond lattice has coordination **4**; +the canon's trie is **16-ary** (1 hex digit = 1 level, `FAN_OUT=16`, tier-of-level += `level >> 2`). A tetrahedral tract therefore does not map one-to-one onto a +nibble level — it maps onto **two bits** of one. Either the diamond is a +*visualization* of the four sub-branches a nibble refines through (in which case +it is rhyme, and harmless), or it is a claim that white-matter routing is 4-ary +at each step (in which case it conflicts with the 3×4 vs 4×3 standing watch and +needs its own probe). **Graded [S] until the operator says which.** Nothing in +§11.7's six probes depends on it. + +**The neuron reading** (dendrites = incoming masks, membrane = permeability mask, +axon = outgoing mask, learning = mask update, state = the 96-bit substrate) and +**constraint soaking** (`U₁ = U₀ ⊗ M₁`, `U₂ = U₁ ⊗ M₂`, … until residual) are §11.1 +points 1 and 6 restated; recorded as vocabulary, not as new claims. + +### 11.10 The amended invariant: `substrate == mask geometry == projection surface` (operator, 2026-09-05) + +**What was missing, in the operator's words.** *"Make the 96-bit object +holographic"* risked being metaphor, because a volumetric cube wants a huge +number of independently addressable states — the 3-D geometry existed with no +economical way of specifying its interior. The Panela / photolithographic layer +closes that: **the cube is never stored voxel-by-voxel.** + +``` +apparent 3-D field (diamond bonds, tracts, the "cube") ← NOT materialized + ⇅ projection +planar bit geometry: EE EE EE … = 6×2×8 register, masks + packed location +``` + +**Holographic, defined falsifiably:** not "every voxel exists" but *the +information required to reconstruct / address the relevant 3-D relation is +distributed through the 2-D representation.* You don't fill the hologram with +bits; you fill the surface with enough invariants that the volume can be +recovered *when something asks a question that requires it.* + +**Assignment of parts** (replaces §11.1 pt 7): + +| supplies | from | +|---|---| +| depth | packed location | +| local curvature | hex adjacency | +| scale | trie prefixes | +| permeability | masks | +| the dumb physics | VPTERNLOGQ | + +**The invariant, strengthened.** §11.1's `substrate == selection geometry == +routing geometry` becomes + +> **`substrate == mask geometry == projection surface`** + +The hardware inversion is the point: not *3-D problem → flatten awkwardly onto +silicon*, but *embrace the 2-D silicon-like representation and make the +higher-dimensional object a mask-address projection of it.* VPTERNLOGQ stays +wonderfully boring — three bit fields, one Boolean function, no knowledge of +neurons, cubes, hexagons or ontology. **The meaning lives in how the fields are +laid out, not in the instruction executing them** — which is `membrane-tiers.md`'s +T1 rule (a primitive is dumb; the tier above it carries the meaning) stated from +the other side. + +⊘ **§11.9's diamond flag is RESOLVED, [S] → [H].** The mismatch I flagged — a +tetrahedral tract (coordination 4) against the 16-ary nibble trie — assumed the +diamond bonds were *walked*. They are not: they are *implied* by address + masks +and recovered on demand. Nothing is 4-ary per step because nothing steps. The +diamond is the shape of the projection, not a level of the trie. No conflict with +the 3×4 standing watch; nothing to probe about arity. + +**What IS now falsifiable, and where it already sits in §11.7:** + +- *Holographic recoverability* = **D-GTM-0l**: can packed-prefix routing + + masks + codebook express every required long-range relation without codebook + entropy exploding? If the surface's invariants are insufficient, the volume + cannot be recovered and the claim fails there — measured on the R2IL/C64 ore. +- *The volume is never allocated* = **D-GTM-0k**: bytes materialized per + inference step → 0. A "projection" that secretly allocates the cube shows up + as bytes. + +Those two probes together are the hologram's test: recover the relation (0l) +without allocating it (0k). Nothing else in the program changes. + +## §12 — WAVE 0 MEASURED: D-GTM-0g/0h/0i/0j/0k run, and the hypothesis's own framing is corrected + +Probe: `examples/hex_trie_vs_gemm_probe.rs` (`--release`, committed). N = 4096, +mask = 512 B, dense relation matrix = 64 MB. Task, identical for both arms: +`D` steps of `state = R(state) ∩ constraint`, with a **correctness gate** — +survivor counts must match or the run aborts. + +### 12.1 The gate fired, and it found a bug in the probe itself + +First run: PREFIX passed at every cell, RANDOM failed immediately +(`gemm 912 vs mask 930`). Cause: the GEMM arm computes +`{ i : srcs(i) ∩ active ≠ ∅ }` while the mask arm was unioning `srcs(i)` over +active `i`. **Those agree only for a SYMMETRIC relation** — true of bucket +membership, false of a random relation. The mask arm must union the TRANSPOSE +(`fwd[j]` = what `j` can activate). Fixed; recorded because the asymmetry is +easy to reintroduce and the gate is the only thing that catches it. + +A second flaw was caught by reading, not by any gate: the mask arm first timed +at 0.000–0.001 ms — **at or below timer resolution**, so every "speedup" in that +table (25,940× … 853,300×) was a ratio against quantization noise. Both arms now +run to a 50 ms floor and report ns/step. + +### 12.2 What was measured (all cells passed the equality gate) + +**State-density sweep**, depth ∈ {1, 8, 32}: + +| relation | mask ns/step | GEMM ns/step | mask B/step | GEMM B/step | +|---|---|---|---|---| +| PREFIX (structured) | **21–79**, flat in state density | 8.4–8.8 M | **0** | 73,728 | +| RANDOM (no structure) | 305 (1%) → 35,522 (99%), scales with active bits | 10.6–13.0 M | **0** | 73,728 | + +**Relation-density sweep** (RANDOM, state 50%, depth 8) — the axis the first +table missed: + +| edges/row | relation density | GEMM ns/step | mask ns/step | speedup | +|---|---|---|---|---| +| 1 | 0.02% | 8,758,129 | 11,743 | 745.8× | +| 16 | 0.39% | 12,050,934 | 42,601 | 282.9× | +| 64 | 1.56% | 12,309,149 | 41,855 | 294.1× | +| 256 | 6.25% | 13,112,322 | 42,667 | 307.3× | +| 1024 | 25.00% | 12,525,352 | 42,559 | 294.3× | +| **4096** | **100.00%** | 12,776,345 | 43,024 | **297.0×** | + +### 12.3 D-GTM-0k — ANSWERED, and it is the one clean result + +**Mask hot path: 0 bytes/step at every density, every depth, both relation +shapes.** GEMM: 73,728 B/step (a packing buffer inside `gemm_f32`). The +invariant's own falsifier passes — nothing is materialized on the mask path, +measured by a counting global allocator rather than asserted. + +### 12.4 D-GTM-0j — there is NO crossover, and that falsifies the framing rather than confirming it + +§11.1 pt 6 says *"GEMM is attractive when information is dense; a hex/trie field +may win when cognition is mostly successive elimination."* **Measured, the +density axis does not produce a crossover at all** — the mask arm wins by ~300× +at 0.02% and by 297× at 100%. Both costs are flat in relation density: GEMM pays +`O(N²)` FMAs regardless, the mask arm pays `O(active · N/64)` word-ORs +regardless. + +⊘ **So "GEMM wins when dense" is FALSE as stated for a Boolean relation.** The +honest correction, and it is a TYPE boundary rather than a density: + +> **Masks win whenever the relation is Boolean; a relation that carries VALUES +> needs a value-aware algorithm.** A bitmask is 32× denser than f32 *before any +> algorithm runs*, so a Boolean relation in f32 was never the right +> representation. Where a weight must be accumulated (evidence strength, a +> learned probability, a distance), the mask arm cannot express the operation at +> all — that, not density, is the boundary. **Which value-aware algorithm wins +> there (dense GEMM, CSR SpMV, something else) is NOT established by this probe**; +> it is the weighted arm (§12.5 pt 2) and nothing here should be read as +> "GEMM is required". + +**Density column correction (post-review).** The §12.4 table's "relation density" +column was the NOMINAL `deg / N`. The Random relation draws `deg` columns per row +with replacement, so collisions make the realized density lower; the probe now +counts non-zero cells. Re-run after the fix: 0.02 / 0.39 / 1.55 / 6.06 / 22.12 / +**63.22 %** for deg = 1 … 4096 (the nominal "100 %" row is really 63 %). The +conclusion is unchanged — both costs stay flat across the whole realized range — +and the correctness gate now compares survivor SETS, not counts, at every cell. + +### 12.5 The baseline is mis-specified, and the headline numbers are NOT evidence + +Stated plainly so no future session cites 297× as support: + +1. **A dense f32 matrix for a 0/1 relation is an unfair baseline.** The mask arm + is not beating GEMM; it is beating a 32×-wasteful *representation* of a + Boolean relation. The number is real and the credit is misattributed. +2. **The missing arm is CSR SpMV.** A sparse f32 baseline costs `O(nnz)`, so at + `deg = 1` it is ~4,096 FMAs — the same order as the mask arm's 11.7 µs, and + it would plausibly cross. Until that arm is built, no claim about "mask beats + sparse GEMM" is available, and none is made here. +3. **The structured/unstructured separation is the one comparison that IS + internally fair** (same arm, same representation, same task): PREFIX 21–79 ns + flat vs RANDOM 305–35,522 ns scaling with active bits. ~3 orders of + magnitude, and it degrades exactly where the mask arm has no structure to + exploit — which is the E-Q8 discipline applied to compute rather than recall. + +### 12.6 D-GTM-0h — a limit of the sandbox, stated rather than papered over + +`perf` is unavailable here, so register/cache residency could not be measured +with counters. The timing proxy is suggestive only: PREFIX per-step cost *falls* +from ~70 ns at depth 1 to ~21–27 ns at depths 8 and 32 (loop-invariant setup +amortizing over more steps), with no knee up to depth 32 — consistent with +staying resident, and **not proof of it**. Graded [S] pending a machine with +counters. + +### 12.7 What W0 now leaves open + +- **D-GTM-0g** — partially answered; needs the CSR SpMV arm (12.5 pt 2) before + "mask/trie vs GEMM" means anything beyond "packed beats unpacked". +- **D-GTM-0l** — unrun. It is now the *decisive* probe: with the density axis + dead, the hypothesis stands or falls on whether packed-prefix routing can + express real long-range relations without codebook entropy exploding, on the + R2IL/C64 ore. +- **The weighted arm** — new, implied by 12.4: the type boundary needs its own + measurement (where does accumulating a value force GEMM?). +- 0c / 0d / 0e remain unrun. + +--- + +## §13 — D-GTM-0l MEASURED: packed-prefix routing does not cover real long-range edges + +Probe: `examples/prefix_tract_coverage_probe.rs` (committed). Ore: the R2IL 6502 +harvest in `AdaWorldAPI/Elite-rs` `.claude/harvest/r2il-6502/` — 34,186 lifted +instruction facts over two images, 427 `CallSite` long-range edges across 2,647 +distinct packed addresses. The ore is passed by path and never vendored; its own +provenance file forbids redistributing the corpus, and this repo stores the +measurement rather than the instruction stream. + +**The claim under test** (operator, grey/white-matter framing): *"white matter should +not be another data structure. It should be an interpretation of packed location +prefixes."* A tract is `(prefix, mask, learned transition)`; routing is +`ADDRESS & PREFIX_MASK == PREFIX`. That is a routing mechanism only if the tract +codebook is much smaller than the edge set it covers. + +### Result 1 — prefix locality is real signal, ~11× over the null + +| shared prefix | real | degree-preserving null (50 shuffles) | +|---|---|---| +| k=0 | 61.36% | 84.90% | +| k=1 | 22.95% | 13.54% | +| k=2 | 13.35% | 1.46% | +| k=3 | 2.34% | 0.06% | +| k=4 | 0.00% | 0.04% | + +k≥2 is 15.69% real against 1.52% null. The mechanism is not measuring noise. + +### Result 2 — but the codebook does not compress at any usable width + +| k | tract width | tracts | edges | compression | +|---|---|---|---|---| +| 1 | 4 KiB | 56 | 427 | 7.62× | +| 2 | 256 B | 257 | 427 | 1.66× | +| 3 | 16 B | 396 | 427 | 1.08× | +| 4 | 1 B | 427 | 427 | 1.00× | + +k=1 is the only width that compresses, and on a 64 KiB image it is 16 buckets — the +routing decision has almost no resolution left. By k=2 the codebook is already +1.66× and by k=3 it is an edge list with extra steps. + +### Result 3 — the non-local residual is DIFFUSE, so no exception table rescues it + +262 non-local edges over 113 distinct targets; top-10 targets cover 32.1%, top-40 +cover 64.9%; target entropy 6.36 bits against 6.82 uniform — only 0.46 bits below +uniform. There is no small hub set. A k=1 tract plus explicit far edges is 318 +entries against 427 edges (1.34×). + +### Verdict — [G] on this ore, with the scope leg named + +**On physically laid-out addresses, packed-prefix routing is a real but partial +signal that cannot carry the transition set alone.** The hypothesis's own success +condition ("not another data structure") fails: to resolve a destination you need +essentially one tract per edge. + +**Scope leg, load-bearing.** This ore's addresses are PHYSICAL — a 1986 linker's +layout, with no reason to be prefix-organized by meaning. The hypothesis was about +packed SEMANTIC addresses (`classid | HEEL | HIP | TWIG`, minted so that the prefix +IS the meaning). Those are different claims, and this measurement falsifies only the +first. It is the hardest case and arguably the wrong one — but it was the decisive +one *offered*, so the result stands as recorded rather than explained away. + +**What this makes the next probe.** Re-run the identical instrument against an +OGAR-minted address space, where prefixes are minted from the concept hierarchy +rather than from a linker. If enrichment rises and k=2/k=3 compression goes with it, +the mechanism survives on the substrate it was actually proposed for. If the +compression column looks like the table above, the tract framing is dead generally +and white matter needs to be a structure after all. diff --git a/.claude/plans/plastic-activation-seam-v1.md b/.claude/plans/plastic-activation-seam-v1.md new file mode 100644 index 00000000..0796a970 --- /dev/null +++ b/.claude/plans/plastic-activation-seam-v1.md @@ -0,0 +1,524 @@ +# Plastic activation seam — research map v1 + +**Status:** DISCOVERY MAP. No learner proposed for implementation. Every number +below is either a MEASUREMENT taken this session, a FACT read from source, or is +marked UNRECOVERED. Probes are committed; nothing production changed. + +Grading used throughout: **[F]** fact read from source · **[M]** measured this +session · **[I]** inference from F/M · **[S]** speculation. + +--- + +## A. Shipped activation physics — what lance-graph can actually do today + +Read from source, callers counted outside `#[cfg(test)]`, `tests/`, `examples/`. + +### A.1 The five-level table + +| capability | exists | reachable | selected by a policy | active in a shipped path | changes a LATER selection | +|---|---|---|---|---|---| +| `PremiseBundle` → `BeliefArena` → `rcr_abduce` differential | Y | Y | Y | **Y** (MedCare open-set, non-default feature) | N — the arena is rebuilt per call | +| `dispatch_thought` wave schedule + recipe kernels | Y | Y | Y | **Y** (MedCare frontier dispatch) | N — see A.3 | +| `ShaderDriver` cycle: free energy → MUL gate → awareness revise → rung elevator | Y | Y | Y | N — lab/serve/grpc bins only | (Y in-process, never reached) | +| `AutocompleteCache` turn loop | Y | Y | Y | N — serve bin only | (Y in-memory, never reached) | +| `MetaOrchestrator` topology revise → `select_next` | Y | Y | Y | N — zero callers | (Y, never reached) | +| `cycle_driver::run_cycle`, held-owner re-poll | Y | Y | Y | N — zero non-test callers | N | +| `MailboxSoA` energy / threshold / `consume_firing` | Y | Y | Y | **N — zero non-test callers** | N | +| Planner 17-strategy affinity selection | Y | Y | Y | N — `LanceNativePlanner` has zero callers | N | +| `select_tactic` / `dispatch_mode::route` | Y | Y | Y | N — examples only | N | +| HHTL `RouteAction` skip/attend table | Y | Y | Y (percentile rule, at build) | N — `graph/audio` has no callers | N — static table | +| `NestedBands` ternlog buckets | Y | Y | N | N — test/probe only | N | +| `EdgeBlock` slot decode | Y | Y | N | N — and **no slot→row resolution exists anywhere** | N | +| BlasGraph `traverse` | Y | Y | N | N — the query path returns `Err` unconditionally | N | +| Gremlin `Traversal.step` | Y | Y | N | N — zero callers | N | +| `OgarAuthority::activate`, `ReinforcementLane` | Y | Y | N | N — zero callers | N | + +**Reaches "active": 2 of 15. Reaches "changes a later selection": 0 of 15.** +That matches MedCare's independently measured 0 of 54. + +### A.2 The plastic field is already built + +`mailbox_soa.rs:348/380` is the literal activation physics: energy accumulates +as `mantissa × confidence`, the row fires when `|energy| ≥ threshold`, once per +cycle. **`apply_edges` and `consume_firing` have zero non-test callers**, in +lance-graph or in MedCare, which touches only the setters. + +So the question "what is the smallest plastic mechanism" has an uncomfortable +answer: a threshold-and-accumulate field **exists, is shipped, and is not +called**. Four further closed loops exist and are equally unreached — an +awareness vector revised each cycle and read back into the gate, a turn cache +that mutates and reads its own state, a topology that records outcomes and +selects on them, and a driver that re-polls held owners. + +### A.3 The one live dispatch path discards its own propagation + +`wave_dispatch.rs:62-87` runs a static dataflow schedule over recipe +`requires`/`writes` masks. Inside the per-wave closure it does +`let mut ctx = seed.clone();` — **every rung restarts from the seed**, so wave +*n+1* never observes wave *n*'s writes. The audit's phrasing: *the plan models a +dependency chain the runtime does not execute.* Its output is a scanpath of +claimed addresses, rendered to strings and dropped. + +This is the sharpest finding in the whole map. The single reached activation +loop is prevented from propagating by one clone, not by a missing subsystem. + +### A.4 Masks, ternlog, edges, traversal + +- **No mask cache exists.** Not implemented, not stubbed, not planned in code. +- **`mask_ternlog_assign` has zero callers in lance-graph.** The only ternlog + use is `nested_bands.rs`, one chain at bucket-build time, nothing in a request + path — and `NestedBands` is constructed only by a test and probes. +- **Edges cannot be walked.** `edge_slots_coarse` returns raw refs and its own + doc says resolving a ref to a neighbour row "needs the basin-local-index→row + convention"; that convention does not exist. Every `EdgeCodecFlavor` resolves + to `CoarseOnly` because the trait default ignores the class. +- **Eight traversal engines exist; none is reachable from MedCare.** The + GraphBLAS path returns `Err("not yet wired to input datasets")` unconditionally + with the traversal below it marked unreachable. +- **The 96-bit facet register is a format, not a selector.** Encode/decode is + used inside the contract crate; nothing in a shipped path selects on a rail + value, and `walk_rails` has zero callers. +- **Skip-vs-attend is decided by nothing today.** `SigmaBandScan::plan` is + literally `Ok(input)`; `TruthPropagation::plan` is the same shape. + +### A.5 Learned and persistent state + +Nothing is loaded at startup and modified by outcomes. The awareness vector is +bootstrapped fresh per process and never saved. `TripleModel` plasticity and the +reinforcement lane are in-memory with persistence documented as deferred. +`EpisodicMemory` is constructed only in tests. + + +--- + +## C. The old 95% finding — what it proves and what it does not + +### C.1 What was actually recovered + +The pipeline is real and its shape matches the reconstruction **[F]**: ELF → +r2sleigh/libsla lift → `ruff_r2il::behavior::FunctionBehavior` (a lossless ore +carrier that "invents no vocabulary, and it decides nothing") → +`furnace::smelt` against a seven-opcode convention (`convention.rs:104-110`: +`Copy, IntAdd, Load, Store, CBranch, Call, Return`) → ore TSV → forward def-use +chains → greedy BPE merges. + +`ogar-loco` exists as a crate and **contains no learner** — one doc-comment +mentions R2IL and nothing else **[F]**. The only BPE code in the four roots is +a throwaway example, `lance-graph-planner/examples/probe_bpe_r2il_loco_microcode.rs`, +which hardcodes loco's constants and asks whether learned merges would fit its +geometry. "The ogar-loco experiment" is that fit question, not a learner in +the crate. + +### C.2 The numbers, classified + +| number | status | +|---|---| +| 1,872 chains · 33 macros | live-derivable from the committed pass-1 corpus; **never asserted in any test or output file** | +| 8,659 / 8,652 / 2.409 / 2.529 / 0.994 | **prose only.** The instrumented harness was reverted; the entry itself says "no source change landed" | +| `ore_all.tsv` (94,536 rows), the 72,567-row serde_json ore | **not on disk anywhere** | +| "99.6% hexagon op-decode, 3 training programs, muscle-memory arc" | **UNRECOVERED.** One operator sentence, propagated into three derived docs, and graded *artifact not located* by a prior session on the same day it was written | + +### C.3 The boring explanations, and which one bites + +The board already deflated its own headline **[F]**: "608/608 looked +tautological for a 7-symbol alphabet"; and the PAL 0.99–1.00 transfer was +corrected the same day as "a **needle** test, not a distribution test … exact +addresses transfer when the address space is shared." + +Reported controls were two shuffle nulls. **No opcode-frequency baseline, no +bigram baseline, and no occurrence-matched control was ever run** **[F]**. + +**The decisive structural fact: the carrier cannot see a loop.** The def-use +chain extractor is forward-only by construction — +`probe_r2il_defuse_macros.rs:234` calls it "the **back-edge exclusion fence**" +**[F]**. So no experiment on this carrier could ever have discovered +WHILE-like structure, however it scored. The "child learns the grammar without +being handed the grammar book" hypothesis is not merely unproven on this +substrate; it is **inexpressible** on it. + +### C.4 This session's independent replication attempt + +Rather than trust the numbers, the same *shape* of experiment was rebuilt on +ore that does exist (`examples/behavioral_soak_probe.rs`): soak one never-reset +codebook across corpora and measure reuse against new mint, with a granularity +ladder so the trivial rung is visible. + +**[M] Cross-ISA exact-match transfer.** A codebook minted from 1986 6502 game +code covers 50.4% and 56.8% of two x86-64 C binaries' basic blocks; the +marginal-preserving null reaches 17.1% and 37.5%. + +**[M] Where that transfer lives.** Stratified by block length: + +| block length | transfer, binary 1 | transfer, binary 2 | +|---|---|---| +| 1–4 ops | 83–100% | 56–96% | +| 8+ ops | **0.0%** (135 blocks) | **1.9%** (103 blocks) | + +Exact sequence identity is the wrong atom for a long block, and the largest +bucket in both binaries transfers essentially not at all. + +**[M] The composition question, and the null that kills it.** Covering long +blocks by concatenating short known tokens — the actual BPE question — gives +98.3% and 97.1% of x86 ops covered by tokens of length ≥ 2. **A shuffled soak +that preserves every block length and the opcode marginal exactly reaches 97.6% +and 95.6%.** Coverage is therefore not evidence. + +**[M] What survives the null: the state budget.** The shuffled codebook needs +**4,526 tokens** where the real one needs **681** — a 6.6× smaller codebook at +matched coverage. The alphabet control explains why coverage saturates: seven +symbols, and the real codebook holds 44.9% of possible bigrams, 15.2% of +trigrams, 0.3% of 6-grams. + +**[M] Direction control — transfer is strongly asymmetric.** Soaking the other +way round, an x86-64 codebook of 275 atoms covers **81.1% and 81.0%** of the two +6502 images, against 50.4% and 56.8% in the forward direction. + +| direction | soak atoms | exact-match reuse on the other ISA | +|---|---|---| +| 6502 → x86-64 | 370 | 50.4% · 56.8% | +| x86-64 → 6502 | 275 | **81.1% · 81.0%** | + +If two architectures had genuinely converged on a shared behavioural basis, +transfer would be roughly symmetric. A 30-point asymmetry says instead that one +repertoire largely *contains* the other: the 6502 corpus is repetitive (299 +atoms over 1,430 blocks) while the x86 corpus is varied (275 atoms over 529 +blocks). Containment of a simpler repertoire by a richer one is the ordinary +expectation, not evidence of convergence. + +**Ruling on F2.** The cross-language regularity is real but it is **not** a +coverage phenomenon and **not** evidence of a behavioural basis in the strong +sense. Restated honestly: *lowering to a seven-opcode IR makes coverage +saturate for anyone, the direction of transfer is asymmetric in the way +containment predicts, and the learned vocabulary's only measured advantage is +that it does the same job with 6.6× less state.* Compression, not +comprehension. + +--- + +## E. Ternlog amortization — measured, and the claim is half right + +`examples/ternlog_amortization_probe.rs`. All arms compute `A ∧ M₁ ∧ … ∧ M_K` +with a correctness gate on the surviving set, and every arm runs to a 60 ms +floor. + +**[M] Depth.** Folding two constraints into one `VPTERNLOGQ` saves +instructions, not bandwidth. T3/T1 sits at 0.44–0.70 while the working set is +L1-resident and **returns to ~1.0 by K=32**, where the masks stop fitting and +cost becomes memory-bound. Per-constraint cost is **flat** in K, not falling. + +**[M] Residency, derived from achieved mask traffic rather than read off +timings.** L1 peaks near 139 GB/s, L2 holds 90–93 GB/s, L3 settles at 29–30 +GB/s. Host is 48 KiB L1d, 2 MiB L2 per core, 260 MiB shared L3. + +**[M] Materialization.** 1,000 chained ternlog steps allocate **0 bytes**, +counted with a global allocator. + +**[M] The density crossover — the result that actually matters.** A bitset pays +for every bit, so the mask arm is **flat at ~3.2 µs per constraint at every +density** over a 2²⁰-bit substrate, while a survivor-walking sparse arm scales +with what survives. They cross **between 0.1% and 0.8% active** (roughly 2,000–7,000 +survivors out of 1,048,576). + +**[I] Consequence for an attention field.** A focus mask's cost is independent +of how few things are lit. For attention — whose whole point is that few things +are active — that is the wrong cost curve **unless the substrate is partitioned +so a mask only ever covers a resident region**. The honest architecture is +therefore hierarchical: sparse index across the substrate, mask within a tier. + +--- + +## F. Membrane — which half is mechanism + +**[F] The mechanical half is already shipped and is not a metaphor.** +`ndarray::simd::ternlog` exposes eight truth tables, and one of them is exactly +the membrane update: `AND2_ANDNOT = 0x40` computes `a & b & !c` over three +same-shaped masks in one instruction. Read as *activation ∧ permeability ∧ +¬inhibition*, that is a local gate deciding what propagates, what is blocked, +and it composes with `OR2_AND` (`(a|b) & c`, either prerequisite gated by a +third) and `MAJ3` (two-of-three majority). Three states — structural, +instantaneous, learned-permeability — can share one geometry with no conversion +and, per E, no allocation. + +**[S] The metaphor half.** Nothing here models calcium channels, and no +measurement supports calling a mask a synapse. The useful content is the +computational role only. + +**[I] What learning would have to be.** Under the membrane reading, learning is +`P[i] ↑ / P[i] ↓` on a permeability mask — a bit flip, or a small counter that +thresholds into a bit. That is the cheapest possible plastic state and it is +ternlog-native. Whether it is *sufficient* is exactly what probe H1 below asks. + +--- + +## G. Hex ruling + +**Killed again, and this time the reason generalizes.** Beyond E-Q6 (failed G1, +G2, G3) and Q8's degree-1 ablation ("identical to four decimals, at 5.5× less +memory … B is a bigram successor table; the topology is decoration"), Q8b +recorded the structural reason **[F]**: *"The atom alphabet is 7. A degree-6 +neighbourhood is therefore the complete graph minus self."* + +**[I]** A six-neighbour topology cannot carry information over an alphabet of +seven. The hex experiments were not unlucky; they were degenerate by +construction. Any future spatial-locality claim must first show its alphabet is +large enough for degree-6 to be a restriction rather than a tautology — that is +now the entry gate, and it is cheap to check. + +An audit also found **no hexagonal or axial adjacency exists anywhere** in +lance-graph, ndarray, or OGAR **[F]**. There was never a hex substrate to have +measured 99.6% on. + +--- + +## J. Scale law — measured, and it says no + +**[M]** From the soak curve, codebook size against cumulative corpus: + +| corpus | cumulative occurrences | codebook | codebook/occ | marginal atoms per occ | +|---|---|---|---|---| +| 6502 image 1 | 1,430 | 299 | 0.209 | 0.209 | +| 6502 image 2 | 1,787 | 370 | 0.207 | 0.199 | +| x86-64 binary 1 | 2,015 | 483 | 0.240 | **0.496** | +| x86-64 binary 2 | 2,316 | 613 | 0.265 | 0.432 | + +**No sublinear growth.** The ratio *rises*, and the marginal cost per new +occurrence more than doubles at the ISA boundary. On this carrier the plastic +state grows at least linearly and accelerates when the domain changes. + +**[I]** The scale-inversion hope is not supported by any measurement in this +workspace. It may still hold for a different atom — this measures block-opcode +sequences, not conductance over a fixed channel set, and a fixed channel set is +bounded by construction in a way a vocabulary is not. That distinction is the +one thing that keeps the hypothesis alive, and it is testable. + +--- + +## K. Kill conditions + +| claim | already dead | would die if | +|---|---|---| +| hex locality | **YES** — degree-6 over a 7-symbol alphabet is the complete graph; three failed gates | — | +| tiny-plastic-state / scale inversion | wounded — [M] shows linear-and-accelerating growth | a fixed-channel conductance field also grows with corpus | +| ternlog amortization | **half dead** — [M] flat, not falling; saves instructions not bandwidth | already answered; do not re-ask | +| membrane masks | alive | a permeability bit cannot beat a per-channel counter at equal bytes | +| learned activation | alive, untested | it cannot beat a bigram successor table at equal state — the Q8 outcome, one level up | +| behavioural basis (F2 strong form) | **dead** — [M] the null reaches 97.6% | — | +| behavioural basis (F2 weak form: 6.6× less state at matched coverage) | alive | an occurrence-matched control closes the gap | + +--- + +## B. Dormancy map — the number is zero + +Measured over MedCare-rs, the richest consumer of the substrate **[F]**: + +| count | what | +|---|---| +| 54 | distinct callable capabilities enumerated | +| 26 | reached only transitively from an HTTP handler, request-local | +| 6 | run once at boot | +| 22 | **no production caller at all** — tests, examples and bake binaries only | +| 1 | a caller that is neither route, test, nor boot (a cohort *generator*), feeding nothing back | +| **0** | **whose output is consumed by another adapter across requests, by a scheduler, or by any persisted outcome** | + +The five-level distinction resolves sharply. Capabilities *exist* (54) and are +mostly *reachable* (32). Selection is by URL: `patient.icd` → one of fifteen +curated enum values → an `is_a` ancestor walk to a depth the query string +supplies. Nothing is *selected* by a policy, and **nothing's activation changes +a later selection**. Every reasoning call recomputes from a boot-time synthetic +cohort and renders. The only per-request mutable state is memoization of +identical bytes, so latency changes and content cannot. + +Dormant-by-design capabilities include the abductive frontier (zero consumers), +a reinforcement lane whose own doc calls persistence "deferred", the cognitive +cycle driver (built only under a feature production never enables), lab trends, +vital stats, the Cypher engine, and audit-event emission — an audit sink is +constructed at boot and **no route ever emits an event**, so its merkle chain +stays at zero. + +**[F] The seam already exists and has zero callers.** `bake_hydrate::append_witness` +plus a `patient_nodes` Lance table are fully implemented, with the module doc +stating that appends "are the normal, expected operation". The table is seeded +once and never appended. Every overlay the reasoners produce is explicitly +ephemeral — one module says its overlay "is transient and is discarded after the +merge"; another pins its cycle counter to a constant so two requests are +byte-identical. + +**[I] So the missing thing is not knowledge and not an adapter.** It is a +*write*. The substrate has an outcome-recording seam, it is finished, and it is +not wired. Everything in §D is a proposal for what to write through it. + +--- + +## D. Plasticity candidates, ranked smallest first + +Ranking is by *state bytes and mechanism size*, not by appeal. Each is judged +against the same standard the workspace has already applied and failed things +by: it must beat a bigram successor table at equal state. + +**D1 — Visit-count conductance on a permeability mask.** *State:* one bit per +channel, flipped when a saturating counter crosses a threshold. *Update:* on a +useful outcome, increment channels on the successful path; decrement on +failure. *Readout:* `ternlog(active, permeable, ¬inhibited)`, one instruction. +*Bytes:* channels/8. At 4,096 channels, **512 bytes**. *Ternlog-native:* yes, +by construction. *Locality:* none required. *Biggest falsifier:* a per-channel +frequency count at the same bytes reaches the same recall — i.e. the Q8 outcome +one level up. + +**D2 — Successor table over channel pairs.** *State:* sparse `(from, to) → +count`. *Bytes:* grows with observed pairs; at 4,096 channels the dense bound +is 2 MB and the sparse reality far less. *Ternlog-native:* no — it is a lookup, +not a mask. *Falsifier:* it is the baseline everything else must beat; if it +wins, the answer is "a transition matrix" and the romance is over. **Rank it +first to run, not first to hope for.** + +**D3 — Eligibility trace over the mask.** *State:* D1 plus a decaying trace of +recently active channels, so credit reaches a path rather than an endpoint. +*Bytes:* one small counter per channel; 4,096 channels × u8 = **4 KB**. +*Ternlog-native:* the readout is, the decay is not. *Falsifier:* a trace of +length 1 (i.e. D1) matches it. + +**D4 — Learned inhibition only.** *State:* one *suppression* mask; nothing is +ever excited, only ruled out. *Bytes:* channels/8. *Rationale:* the substrate's +measured strength is Boolean elimination, and `AND2_ANDNOT` makes inhibition +one instruction. *Falsifier:* inhibition alone cannot bring a useful target +*earlier*, only cheaper — which section H1 measures directly. + +**D5 — Per-class conductance keyed by classid.** *State:* one counter per +`classid`, not per node — so state is bounded by the codebook (98 concepts +today), not by the graph. *Bytes:* ~100 counters, **under 1 KB, and constant in +graph size**. *This is the only candidate whose state provably cannot grow with +the substrate*, which is exactly what §J found the vocabulary carrier could not +promise. *Falsifier:* class granularity is too coarse to discriminate useful +from useless targets within a class. + +**D6 — Bloom-shaped "seen-useful" filter.** *State:* one bitset, hashed +membership of channel sets that preceded a good outcome. *Bytes:* tunable. +*Falsifier:* false-positive rate makes precision worse than D1 at equal bytes. + +**D7 — Logistic regression over channel features.** *State:* one weight per +channel, f32. *Bytes:* 16 KB at 4,096 channels. *Ternlog-native:* no — this is +the numeric rail. *Included deliberately* as the conventional baseline §14 of +the brief demands; if it wins decisively the Boolean framing is wrong. + +**D8 — Do nothing plastic; cache the frontier.** *State:* memoized BFS results. +*Rationale:* the honest null for the whole programme. If cached traversal at +equal bytes matches every learner, there is no plasticity finding. + +--- + +## H. Two minimal probes + +### H1 — focus propagation (arm A, smallest honest form) + +*Question:* at an identical expansion budget, does any plastic state reach a +useful target earlier than the shipped controller? + +*Substrate:* the largest edge set that can be fetched — SNOMED at 2,053,329 +edges over 484,036 concepts, bridged to MONDO by 9,161 xref pairs. Note this is +2M, not 10M; the 10M figure is reachable only as transitive closure (4,273,203 +ancestor pairs under `is_a ∪ part_of`), and closure is not the same object. + +*Seeds:* per patient, `labs(id)` plus the grounded anamnese entries; the oracle +is `SoaPatient.icd` resolved through `crosswalk::resolve_icd10`. + +*Arms:* A0 shipped controller (curated enum + `is_a` walk) · A1 candidate from +§D · A2 shuffled A1 · A3 bigram successor · A4 random state at identical bytes · +A5 cached BFS. + +*Gates, all mandatory:* the candidate space must be the open-set 5,750-key axis, +never the fifteen-value enum — the repo's own note calls the latter "15× +hindsight contamination". Activation sparsity is reported beside recall, so a +learner that lights everything is visibly disqualified. State bytes are equal +across arms by construction. + +### H2 — persistence (arm B, the actual plasticity question) + +*Question:* does episode *n+1* get behaviourally easier without the ontology +changing? + +*Mechanism:* wire `append_witness` — the seam that exists and has no caller — +so an episode's outcome lands in `patient_nodes`. Run E1…En updating `P`, freeze, +measure on held-out. + +*Curves:* expansions-to-target vs episode · precision vs episode · **state bytes +vs episode** (the §J question, re-asked on a fixed channel set where it may +answer differently) · recovery after unrelated episodes. + +*Kill:* the curve is flat, or state bytes track episodes linearly. + +**Neither probe is authorized here.** H2 in particular is a *write* into a +private clinical repo and is the operator's call, not an agent's. + +--- + +## I. Surprise criterion, quantitatively + +An event counts as unprogrammed-but-useful when **all four** hold: + +1. A capability becomes active that the shipped controller would **not** have + selected for this seed — verified by running A0 on the same seed and + confirming absence from its expansion set. +2. The activation is **useful**: it lies on the oracle path, or it strictly + reduces expansions-to-target versus A0. +3. It is **attributable to learned state**: the shuffled arm A2, at identical + bytes, does not produce it. This is the clause that makes the criterion + falsifiable rather than decorative. +4. The full causal trace is logged — seed, channel sequence, the plastic entries + consulted, and the outcome. + +Reported as a rate, not an anecdote: *surprises per 100 held-out episodes*, with +its A2 rate beside it. A single spectacular trace is not evidence. + +--- + +## L. Epiphanies + +**[F] The plastic field, the closed loops, and the write seam are all already +built — and none is called.** A threshold-and-accumulate activation substrate +(`MailboxSoA` energy/`consume_firing`), four in-memory closed loops, and an +outcome-recording write (`append_witness` + `patient_nodes`) exist in shipped +source with zero non-test callers between them. The missing thing is not a +mechanism. + +**[F] The one live dispatch path cannot propagate, by one line.** +`dispatch_thought` clones the seed at every rung, so its own wave schedule never +carries wave *n*'s writes into wave *n+1*. Before any learner is designed, this +is the cheapest possible experiment in the entire map. + +**[F] The activation seam is already built and unwired.** `append_witness` plus +`patient_nodes` are complete, documented as the normal path, and have zero +callers. The gap between "has knowledge" and "has behaviour" is currently one +missing write, not a missing subsystem. + +**[M] Zero of fifty-four capabilities are outcome-driven.** Nothing in the +richest consumer becomes easier or harder to reach because of anything that +happened before. + +**[M] Coverage saturates; only the state budget carries signal.** Real and +shuffled soaks reach 98.3% and 97.6%. The real codebook does it with 6.6× less +state. Report budgets, never coverage. + +**[M] Ternlog saves instructions, not bandwidth.** Its advantage is real while +L1-resident and gone by K=32. Chaining is not a way to make more constraints +free; it is a way to make each one cheaper until the masks stop fitting. + +**[M] A bitset's cost is independent of sparsity, and attention is sparse.** The +mask/sparse crossover sits between 0.1% and 0.8% active. This is the strongest +architectural constraint found today: masks belong *within* a resident tier, +sparse indices *across* the substrate. + +**[F] The def-use carrier excludes back-edges by construction.** Loop discovery +was never expressible on the substrate every macro experiment used. + +**[F] Degree-6 over a seven-symbol alphabet is the complete graph minus self.** +The hex experiments were degenerate, not unlucky. + +**[I] The one candidate with a bounded state promise is per-class conductance.** +Everything keyed per node or per vocabulary item grew with the corpus when +measured; state keyed by classid cannot, because the codebook is 98 entries and +minted by hand. + +**[S] Focus as an emergent observable rather than a subsystem.** Untested. The +substrate has no salience, LOD, or label-budget layer — its own UX contract says +"layer 2 does not exist" — so there is currently nothing for focus to emerge +*into*, and that gap is a prerequisite, not a consequence. diff --git a/Cargo.toml b/Cargo.toml index ef7f9d9e..11ba2ddb 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -51,6 +51,14 @@ required-features = ["splat3d"] # AMX examples import `ndarray::simd` / `ndarray::hpc`, both `#[cfg(feature = # "std")]`, so they must be skipped in `--no-default-features` CI jobs. +[[example]] +name = "hex_trie_vs_gemm_probe" +required-features = ["std"] + +[[example]] +name = "ternlog_amortization_probe" +required-features = ["std"] + [[example]] name = "amx_gemm_bench" required-features = ["std"] diff --git a/examples/behavioral_soak_probe.rs b/examples/behavioral_soak_probe.rs new file mode 100644 index 00000000..b122912e --- /dev/null +++ b/examples/behavioral_soak_probe.rs @@ -0,0 +1,450 @@ +//! D-GTM-0m — successive soaking: how much of a NEW corpus is expressible in a +//! behavioural vocabulary already minted from earlier ones? +//! +//! This replaces the address-prefix framing that D-GTM-0l falsified. The claim +//! now under test is not that semantically related things get nearby addresses, +//! but that *machine behaviour converges onto a small reusable basis*: soak the +//! codebook on corpus 1, feed corpus 2 through it without resetting, and measure +//! reuse against new mint. If reuse rises while the codebook grows sublinearly, +//! the substrate is a sponge. If the codebook grows with the corpus, it is a +//! dictionary of one entry per thing seen, which is not a basis. +//! +//! ## The vacuity trap this probe is built around +//! +//! Reuse is trivially ~100% at a coarse enough granularity — there are only a +//! dozen distinct p-code opcodes, so "opcode reuse" saturates on the second +//! basic block of the first corpus and says nothing. So the probe reports a +//! LADDER of granularities and a null, and a result only counts if reuse stays +//! high at a granularity whose vocabulary is still growing. +//! +//! | rung | atom | what it claims | +//! |---|---|---| +//! | G0 | opcode | control — must saturate; proves nothing | +//! | G1 | (kind, opcode) | still near-trivial | +//! | G2 | block opcode sequence | the behavioural BPE token | +//! | G3 | block UNORDERED opcode:in-arity:out-arity profile | dataflow-shape, order-blind | +//! | G4 | function block-token sequence | whole-routine shape | +//! +//! The null shuffles opcodes across blocks while preserving every block's LENGTH +//! and the corpus-wide opcode marginal. If measured reuse at G2/G3 is no better +//! than the null's, the "behavioural basis" is just the opcode frequency +//! distribution reappearing, not structure. +//! +//! Usage: `cargo run --release --example behavioral_soak_probe -- ...` +//! Corpora are soaked in argument order and the codebook is never reset. +//! Ore is read by path and never vendored. + +use std::collections::{BTreeMap, HashMap, HashSet}; + +/// One lifted instruction fact, reduced to what the atoms are built from. +struct Fact { + block: (String, String, String), + id: u64, + kind: String, + opcode: String, + ins: u32, + outs: u32, +} + +/// SplitMix64 — the null's shuffle must be reproducible. +struct SplitMix(u64); +impl SplitMix { + fn next(&mut self) -> u64 { + self.0 = self.0.wrapping_add(0x9E37_79B9_7F4A_7C15); + let mut z = self.0; + z = (z ^ (z >> 30)).wrapping_mul(0xBF58_476D_1CE4_E5B9); + z = (z ^ (z >> 27)).wrapping_mul(0x94D0_49BB_1331_11EB); + z ^ (z >> 31) + } +} + +fn parse(path: &str) -> Vec { + let text = std::fs::read_to_string(path).expect("read ore"); + let mut out = Vec::new(); + for line in text.lines() { + if line.starts_with('#') { + continue; + } + let f: Vec<&str> = line.split('\t').collect(); + if f.len() < 11 { + continue; + } + let Ok(id) = f[2].parse::() else { continue }; + out.push(Fact { + block: (f[0].to_string(), f[1].to_string(), f[10].to_string()), + id, + kind: f[5].to_string(), + opcode: f[6].to_string(), + ins: u32::from(f[5] == "OperandIn"), + outs: u32::from(f[5] == "OperandOut"), + }); + } + out +} + +/// Group facts into blocks, each block a fact list ordered by fact id. +fn blocks(facts: &[Fact]) -> Vec> { + let mut m: BTreeMap<&(String, String, String), Vec<&Fact>> = BTreeMap::new(); + for f in facts { + m.entry(&f.block).or_default().push(f); + } + let mut v: Vec> = m.into_values().collect(); + for b in &mut v { + b.sort_by_key(|f| f.id); + } + v +} + +/// The five rungs of the granularity ladder, as atom occurrence lists. +fn atoms(bs: &[Vec<&Fact>], rung: usize) -> Vec { + match rung { + 0 => bs.iter().flatten().map(|f| f.opcode.clone()).collect(), + 1 => bs + .iter() + .flatten() + .map(|f| format!("{}|{}", f.kind, f.opcode)) + .collect(), + 2 => bs + .iter() + .map(|b| { + b.iter() + .filter(|f| f.kind == "Op") + .map(|f| f.opcode.as_str()) + .collect::>() + .join(",") + }) + .filter(|s| !s.is_empty()) + .collect(), + 3 => bs + .iter() + .map(|b| { + // Per-block dataflow profile: for each opcode, how many values it + // consumed/produced across the block. Deliberately ORDER-BLIND and + // occurrence-merged, so G3 is not strictly finer than G2 — a block + // `load,store,load` and `store,load,load` share one G3 atom. It is + // a different cut (dataflow shape), not a deeper rung of G2. + let mut per: BTreeMap<&str, (u32, u32)> = BTreeMap::new(); + for f in b { + let e = per.entry(f.opcode.as_str()).or_default(); + e.0 += f.ins; + e.1 += f.outs; + } + per.iter() + .map(|(o, (i, u))| format!("{o}:{i}:{u}")) + .collect::>() + .join(",") + }) + .filter(|s| !s.is_empty()) + .collect(), + _ => { + // whole-function shape: the sequence of its blocks' G2 tokens + let mut by_fn: BTreeMap<(&str, &str), Vec> = BTreeMap::new(); + for b in bs { + let tok = b + .iter() + .filter(|f| f.kind == "Op") + .map(|f| f.opcode.as_str()) + .collect::>() + .join(","); + if tok.is_empty() { + continue; + } + by_fn + .entry((b[0].block.0.as_str(), b[0].block.1.as_str())) + .or_default() + .push(tok); + } + by_fn.into_values().map(|v| v.join(";")).collect() + } + } +} + +/// Soak `corpora` in order through one never-reset codebook; report the curve. +fn soak(names: &[String], per_corpus: &[Vec], label: &str) { + println!("\n{label}"); + println!(" corpus | occurrences | reuse% | new atoms | codebook | bits/occ"); + let mut book: HashMap = HashMap::new(); + for (name, occ) in names.iter().zip(per_corpus) { + let before = book.len(); + let mut hits = 0usize; + for a in occ { + if book.contains_key(a) { + hits += 1; + } else { + let n = book.len(); + book.insert(a.clone(), n); + } + } + let minted = book.len() - before; + // cost of transmitting this corpus given the standing codebook: an index + // per occurrence, plus a literal description for each newly minted atom. + let idx_bits = if book.len() > 1 { + (book.len() as f64).log2() + } else { + 0.0 + }; + let new_bits: f64 = occ + .iter() + .collect::>() + .iter() + .filter(|a| book[**a] >= before) + .map(|a| a.len() as f64 * 8.0) + .sum(); + let total = occ.len() as f64 * idx_bits + new_bits; + println!( + " {:<25} | {:11} | {:5.1}% | {:9} | {:8} | {:8.2}", + name, + occ.len(), + 100.0 * hits as f64 / occ.len().max(1) as f64, + minted, + book.len(), + total / occ.len().max(1) as f64 + ); + } +} + +/// Reuse stratified by atom LENGTH — the control that decides whether transfer +/// is real. A one-op block is trivially shared by every architecture ever, so a +/// headline reuse figure dominated by short blocks says nothing. This reports, +/// for the corpora soaked AFTER the first, what fraction of blocks of each +/// length were already in the codebook, and how many blocks that is. +fn stratified(names: &[String], per_corpus: &[Vec], first_n: usize) { + println!("\nG2 reuse stratified by block length (transfer corpora only)"); + let mut book: HashSet = HashSet::new(); + for (i, occ) in per_corpus.iter().enumerate() { + if i >= first_n { + let mut hit: BTreeMap = BTreeMap::new(); + for a in occ { + let len = a.split(',').count(); + let bucket = if len >= 8 { 8 } else { len }; + let e = hit.entry(bucket).or_default(); + e.1 += 1; + if book.contains(a) { + e.0 += 1; + } + } + println!(" {}", names[i]); + for (len, (h, t)) in &hit { + let label = if *len == 8 { "8+".to_string() } else { len.to_string() }; + println!( + " len {:>2} ops | {:5} blocks | {:5.1}% already known", + label, + t, + 100.0 * *h as f64 / *t as f64 + ); + } + } + for a in occ { + book.insert(a.clone()); + } + } +} + +/// The composition test — the one exact-match cannot answer. +/// +/// A long block failing to match a soaked block EXACTLY does not mean it has no +/// reusable structure; it means one-token-per-block is the wrong atom for it. +/// That is what byte-pair encoding exists to fix: cover a long sequence by +/// concatenating short known tokens. So build the codebook from every contiguous +/// opcode n-gram (n = 1..=MAX_TOKEN) seen in the SOAK corpora, then greedily +/// longest-match-tokenize each TRANSFER block against it. +/// +/// The metric cannot be coverage: length-1 tokens make coverage trivially 100%. +/// What matters is (a) ops per token — how much of the sequence a single known +/// token accounts for — and (b) the share of ops covered by tokens of length ≥ 2, +/// which is the share that is genuinely structural rather than per-opcode. +fn composition(label: &str, names: &[String], per_block_ops: &[Vec>], first_n: usize) { + const MAX_TOKEN: usize = 6; + let mut book: HashSet> = HashSet::new(); + for corpus in per_block_ops.iter().take(first_n) { + for b in corpus { + for n in 1..=MAX_TOKEN.min(b.len()) { + for w in b.windows(n) { + book.insert(w.to_vec()); + } + } + } + } + // Alphabet-saturation control. With a small opcode alphabet, an n-gram + // codebook can cover almost any sequence by combinatorics alone, which is + // the boring explanation this workspace has already caught once: a 7-symbol + // alphabet made held-out coverage look tautological, and made a degree-6 + // neighbourhood the complete graph minus self. So report what fraction of + // the POSSIBLE short n-grams the codebook holds; if that is near 1, high + // composition coverage is arithmetic rather than structure. + let alphabet: HashSet<&String> = per_block_ops + .iter() + .take(first_n) + .flatten() + .flatten() + .collect(); + let a = alphabet.len(); + println!("\n{label} — {}-token codebook (n-grams n<={MAX_TOKEN})", book.len()); + print!(" alphabet {a} symbols; codebook holds"); + for n in 1..=MAX_TOKEN { + let possible = (a as f64).powi(n as i32); + let held = book.iter().filter(|t| t.len() == n).count(); + print!(" {n}-gram {:.1}%", 100.0 * held as f64 / possible); + } + println!(" <- near 100% would make coverage arithmetic, not structure"); + println!( + " corpus | blocks | ops | tokens | ops/token | ops in len>=2 tokens | unseen opcodes" + ); + for (i, corpus) in per_block_ops.iter().enumerate() { + if i < first_n { + continue; + } + let (mut ops, mut toks, mut structural, mut unseen) = (0usize, 0usize, 0usize, 0usize); + for b in corpus { + let mut p = 0usize; + while p < b.len() { + let mut best = 0usize; + for n in (1..=MAX_TOKEN.min(b.len() - p)).rev() { + if book.contains(&b[p..p + n]) { + best = n; + break; + } + } + if best == 0 { + // a genuinely novel opcode: no token of any length covers it + unseen += 1; + best = 1; + } else if best >= 2 { + structural += best; + } + toks += 1; + p += best; + } + ops += b.len(); + } + println!( + " {:<25} | {:6} | {:5} | {:6} | {:9.2} | {:19.1}% | {:14}", + names[i].rsplit('/').next().unwrap_or(&names[i]), + corpus.len(), + ops, + toks, + ops as f64 / toks.max(1) as f64, + 100.0 * structural as f64 / ops.max(1) as f64, + unseen + ); + } +} + +fn main() { + let paths: Vec = std::env::args().skip(1).collect(); + if paths.is_empty() { + eprintln!("usage: behavioral_soak_probe ..."); + std::process::exit(2); + } + + let mut names = Vec::new(); + let mut all: Vec> = Vec::new(); + for p in &paths { + let facts = parse(p); + // one ore file may hold several images; soak each image separately so a + // second image inside one file still counts as a second corpus. + let mut by_img: BTreeMap> = BTreeMap::new(); + for f in facts { + by_img.entry(f.block.0.clone()).or_default().push(f); + } + for (img, fs) in by_img { + names.push(img); + all.push(fs); + } + } + println!("corpora soaked in order: {}", names.join(" -> ")); + for (n, f) in names.iter().zip(&all) { + let b = blocks(f); + println!(" {n}: {} facts, {} blocks", f.len(), b.len()); + } + + for rung in 0..5 { + let label = [ + "G0 opcode (control)", + "G1 kind|opcode", + "G2 block opcode sequence", + "G3 block opcode:arity shape", + "G4 function block-token sequence", + ][rung]; + let per: Vec> = all.iter().map(|f| atoms(&blocks(f), rung)).collect(); + soak(&names, &per, label); + } + + // The length control: transfer restricted to blocks long enough to mean + // something. `first_n` is the count of corpora treated as the soak. + let g2: Vec> = all.iter().map(|f| atoms(&blocks(f), 2)).collect(); + let first_n: usize = std::env::var("SOAK_FIRST_N") + .ok() + .and_then(|v| v.parse().ok()) + .unwrap_or(1); + stratified(&names, &g2, first_n); + + let per_block_ops: Vec>> = all + .iter() + .map(|f| { + blocks(f) + .iter() + .map(|b| { + b.iter() + .filter(|x| x.kind == "Op") + .map(|x| x.opcode.clone()) + .collect::>() + }) + .filter(|b: &Vec| !b.is_empty()) + .collect() + }) + .collect(); + composition("BPE-style composition, REAL soak", &names, &per_block_ops, first_n); + + // The null that decides whether composition coverage is structure. Shuffle + // the SOAK corpora's opcodes so every block keeps its length and the corpus + // keeps its opcode marginal exactly, but co-occurrence order is destroyed; + // mint the codebook from that and tokenize the REAL transfer corpora. If + // coverage and ops/token survive, the result was the opcode distribution + // reappearing, not a behavioural basis. + let mut nrng = SplitMix(0x5EED_1234_5678_9ABC); + let mut shuffled = per_block_ops.clone(); + for corpus in shuffled.iter_mut().take(first_n) { + let mut pool: Vec = corpus.iter().flatten().cloned().collect(); + for i in (1..pool.len()).rev() { + pool.swap(i, (nrng.next() % (i as u64 + 1)) as usize); + } + let mut cur = 0usize; + for b in corpus.iter_mut() { + let n = b.len(); + b.clone_from_slice(&pool[cur..cur + n]); + cur += n; + } + } + composition("COMPOSITION NULL, shuffled soak", &names, &shuffled, first_n); + + // Null: preserve every block's length and the corpus-wide opcode marginal, + // destroy which opcodes co-occur in a block. Reported at G2, the rung whose + // whole claim is that co-occurrence is structured. + let mut rng = SplitMix(0x9E37_79B9_7F4A_7C15); + let mut null_per: Vec> = Vec::new(); + for f in &all { + let bs = blocks(f); + let mut pool: Vec = bs + .iter() + .flatten() + .filter(|x| x.kind == "Op") + .map(|x| x.opcode.clone()) + .collect(); + for i in (1..pool.len()).rev() { + pool.swap(i, (rng.next() % (i as u64 + 1)) as usize); + } + let mut cur = 0usize; + let mut toks = Vec::new(); + for b in &bs { + let len = b.iter().filter(|x| x.kind == "Op").count(); + if len == 0 { + continue; + } + toks.push(pool[cur..cur + len].join(",")); + cur += len; + } + null_per.push(toks); + } + soak(&names, &null_per, "G2 NULL — block lengths and opcode marginal kept, co-occurrence destroyed"); +} diff --git a/examples/hex_trie_vs_gemm_probe.rs b/examples/hex_trie_vs_gemm_probe.rs new file mode 100644 index 00000000..33c28655 --- /dev/null +++ b/examples/hex_trie_vs_gemm_probe.rs @@ -0,0 +1,380 @@ +//! W0 probes D-GTM-0g/0h/0i/0j/0k — the mask/trie field against dense GEMM. +//! +//! Plan: `.claude/plans/gemm-ternlog-mask-consolidation-v1.md` §11.7. +//! Hypothesis under test (§11.1 pt 6, operator's own fence): **NOT** "TERNLOGQ +//! replaces GEMM". GEMM is attractive when information is dense; a learned +//! hex/trie field may win when cognition is mostly successive elimination of +//! possibility. The invariant (§11.10): `substrate == mask geometry == +//! projection surface` — expanding a mask into IDs or materializing a neighbour +//! list on the hot path is the loss condition. +//! +//! ## The task, identical for both arms +//! +//! `D` steps of `state = R(state) ∩ constraint_i` over `N` items — one hop +//! through a relation, then a filter. Both arms must return the same survivor +//! set; a mismatch aborts the run (a comparison of two different computations +//! measures nothing). +//! +//! ## Two relation shapes — this is what makes the probe two-sided +//! +//! * `Prefix` — `R(x)` = items sharing `x`'s prefix. The hypothesis's home +//! turf: successors of a SET are the union of touched prefix buckets, and a +//! bucket's member mask is a contiguous run of words, so propagation is a +//! word scan with no ID list and no allocation. +//! * `Random` — an arbitrary sparse relation with the SAME edge count and no +//! prefix structure. The mask arm has nothing to exploit and must OR one +//! successor row per active bit. If the mask arm does not lose here, the +//! probe is rigged. +//! +//! ## Metrics +//! +//! * time/step (0g, 0j) across density (0i) and chain depth (0h — a timing +//! proxy; `perf` is unavailable in this sandbox, so residency is inferred +//! from time/step vs depth, and that limit is stated rather than hidden). +//! * **bytes materialized per step (0k)** — a counting allocator, reported +//! separately for one-time setup (amortized per §11.6) and the hot path. The +//! invariant predicts the mask arm's hot path approaches zero. +//! +//! cargo run --release --example hex_trie_vs_gemm_probe --features std + +use std::alloc::{GlobalAlloc, Layout, System}; +use std::sync::atomic::{AtomicUsize, Ordering}; +use std::time::Instant; + +// ── counting allocator: the 0k instrument ──────────────────────────────────── +static ALLOCED: AtomicUsize = AtomicUsize::new(0); +static COUNTING: AtomicUsize = AtomicUsize::new(0); + +struct Counting; +// SAFETY: a pure pass-through allocator. Every call forwards the caller's own +// `Layout` (and, for `dealloc`, the pointer that `alloc` returned for that +// layout) to `System` unchanged, so `System` upholds the `GlobalAlloc` +// contract on our behalf; the only added work is a relaxed atomic counter +// that never touches the allocation. +unsafe impl GlobalAlloc for Counting { + unsafe fn alloc(&self, l: Layout) -> *mut u8 { + if COUNTING.load(Ordering::Relaxed) == 1 { + ALLOCED.fetch_add(l.size(), Ordering::Relaxed); + } + // SAFETY: `l` is the layout the caller passed, forwarded verbatim. + unsafe { System.alloc(l) } + } + unsafe fn dealloc(&self, p: *mut u8, l: Layout) { + // SAFETY: `p` was returned by `System.alloc(l)` above for this same `l`. + unsafe { System.dealloc(p, l) } + } +} +#[global_allocator] +static A: Counting = Counting; + +fn count_on() { + ALLOCED.store(0, Ordering::Relaxed); + COUNTING.store(1, Ordering::Relaxed); +} +fn count_off() -> usize { + COUNTING.store(0, Ordering::Relaxed); + ALLOCED.load(Ordering::Relaxed) +} + +fn splitmix(s: &mut u64) -> u64 { + *s = s.wrapping_add(0x9E37_79B9_7F4A_7C15); + let mut z = *s; + z = (z ^ (z >> 30)).wrapping_mul(0xBF58_476D_1CE4_E5B9); + z = (z ^ (z >> 27)).wrapping_mul(0x94D0_49BB_1331_11EB); + z ^ (z >> 31) +} + +const N: usize = 4096; +const W: usize = N / 64; // 64 mask words = 512 bytes +const BUCKET: usize = 64; // prefix bucket = 1 word = 64 items + +#[derive(Clone, Copy, PartialEq)] +enum Rel { + Prefix, + Random, +} + +/// Dense 0/1 relation matrix, `N×N` f32 — what the GEMM arm needs materialized. +fn build_matrix_deg(rel: Rel, seed: u64, deg: usize) -> Vec { + let mut m = vec![0.0f32; N * N]; + let mut s = seed; + match rel { + Rel::Prefix => { + // successors of x = x's bucket. Edge count = N * BUCKET. + for i in 0..N { + let b = i / BUCKET; + for j in b * BUCKET..(b + 1) * BUCKET { + m[i * N + j] = 1.0; + } + } + } + Rel::Random => { + // Same NOMINAL degree, no prefix structure. Random column draws + // collide, so the realized edge count is below N*deg; the table + // reports the realized density, counted from the matrix. + for i in 0..N { + for _ in 0..deg { + let j = (splitmix(&mut s) as usize) % N; + m[i * N + j] = 1.0; + } + } + } + } + m +} + +/// Forward masks — `fwd[j]` = the items `j` can activate, i.e. the TRANSPOSE of +/// the relation matrix's rows. +/// +/// The transpose is load-bearing and the probe's own correctness gate is what +/// found it: the GEMM arm computes `{ i : srcs(i) ∩ active ≠ ∅ }`, so the mask +/// arm must union `fwd[j]` over active `j`, never `srcs(i)` over active `i`. +/// Those two agree only for a SYMMETRIC relation — true of bucket membership, +/// false of a random relation, which is exactly where the gate fired. +fn build_fwd_masks(matrix: &[f32]) -> Vec { + let mut t = vec![0u64; N * W]; + for i in 0..N { + for j in 0..N { + if matrix[i * N + j] != 0.0 { + t[j * W + i / 64] |= 1u64 << (i % 64); + } + } + } + t +} + +fn dense_state(mask: &[u64]) -> Vec { + (0..N) + .map(|i| if mask[i / 64] >> (i % 64) & 1 == 1 { 1.0 } else { 0.0 }) + .collect() +} + +/// GEMM arm: one hop = matvec through the dense relation, threshold, filter. +fn step_gemm(matrix: &[f32], state: &mut [f32], scratch: &mut [f32], constraint: &[f32]) { + ndarray::backend::gemm_f32(N, 1, N, 1.0, matrix, N, state, 1, 0.0, scratch, 1); + for i in 0..N { + state[i] = if scratch[i] > 0.0 { constraint[i] } else { 0.0 }; + } +} + +/// Mask arm, `Prefix`: propagation is a word scan — a non-empty bucket becomes +/// full. No ID list, no allocation, no gather. +fn step_mask_prefix(state: &mut [u64], c1: &[u64], c2: &[u64]) { + for w in state.iter_mut() { + if *w != 0 { + *w = u64::MAX; // bucket == one word: the union of its member mask + } + } + // filter: state &= c1 & c2 — one VPTERNLOGQ per 512 bits + ndarray::simd::mask_ternlog_assign::<{ ndarray::simd::ternlog::AND3 }>(state, c1, c2); +} + +/// Mask arm, `Random`: no structure to exploit — OR one successor row per +/// active bit. This is the fallback the invariant calls the loss condition. +fn step_mask_random(state: &mut [u64], fwd: &[u64], out: &mut [u64], c1: &[u64], c2: &[u64]) { + out.fill(0); + for (wi, &w0) in state.iter().enumerate() { + let mut word = w0; + while word != 0 { + let b = word.trailing_zeros() as usize; + word &= word - 1; + let row = (wi * 64 + b) * W; + for k in 0..W { + out[k] |= fwd[row + k]; + } + } + } + state.copy_from_slice(out); + ndarray::simd::mask_ternlog_assign::<{ ndarray::simd::ternlog::AND3 }>(state, c1, c2); +} + +fn popcnt(m: &[u64]) -> u32 { + m.iter().map(|w| w.count_ones()).sum() +} + +/// The GEMM arm's survivor vector as a bitset, so the gate compares SETS. +/// Two different survivor sets of equal size must not pass as agreement. +fn gemm_bits(gs: &[f32]) -> Vec { + let mut m = vec![0u64; W]; + for (i, &v) in gs.iter().enumerate() { + if v > 0.0 { + m[i / 64] |= 1u64 << (i % 64); + } + } + m +} + +fn main() { + println!("hex/trie vs GEMM — N={N}, mask={} B, matrix={} MB\n", W * 8, N * N * 4 / 1_048_576); + + for rel in [Rel::Prefix, Rel::Random] { + let name = if rel == Rel::Prefix { + "PREFIX (structured)" + } else { + "RANDOM (no structure)" + }; + println!("═══ relation: {name} ═══"); + + count_on(); + let matrix = build_matrix_deg(rel, 0xC0FFEE, BUCKET); + let gemm_setup = count_off(); + count_on(); + let succ = build_fwd_masks(&matrix); + let mask_setup = count_off(); + + println!(" setup bytes (amortized, §11.6): GEMM {:>10} MASK {:>10}", gemm_setup, mask_setup); + println!( + " {:>6} {:>7} {:>11} {:>11} {:>9} {:>13} {:>13}", + "dens%", "depth", "gemm ns/st", "mask ns/st", "speedup", "gemm B/step", "mask B/step" + ); + + for &dens in &[1usize, 5, 10, 25, 50, 75, 90, 99] { + for &depth in &[1usize, 8, 32] { + // initial state at the requested density + let mut s = 0x5EEDu64 ^ (dens as u64) << 8; + let mut m0 = vec![0u64; W]; + for i in 0..N { + if (splitmix(&mut s) % 100) < dens as u64 { + m0[i / 64] |= 1u64 << (i % 64); + } + } + // constraints: two learned permeability masks per step + let mut c1 = vec![0u64; W]; + let mut c2 = vec![0u64; W]; + for w in 0..W { + c1[w] = splitmix(&mut s) | splitmix(&mut s); + c2[w] = splitmix(&mut s) | splitmix(&mut s); + } + let (c1f, c2f) = (dense_state(&c1), dense_state(&c2)); + let dense_m0 = dense_state(&m0); + + // ── GEMM arm ── (repeat to a 50 ms floor; a sub-resolution + // timing divided into a ratio is noise, not a speedup) + let mut gs = dense_state(&m0); + let mut scratch = vec![0.0f32; N]; + let mut reps = 0usize; + count_on(); + let t = Instant::now(); + while t.elapsed().as_secs_f64() < 0.05 { + gs.copy_from_slice(&dense_m0); + for _ in 0..depth { + step_gemm(&matrix, &mut gs, &mut scratch, &c1f); + for i in 0..N { + gs[i] *= c2f[i]; + } + } + reps += 1; + } + let el = t.elapsed().as_secs_f64(); + let gemm_b = count_off() / reps.max(1); + let gemm_ns = el * 1e9 / (reps * depth) as f64; + std::hint::black_box(&gs); + + // ── MASK arm ── + let mut ms_ = m0.clone(); + let mut out = vec![0u64; W]; + let mut mreps = 0usize; + count_on(); + let t = Instant::now(); + while t.elapsed().as_secs_f64() < 0.05 { + ms_.copy_from_slice(&m0); + for _ in 0..depth { + match rel { + Rel::Prefix => step_mask_prefix(&mut ms_, &c1, &c2), + Rel::Random => step_mask_random(&mut ms_, &succ, &mut out, &c1, &c2), + } + } + mreps += 1; + } + let el = t.elapsed().as_secs_f64(); + let mask_b = count_off() / mreps.max(1); + let mask_ns = el * 1e9 / (mreps * depth) as f64; + std::hint::black_box(&ms_); + + // ── correctness gate: same survivors, or the numbers mean nothing ── + let gemm_set = gemm_bits(&gs); + assert!( + gemm_set == ms_, + "ARMS DISAGREE at dens={dens} depth={depth} ({name}): gemm {} vs mask {} survivors (sets differ)", + popcnt(&gemm_set), + popcnt(&ms_) + ); + + println!( + " {:>6} {:>7} {:>11.3} {:>11.3} {:>8.1}x {:>13} {:>13}", + dens, + depth, + gemm_ns, + mask_ns, + gemm_ns / mask_ns.max(1e-9), + gemm_b, + mask_b + ); + } + } + println!(); + } + + // ── D-GTM-0i, the axis the first table missed: RELATION density ────────── + // State density fixed at 50%, depth 8, RANDOM relation (no structure to + // exploit), sweeping edges-per-row from 1 to N. At deg = N the relation is + // fully dense — the regime §11.1 pt 6 concedes to GEMM. + println!("═══ D-GTM-0i: RELATION density sweep (RANDOM, state 50%, depth 8) ═══"); + println!(" {:>8} {:>9} {:>12} {:>12} {:>9}", "deg", "rel dens%", "gemm ns/st", "mask ns/st", "speedup"); + for ° in &[1usize, 16, 64, 256, 1024, 4096] { + let matrix = build_matrix_deg(Rel::Random, 0xC0FFEE, deg); + let fwd = build_fwd_masks(&matrix); + let mut s = 0xBEEFu64; + let mut m0 = vec![0u64; W]; + for i in 0..N { + if splitmix(&mut s) % 100 < 50 { + m0[i / 64] |= 1u64 << (i % 64); + } + } + let (mut c1, mut c2) = (vec![0u64; W], vec![0u64; W]); + for w in 0..W { + c1[w] = splitmix(&mut s) | splitmix(&mut s); + c2[w] = splitmix(&mut s) | splitmix(&mut s); + } + let (c1f, c2f) = (dense_state(&c1), dense_state(&c2)); + let dense_m0 = dense_state(&m0); + + let mut gs = dense_m0.clone(); + let mut scratch = vec![0.0f32; N]; + let (mut r, t) = (0usize, Instant::now()); + while t.elapsed().as_secs_f64() < 0.05 { + gs.copy_from_slice(&dense_m0); + for _ in 0..8 { + step_gemm(&matrix, &mut gs, &mut scratch, &c1f); + for i in 0..N { + gs[i] *= c2f[i]; + } + } + r += 1; + } + let gemm_ns = t.elapsed().as_secs_f64() * 1e9 / (r * 8) as f64; + + let mut ms_ = m0.clone(); + let mut out = vec![0u64; W]; + let (mut mr, t) = (0usize, Instant::now()); + while t.elapsed().as_secs_f64() < 0.05 { + ms_.copy_from_slice(&m0); + for _ in 0..8 { + step_mask_random(&mut ms_, &fwd, &mut out, &c1, &c2); + } + mr += 1; + } + let mask_ns = t.elapsed().as_secs_f64() * 1e9 / (mr * 8) as f64; + + let gemm_set = gemm_bits(&gs); + assert!(gemm_set == ms_, "ARMS DISAGREE at deg={deg}: survivor sets differ"); + println!( + " {:>8} {:>8.2}% {:>12.0} {:>12.0} {:>8.1}x", + deg, + 100.0 * matrix.iter().filter(|&&v| v != 0.0).count() as f64 / (N * N) as f64, + gemm_ns, + mask_ns, + gemm_ns / mask_ns.max(1e-9) + ); + } +} diff --git a/examples/prefix_tract_coverage_probe.rs b/examples/prefix_tract_coverage_probe.rs new file mode 100644 index 00000000..bf215b3e --- /dev/null +++ b/examples/prefix_tract_coverage_probe.rs @@ -0,0 +1,160 @@ +//! D-GTM-0l — does packed-prefix routing express real long-range transitions? +//! +//! The hypothesis under test (operator, grey/white-matter framing): *"white matter +//! should not be another data structure. It should be an interpretation of packed +//! location prefixes."* Concretely: a tract is `(prefix, mask, learned transition)`, +//! and a transition is routed by `ADDRESS & PREFIX_MASK == PREFIX`. That is only a +//! routing mechanism if the tract codebook is much smaller than the edge set it +//! covers. If one tract is needed per edge, the "interpretation" IS an edge list. +//! +//! Falsifier: measure, on a real long-range relation set, (a) whether edges are more +//! prefix-local than a degree-preserving null, (b) how far the tract codebook +//! compresses the edge set at each prefix width, and (c) whether the non-local +//! residual is concentrated (a small exception table keeps the mechanism) or diffuse +//! (it does not). +//! +//! Input is passed by path and never vendored: the corpus this was measured on is a +//! disassembly harvest that its own provenance file forbids redistributing. The probe +//! reads derived facts only, and the repository stores the measurement, not the ore. +//! +//! Usage: `cargo run --release --example prefix_tract_coverage_probe -- ` +//! +//! Ore schema (tab-separated, `#`-prefixed header lines skipped): field 1 = image, +//! field 4 = the 128-bit packed address as 32 hex nibbles, field 6 = fact kind, +//! field 8 = the call target as a decimal address. `CallSite` rows are the edges. + +use std::collections::{BTreeMap, BTreeSet}; + +/// Address nibbles 9..12 of the packed key hold the 16-bit program counter, +/// little-endian byte order — `..361d..` is `0x1d36`. +fn unpack_pc(hex: &str) -> Option { + if hex.len() < 12 { + return None; + } + let lo = u32::from_str_radix(&hex[8..10], 16).ok()?; + let hi = u32::from_str_radix(&hex[10..12], 16).ok()?; + Some((hi << 8) | lo) +} + +/// The leading `k` nibbles of a 16-bit address — the `PREFIX_MASK` reading. +fn prefix(addr: u32, k: u32) -> u32 { + addr >> (4 * (4 - k)) +} + +/// Longest shared prefix, in nibbles, of two addresses. +fn shared(a: u32, b: u32) -> u32 { + (0..=4) + .rev() + .find(|&k| prefix(a, k) == prefix(b, k)) + .unwrap_or(0) +} + +/// SplitMix64 — deterministic shuffling so the null is reproducible. +struct SplitMix(u64); +impl SplitMix { + fn next(&mut self) -> u64 { + self.0 = self.0.wrapping_add(0x9E37_79B9_7F4A_7C15); + let mut z = self.0; + z = (z ^ (z >> 30)).wrapping_mul(0xBF58_476D_1CE4_E5B9); + z = (z ^ (z >> 27)).wrapping_mul(0x94D0_49BB_1331_11EB); + z ^ (z >> 31) + } +} + +fn main() { + let path = match std::env::args().nth(1) { + Some(p) => p, + None => { + eprintln!("usage: prefix_tract_coverage_probe "); + std::process::exit(2); + } + }; + let text = std::fs::read_to_string(&path).expect("read ore"); + + let mut edges: Vec<(String, u32, u32)> = Vec::new(); + for line in text.lines() { + if line.starts_with('#') { + continue; + } + let f: Vec<&str> = line.split('\t').collect(); + if f.len() < 9 || f[5] != "CallSite" { + continue; + } + let (Some(src), Ok(dst)) = (unpack_pc(f[3]), f[7].parse::()) else { + continue; + }; + edges.push((f[0].to_string(), src, dst)); + } + let n = edges.len(); + assert!(n > 0, "no CallSite edges parsed — schema drift?"); + println!("long-range edges: {n}"); + + // (a) locality vs a degree-preserving null: same sources, destinations reshuffled. + let mut real = [0usize; 5]; + for (_, s, d) in &edges { + real[shared(*s, *d) as usize] += 1; + } + let mut null = [0usize; 5]; + let mut dsts: Vec = edges.iter().map(|e| e.2).collect(); + let mut rng = SplitMix(0x9E37_79B9_7F4A_7C15); + const ROUNDS: usize = 50; + for _ in 0..ROUNDS { + for i in (1..dsts.len()).rev() { + dsts.swap(i, (rng.next() % (i as u64 + 1)) as usize); + } + for (e, d) in edges.iter().zip(&dsts) { + null[shared(e.1, *d) as usize] += 1; + } + } + println!("\nshared prefix | real | null (k nibbles; higher = more local)"); + for k in 0..5 { + println!( + " k={k} | {:6.2}% | {:6.2}%", + 100.0 * real[k] as f64 / n as f64, + 100.0 * null[k] as f64 / (ROUNDS * n) as f64 + ); + } + + // (b) does the tract codebook actually compress the edge set? + let uniq: BTreeSet<_> = edges.iter().cloned().collect(); + println!("\nk | tracts | edges | compression"); + for k in 1..=4 { + let tracts: BTreeSet<_> = edges + .iter() + .map(|(b, s, d)| (b.clone(), prefix(*s, k), prefix(*d, k))) + .collect(); + println!("{k} | {:6} | {:5} | {:8.2}x", tracts.len(), uniq.len(), uniq.len() as f64 / tracts.len() as f64); + } + + // (c) is the non-local residual a small exception table, or diffuse? + let far: Vec<_> = edges + .iter() + .filter(|(_, s, d)| shared(*s, *d) == 0) + .collect(); + let mut hist: BTreeMap = BTreeMap::new(); + for (_, _, d) in &far { + *hist.entry(*d).or_default() += 1; + } + let tot = far.len() as f64; + let mut counts: Vec = hist.values().copied().collect(); + counts.sort_unstable_by(|a, b| b.cmp(a)); + let entropy: f64 = counts + .iter() + .map(|&c| { + let p = c as f64 / tot; + -p * p.log2() + }) + .sum(); + println!("\nnon-local edges {} over {} distinct targets", far.len(), hist.len()); + let mut cum = 0usize; + for (i, c) in counts.iter().enumerate() { + cum += c; + if matches!(i + 1, 1 | 5 | 10 | 20 | 40) { + println!(" top {:3} targets cover {:5.1}%", i + 1, 100.0 * cum as f64 / tot); + } + } + println!( + " target entropy {entropy:.2} bits vs {:.2} uniform — a small hub set would sit far below", + (hist.len() as f64).log2() + ); +} diff --git a/examples/ternlog_amortization_probe.rs b/examples/ternlog_amortization_probe.rs new file mode 100644 index 00000000..dc2f1589 --- /dev/null +++ b/examples/ternlog_amortization_probe.rs @@ -0,0 +1,367 @@ +//! D-GTM-0n — is a *further* learned constraint cheap once the representation +//! is resident, and where does residency actually break? +//! +//! The architectural claim under test is NOT that ternary Boolean logic is +//! intelligent. It is narrower and mechanical: if learned state and active +//! state are the same shape, then stacking one more constraint should cost +//! roughly *one more resident load plus one instruction*, rather than another +//! pass of graph materialization. That is a claim about an amortization curve, +//! so this probe measures the curve rather than a headline ratio. +//! +//! Four arms compute the IDENTICAL logical result — `A ∧ M₁ ∧ … ∧ M_K` — and a +//! correctness gate aborts the run unless every arm agrees, both on the +//! survivor count and on the surviving set itself. +//! +//! | arm | representation | work per constraint | +//! |---|---|---| +//! | T0 | materialized `Vec` id list | rebuild the candidate list | +//! | T1 | bitset, `mask_and_assign` | one pass over the mask | +//! | T3 | bitset, `mask_ternlog_assign::` | one pass per TWO constraints | +//! | T5 | sorted index intersection | merge over the sparse sets | +//! +//! T3 is the arm the claim is about: `VPTERNLOGQ` folds two constraints into +//! one instruction, so if the cost model is "a pass over resident bytes", T3 +//! should sit near half of T1 and the per-constraint cost should be FLAT in K. +//! Flat is the honest success criterion — a per-constraint cost that *falls* +//! with K would mean something else is happening (and would need explaining, +//! not celebrating). +//! +//! Residency is derived, not asserted: the probe reports achieved mask-traffic +//! bandwidth, and the knee where GB/s collapses is where the working set left a +//! cache level. Reading residency off "pretty nanosecond numbers" is exactly +//! what D-GTM-0h was graded [S] for, so the bandwidth column is the evidence +//! and the ns column is not. +//! +//! Usage: `cargo run --release --example ternlog_amortization_probe` + +use std::alloc::{GlobalAlloc, Layout, System}; +use std::sync::atomic::{AtomicUsize, Ordering}; +use std::time::Instant; + +use ndarray::simd::ternlog::AND3; +use ndarray::simd::{mask_and_assign, mask_ternlog_assign}; + +/// Counting allocator — "materialized 0 bytes" is a measurement here, not a claim. +struct Counting; +static ALLOCATED: AtomicUsize = AtomicUsize::new(0); +// SAFETY: a pure pass-through allocator. Every call forwards the caller's own +// `Layout` (and, for `dealloc`, the pointer that `alloc` returned for that +// layout) to `System` unchanged, so `System` upholds the `GlobalAlloc` +// contract on our behalf; the only added work is a relaxed atomic counter. +unsafe impl GlobalAlloc for Counting { + unsafe fn alloc(&self, l: Layout) -> *mut u8 { + ALLOCATED.fetch_add(l.size(), Ordering::Relaxed); + // SAFETY: `l` is the layout the caller passed, forwarded verbatim. + unsafe { System.alloc(l) } + } + unsafe fn dealloc(&self, p: *mut u8, l: Layout) { + // SAFETY: `p` was returned by `System.alloc(l)` above for this same `l`. + unsafe { System.dealloc(p, l) } + } +} +#[global_allocator] +static A: Counting = Counting; + +struct SplitMix(u64); +impl SplitMix { + fn next(&mut self) -> u64 { + self.0 = self.0.wrapping_add(0x9E37_79B9_7F4A_7C15); + let mut z = self.0; + z = (z ^ (z >> 30)).wrapping_mul(0xBF58_476D_1CE4_E5B9); + z = (z ^ (z >> 27)).wrapping_mul(0x94D0_49BB_1331_11EB); + z ^ (z >> 31) + } +} + +/// Bits set in a mask, as the arms' shared correctness currency. +fn popcnt(m: &[u64]) -> u32 { + m.iter().map(|w| w.count_ones()).sum() +} + +fn to_ids(m: &[u64]) -> Vec { + let mut v = Vec::new(); + for (w, word) in m.iter().enumerate() { + let mut b = *word; + while b != 0 { + v.push((w * 64 + b.trailing_zeros() as usize) as u32); + b &= b - 1; + } + } + v +} + +/// Run one (N, K) cell across all four arms. Returns per-arm ns/constraint. +fn cell(n_bits: usize, k: usize, seed: u64) -> [f64; 4] { + let words = n_bits / 64; + let mut rng = SplitMix(seed); + + // Constraint masks are dense enough that survivors persist to K=64: each + // keeps ~97% of bits, so the intersection stays non-empty and every arm + // does real work at every depth. A sparser mask would empty the set after + // a few constraints and measure nothing but early exit. + let masks: Vec> = (0..k) + .map(|_| (0..words).map(|_| !(1u64 << (rng.next() % 64))).collect()) + .collect(); + let base: Vec = (0..words).map(|_| rng.next() | rng.next()).collect(); + + // ---- correctness gate, before any timing ---- + let mut t1 = base.clone(); + for m in &masks { + mask_and_assign(&mut t1, m); + } + let mut t3 = base.clone(); + let mut i = 0; + while i + 1 < k { + mask_ternlog_assign::(&mut t3, &masks[i], &masks[i + 1]); + i += 2; + } + if i < k { + mask_and_assign(&mut t3, &masks[i]); + } + let gold = to_ids(&t1); + assert_eq!(gold, to_ids(&t3), "T3 disagrees with T1 at N={n_bits} K={k}"); + + // sparse arms share the same logical sets, expressed as sorted id lists + let sparse: Vec> = masks.iter().map(|m| to_ids(m)).collect(); + let base_ids = to_ids(&base); + let mut t5 = base_ids.clone(); + for s in &sparse { + let mut out = Vec::with_capacity(t5.len()); + let (mut a, mut b) = (0usize, 0usize); + while a < t5.len() && b < s.len() { + match t5[a].cmp(&s[b]) { + std::cmp::Ordering::Equal => { + out.push(t5[a]); + a += 1; + b += 1; + } + std::cmp::Ordering::Less => a += 1, + std::cmp::Ordering::Greater => b += 1, + } + } + t5 = out; + } + assert_eq!(gold, t5, "T5 disagrees with T1 at N={n_bits} K={k}"); + + // ---- timing: each arm runs to a 60 ms floor, well past timer resolution ---- + const FLOOR: f64 = 0.060; + let mut out = [0.0f64; 4]; + + // T1 — one pass per constraint + let mut scratch = base.clone(); + let (mut iters, t0) = (0u64, Instant::now()); + while t0.elapsed().as_secs_f64() < FLOOR { + scratch.copy_from_slice(&base); + for m in &masks { + mask_and_assign(&mut scratch, m); + } + iters += 1; + } + out[1] = t0.elapsed().as_secs_f64() * 1e9 / (iters as f64 * k as f64); + + // T3 — one pass per TWO constraints + let (mut iters3, t3c) = (0u64, Instant::now()); + while t3c.elapsed().as_secs_f64() < FLOOR { + scratch.copy_from_slice(&base); + let mut i = 0; + while i + 1 < k { + mask_ternlog_assign::(&mut scratch, &masks[i], &masks[i + 1]); + i += 2; + } + if i < k { + mask_and_assign(&mut scratch, &masks[i]); + } + iters3 += 1; + } + out[2] = t3c.elapsed().as_secs_f64() * 1e9 / (iters3 as f64 * k as f64); + + // T5 — sorted-index intersection (the honest sparse arm) + let (mut iters5, t5c) = (0u64, Instant::now()); + while t5c.elapsed().as_secs_f64() < FLOOR { + let mut cur = base_ids.clone(); + for s in &sparse { + let mut o = Vec::with_capacity(cur.len()); + let (mut a, mut b) = (0usize, 0usize); + while a < cur.len() && b < s.len() { + match cur[a].cmp(&s[b]) { + std::cmp::Ordering::Equal => { + o.push(cur[a]); + a += 1; + b += 1; + } + std::cmp::Ordering::Less => a += 1, + std::cmp::Ordering::Greater => b += 1, + } + } + cur = o; + } + std::hint::black_box(&cur); + iters5 += 1; + } + out[3] = t5c.elapsed().as_secs_f64() * 1e9 / (iters5 as f64 * k as f64); + + // T0 — materialize the surviving id list after every constraint + let (mut iters0, t0c) = (0u64, Instant::now()); + while t0c.elapsed().as_secs_f64() < FLOOR { + let mut cur = base.clone(); + for m in &masks { + mask_and_assign(&mut cur, m); + std::hint::black_box(to_ids(&cur)); + } + iters0 += 1; + } + out[0] = t0c.elapsed().as_secs_f64() * 1e9 / (iters0 as f64 * k as f64); + + out +} + +fn main() { + println!("D-GTM-0n — ternlog amortization + residency boundary"); + println!("host L1d 48 KiB/core, L2 2 MiB/core, L3 260 MiB shared\n"); + + // Depth sweep at a fixed L1-resident mask, isolating K from working-set size. + let n = 1 << 15; // 32768 bits = 4 KiB per mask + println!("A. DEPTH SWEEP — mask 4 KiB, working set = (K+1)x4 KiB"); + println!(" K | working set | T0 mat'd | T1 and | T3 ternlog | T5 sparse | T3/T1 | T3 GB/s"); + for k in [1usize, 2, 4, 8, 16, 32, 64] { + let r = cell(n, k, 0x1234 + k as u64); + let ws = (k + 1) * n / 8; + // each constraint moves one mask in and the accumulator through + let gbs = (n as f64 / 8.0 * 2.0) / r[2]; + println!( + " {k:2} | {:8} B | {:8.1} | {:6.1} | {:10.1} | {:9.1} | {:5.2} | {:7.1}", + ws, + r[0], + r[1], + r[2], + r[3], + r[2] / r[1], + gbs + ); + } + + // Working-set sweep at fixed depth: the knee locates the cache boundary. + println!("\nB. RESIDENCY SWEEP — K=8 constraints, mask size grows"); + println!(" mask | working set | T1 and | T3 ternlog | T3/T1 | T3 GB/s | level"); + for lg in [12usize, 14, 16, 18, 20, 22, 24, 26] { + let nb = 1usize << lg; + let mask_b = nb / 8; + let ws = 9 * mask_b; + let r = cell(nb, 8, 0x99 + lg as u64); + let gbs = (mask_b as f64 * 2.0) / r[2]; + let level = if ws <= 48 * 1024 { + "L1" + } else if ws <= 2 * 1024 * 1024 { + "L2" + } else if ws <= 260 * 1024 * 1024 { + "L3" + } else { + "DRAM" + }; + println!( + " {:6} B | {:9} B | {:6.1} | {:10.1} | {:5.2} | {:7.1} | {level}", + mask_b, + ws, + r[1], + r[2], + r[2] / r[1], + gbs + ); + } + + // D. DENSITY SWEEP — the arm that decides whether masks are the right + // currency at all. A focus field over a huge substrate is SPARSE, and a + // bitset pays for every bit whether set or not, while an id list pays only + // for survivors. Sections A and B ran at ~97% density, which is a + // mis-specified baseline for the sparse arms in exactly the way a dense f32 + // matrix was a mis-specified baseline for a Boolean relation in D-GTM-0j — + // so their T0/T5 columns are void as evidence and this sweep replaces them. + println!("\nD. DENSITY SWEEP — N=2^20 bits (128 KiB mask), K=8 constraints"); + println!(" active | survivors | T1 and | T3 ternlog | T5 sparse | winner"); + let nb = 1usize << 20; + let words = nb / 64; + for (label, keep_shift) in [("100%", 0u32), ("50%", 1), ("6%", 4), ("0.8%", 7), ("0.1%", 10), ("0.012%", 13)] { + let mut rng = SplitMix(0xDEAD + keep_shift as u64); + // base: keep roughly 1 bit in 2^keep_shift + let base: Vec = (0..words) + .map(|_| { + let mut w = u64::MAX; + for _ in 0..keep_shift { + w &= rng.next(); + } + w + }) + .collect(); + // constraints stay near-total so survivors track the base density + let masks: Vec> = (0..8) + .map(|_| (0..words).map(|_| !(1u64 << (rng.next() % 64))).collect()) + .collect(); + + let mut t1v = base.clone(); + for m in &masks { + mask_and_assign(&mut t1v, m); + } + let survivors = popcnt(&t1v); + let base_ids = to_ids(&base); + + const FLOOR: f64 = 0.060; + let mut scratch = base.clone(); + let (mut i1, c1) = (0u64, Instant::now()); + while c1.elapsed().as_secs_f64() < FLOOR { + scratch.copy_from_slice(&base); + for m in &masks { + mask_and_assign(&mut scratch, m); + } + i1 += 1; + } + let ns1 = c1.elapsed().as_secs_f64() * 1e9 / (i1 as f64 * 8.0); + + let (mut i3, c3) = (0u64, Instant::now()); + while c3.elapsed().as_secs_f64() < FLOOR { + scratch.copy_from_slice(&base); + let mut i = 0; + while i + 1 < 8 { + mask_ternlog_assign::(&mut scratch, &masks[i], &masks[i + 1]); + i += 2; + } + i3 += 1; + } + let ns3 = c3.elapsed().as_secs_f64() * 1e9 / (i3 as f64 * 8.0); + + // The sparse arm gets its BEST case: it walks only survivors, and the + // constraint sets are checked by direct bit test rather than by merging + // two long lists — a list merge against a near-total set is the sparse + // representation's worst case and would not be how anyone builds this. + let (mut i5, c5) = (0u64, Instant::now()); + while c5.elapsed().as_secs_f64() < FLOOR { + let mut cur = base_ids.clone(); + for m in &masks { + cur.retain(|&id| m[id as usize / 64] >> (id % 64) & 1 == 1); + } + std::hint::black_box(&cur); + i5 += 1; + } + let ns5 = c5.elapsed().as_secs_f64() * 1e9 / (i5 as f64 * 8.0); + + let winner = if ns5 < ns3 { "sparse" } else { "mask" }; + println!(" {:8} | {:9} | {:6.0} | {:10.0} | {:9.0} | {winner}", label, survivors, ns1, ns3, ns5); + } + + // Materialization: the claim is that the Boolean arms never build a set. + let before = ALLOCATED.load(Ordering::Relaxed); + let words = (1usize << 16) / 64; + let mut acc: Vec = vec![u64::MAX; words]; + let m1: Vec = vec![0xF0F0_F0F0_F0F0_F0F0; words]; + let m2: Vec = vec![0xFF00_FF00_FF00_FF00; words]; + let setup = ALLOCATED.load(Ordering::Relaxed); + for _ in 0..1000 { + mask_ternlog_assign::(&mut acc, &m1, &m2); + } + let after = ALLOCATED.load(Ordering::Relaxed); + println!( + "\nC. MATERIALIZATION — setup {} B, then {} B over 1000 chained steps (popcount {})", + setup - before, + after - setup, + popcnt(&acc) + ); +}