Named u8↔i8 bit reinterpret; Codec9 runtime scale and batch encode

.claude/knowledge/dr512-epistemic-pm-invariant.md

@@ -58,6 +58,20 @@ enums with 512 arbitrary tokens are different from DR512 signed numeric
 codes: they must retain their own typed IDs and source mapping. Never
 reinterpret `bgz-tensor::WeightPalette` u8 archetypes as signed magnitudes.
 
+**Runtime scale (operator, 2026-10-30).** The goal is both exact
+`LUT[a, b]` work and learning. `Codec9Runtime<R>` takes the scale as a
+value, checked once (positive, finite), and calls the same encoder and
+decoder as `Codec9<R, C>`: for equal scales the codewords are identical
+(tested over a 6,000-value sweep plus every codeword). Rounding stays a type
+parameter restricted to nearest; storage, polarity, Sym and NaN stay fixed.
+Use `Codec9` where a table over codewords must stay valid for everything
+encoded with it; use `Codec9Runtime` where the scale is learned or
+calibrated. Its `encode_touched_batch` / `decode_batch` read and write the
+magnitude lane plus the presence plane, refuse a non-finite value before
+writing anything, and clear presence bits past the last lane. A table built
+under one scale is invalid under another: a consumer that moves its scale
+rebuilds its tables. The batch loops are scalar today (not measured).
+
 ## Cross-repo ownership and CI tripwires
 
 1. **ndarray:** `src/hpc/codec9.rs`, `src/simd_stride.rs`,

.claude/knowledge/synergy-map-2026-10-29.md

@@ -224,6 +224,18 @@ table (`:301-310`), fruit list (`:409-421`) and `perm.rs:255-290` were read
 directly on the main thread, which was cheaper than briefing a card. The
 facet-stride count was re-run (166).
 
+## R4. Two rulings implemented (operator, 2026-10-30)
+
+| ruling | implemented as | closes / changes |
+|---|---|---|
+| u8→i8 only "as in explicit calling on intentional explicit parameter" | `ReinterpretBits` (zero-sized marker) and `U8x64::reinterpret_as_i8(ReinterpretBits)` / `I8x64::reinterpret_as_u8(ReinterpretBits)` in `src/simd.rs`; a bit reinterpret, no conversion; FACET STRIDE rule 3 amended in `CLAUDE.md` | V10/N14 decided. U6's facade gap is closed; the probe's `kernels::bits_as_i8` stand-in is not migrated (its timings stay reproducible). The 2 existing R3 hits are `u8` splats cast to an `i8` intrinsic parameter, not lane data, so they are not covered by this word and stay on the burn-down list |
+| Codec9 runtime scale + batch encoder, "to enable both extreme accuracy LUT[a,b] and learning" | `Codec9Runtime<R>` in `src/hpc/codec9.rs`: checked scale, same encoder/decoder as `Codec9<R, C>` (equal scales give identical codewords), `encode_touched_batch` / `decode_batch` over magnitudes + presence plane; DR512 law doc amended | V1 and N4 decided. Compile-time `Codec9` stays the form for codeword tables; `Codec9Runtime` is the form for learned scales. Open: the batch loops are scalar (a SIMD batch is the next step once a consumer is timed); NSL has no ndarray dependency, so it either takes one or keeps its mirror behind a pairing test against these codewords (L3); lance-graph `scent_learn.rs:345-356` can replace its hand-built presence loop with `encode_touched_batch` (N5) |
+
+Tests: 8 new in `codec9` (12 total), 4 in `simd::reinterpret_bits_parity`;
+5 disable runs, each red (scale ignored, tail bits not cleared, write before
+the finiteness check, saturating reinterpret, scale check removed). Lib suite
+2,676 passed; AVX2 tier, aarch64 build and the wasm/NEON parity programs pass.
+
 ## 0. What the six ndarray PRs did
 
 | PR | closed | opened or left |

CLAUDE.md

@@ -38,6 +38,11 @@ with no shuffle anywhere.
 3. **u8 has no sign. Never mix u8 with i8.** No `as i8` on unsigned data in
    SIMD code, no signed compare on unsigned bytes, no u8 <-> i8 conversion
    lines. Use the unsigned type end to end.
+   The one sanctioned crossing (operator, 2026-10-30) is a bit reinterpret
+   with an explicit intent parameter: `U8x64::reinterpret_as_i8(ReinterpretBits)`
+   / `I8x64::reinterpret_as_u8(ReinterpretBits)`. Nothing is converted, the
+   marker must be written at the call site, and arithmetic afterwards follows
+   the new type. A plain `as i8` stays forbidden.
 
 **Guard:** `.claude/hooks/facet-stride-guard.sh` runs after every Edit/Write
 (wired in `.claude/settings.json`) and checks the lines the change adds:

src/hpc/codec9.rs

@@ -17,8 +17,24 @@
 //!
 //! Storage, polarity, Sym and NaN policy are intentionally NOT configurable.
 //! An Asym mode, a separate sign/validity plane, or an optional NaN policy
-//! would violate this contract. The only compile-time knobs are the existing
-//! nearest rounding mode and scale calibration.
+//! would violate this contract. The only knobs are the nearest rounding mode
+//! (always a type parameter) and the scale.
+//!
+//! The scale comes in two forms that share one encoder, so equal scales give
+//! byte-identical codewords:
+//!
+//! - [`Codec9`]: the scale is a compile-time [`Calibration`]. Use it where a
+//!   table over codewords (`lut9`, a `LUT[a, b]`) must stay valid for every
+//!   value ever encoded with it.
+//! - [`Codec9Runtime`]: the scale is a value checked once at construction.
+//!   Use it where the scale is learned or calibrated from data, for example a
+//!   training loop that re-encodes features as its scale moves. It adds the
+//!   batch forms [`Codec9Runtime::encode_touched_batch`] and
+//!   [`Codec9Runtime::decode_batch`], which write and read the magnitude
+//!   lane plus the one-bit presence plane (64 lanes per `u64`).
+//!
+//! A table built under one scale is not valid under another; a consumer that
+//! changes the scale rebuilds its tables.
 //!
 //! ```
 //! use ndarray::hpc::codec8::{Calibration, TiesEven};
@@ -113,6 +129,33 @@ impl EpiLane9 {
 pub enum Codec9Error {
     /// An explicitly touched NaN or infinity is not a numeric value.
     NonFinite,
+    /// A runtime scale that is zero, negative, NaN or infinite.
+    InvalidScale,
+    /// Batch slices whose lengths do not describe the same lanes.
+    LengthMismatch,
+}
+
+/// The one encoder both scale forms call.
+#[inline(always)]
+fn encode_with_scale<R: NearestRounding9>(value: f32, scale: f32) -> Result<EpiLane9, Codec9Error> {
+    if !value.is_finite() {
+        return Err(Codec9Error::NonFinite);
+    }
+    let quantized = R::round(value / scale);
+    let magnitude = quantized.abs().clamp(0.0, 255.0) as u8;
+    Ok(EpiLane9 {
+        magnitude,
+        positive: quantized >= 0.0 || magnitude == 0,
+    })
+}
+
+/// The one decoder both scale forms call.
+#[inline(always)]
+fn decode_with_scale(lane: EpiLane9, scale: f32) -> f32 {
+    match lane.value() {
+        None => f32::NAN,
+        Some(v) => f32::from(v) * scale,
+    }
 }
 
 /// Compile-time-calibrated canonical DR512 numeric codec.
@@ -147,26 +190,114 @@ impl<R: NearestRounding9, C: Calibration> Codec9<R, C> {
     /// Finite out-of-range values saturate at magnitude 255.
     pub fn encode_touched(value: f32) -> Result<EpiLane9, Codec9Error> {
         Self::assert_calibration();
-        if !value.is_finite() {
-            return Err(Codec9Error::NonFinite);
-        }
-        let quantized = R::round(value / C::SCALE);
-        let magnitude = quantized.abs().clamp(0.0, 255.0) as u8;
-        Ok(EpiLane9 {
-            magnitude,
-            positive: quantized >= 0.0 || magnitude == 0,
-        })
+        encode_with_scale::<R>(value, C::SCALE)
     }
 
     /// Decodes -0 into a floating NaN and all other codes using calibration.
     /// This is an explicit numeric view, not a permission or NULL decision.
     #[must_use]
     pub fn decode(lane: EpiLane9) -> f32 {
         Self::assert_calibration();
-        match lane.value() {
-            None => f32::NAN,
-            Some(v) => f32::from(v) * C::SCALE,
+        decode_with_scale(lane, C::SCALE)
+    }
+}
+
+/// DR512 Codec9 with a scale chosen at run time.
+///
+/// Same storage, polarity, Sym and NaN contract as [`Codec9`], and the same
+/// encoder: for a scale equal to a [`Calibration`]'s `SCALE` the codewords are
+/// identical. The scale must be positive and finite; there is no zero point.
+///
+/// ```
+/// use ndarray::hpc::codec8::TiesEven;
+/// use ndarray::hpc::codec9::Codec9Runtime;
+/// let c = Codec9Runtime::<TiesEven>::new(0.5).unwrap();
+/// assert_eq!(c.encode_touched(-0.2).unwrap().codeword(), 0x100); // touched zero is +0
+/// let (mut m, mut p) = ([0u8; 2], [0u64; 1]);
+/// c.encode_touched_batch(&[1.0, -1.0], &mut m, &mut p).unwrap();
+/// assert_eq!((m, p[0]), ([2, 2], 0b01));
+/// ```
+#[derive(Clone, Copy, Debug, PartialEq)]
+pub struct Codec9Runtime<R: NearestRounding9> {
+    scale: f32,
+    _rounding: PhantomData<R>,
+}
+
+impl<R: NearestRounding9> Codec9Runtime<R> {
+    /// Checks the scale once; every later call relies on it.
+    pub fn new(scale: f32) -> Result<Self, Codec9Error> {
+        if scale > 0.0 && scale < f32::INFINITY {
+            Ok(Self {
+                scale,
+                _rounding: PhantomData,
+            })
+        } else {
+            Err(Codec9Error::InvalidScale)
+        }
+    }
+
+    /// The scale this codec encodes with.
+    #[must_use]
+    pub fn scale(&self) -> f32 {
+        self.scale
+    }
+
+    /// Encodes one touched finite value; see [`Codec9::encode_touched`].
+    pub fn encode_touched(&self, value: f32) -> Result<EpiLane9, Codec9Error> {
+        encode_with_scale::<R>(value, self.scale)
+    }
+
+    /// Decodes one lane; never-touched becomes NaN.
+    #[must_use]
+    pub fn decode(&self, lane: EpiLane9) -> f32 {
+        decode_with_scale(lane, self.scale)
+    }
+
+    /// Encodes `values` as touched lanes: `magnitudes[i]` and bit `i % 64` of
+    /// `presence[i / 64]`.
+    ///
+    /// `magnitudes` must have `values.len()` entries and `presence` exactly
+    /// `values.len().div_ceil(64)` words. Presence bits past the last value
+    /// are cleared: those lanes do not exist, so they read as never touched.
+    /// Every value is checked before anything is written, so a non-finite
+    /// value leaves both outputs unchanged.
+    pub fn encode_touched_batch(
+        &self, values: &[f32], magnitudes: &mut [u8], presence: &mut [u64],
+    ) -> Result<(), Codec9Error> {
+        if magnitudes.len() != values.len() || presence.len() != values.len().div_ceil(64) {
+            return Err(Codec9Error::LengthMismatch);
+        }
+        if !values.iter().all(|v| v.is_finite()) {
+            return Err(Codec9Error::NonFinite);
         }
+        for (w, word) in presence.iter_mut().enumerate() {
+            let lanes = (values.len() - 64 * w).min(64);
+            let mut bits = 0u64;
+            for k in 0..lanes {
+                let i = 64 * w + k;
+                let lane = encode_with_scale::<R>(values[i], self.scale)?;
+                magnitudes[i] = lane.magnitude;
+                bits |= u64::from(lane.positive) << k;
+            }
+            *word = bits;
+        }
+        Ok(())
+    }
+
+    /// Decodes a magnitude lane and its presence plane into `out`;
+    /// never-touched lanes (P=0, M=0) become NaN.
+    ///
+    /// `out` must have `magnitudes.len()` entries and `presence` exactly
+    /// `magnitudes.len().div_ceil(64)` words.
+    pub fn decode_batch(&self, magnitudes: &[u8], presence: &[u64], out: &mut [f32]) -> Result<(), Codec9Error> {
+        if out.len() != magnitudes.len() || presence.len() != magnitudes.len().div_ceil(64) {
+            return Err(Codec9Error::LengthMismatch);
+        }
+        for (i, (&m, o)) in magnitudes.iter().zip(out.iter_mut()).enumerate() {
+            let positive = presence[i / 64] >> (i % 64) & 1 == 1;
+            *o = decode_with_scale(EpiLane9 { magnitude: m, positive }, self.scale);
+        }
+        Ok(())
     }
 }
 
@@ -237,6 +368,123 @@ mod tests {
         }
     }
 
+    struct Tenth;
+    impl Calibration for Tenth {
+        const SCALE: f32 = 0.1;
+        const ZERO_POINT: i32 = 0;
+    }
+
+    fn probe_values() -> Vec<f32> {
+        let mut v: Vec<f32> = (-3000..=3000).map(|i| i as f32 * 0.0137).collect();
+        v.extend([0.0, -0.0, -0.04, -0.05, -0.050001, 0.05, 1e-30, -1e-30, 25.5, -25.5, 1e9, -1e9]);
+        v
+    }
+
+    #[test]
+    fn runtime_scale_gives_the_same_codewords_as_the_compile_time_codec() {
+        let even = Codec9Runtime::<TiesEven>::new(0.1).unwrap();
+        let away = Codec9Runtime::<TiesAway>::new(0.1).unwrap();
+        for v in probe_values() {
+            assert_eq!(even.encode_touched(v), Codec9::<TiesEven, Tenth>::encode_touched(v), "{v}");
+            assert_eq!(away.encode_touched(v), Codec9::<TiesAway, Tenth>::encode_touched(v), "{v}");
+        }
+        for code in 0..512u16 {
+            let lane = EpiLane9::from_codeword(code).unwrap();
+            let (rt, ct) = (even.decode(lane), Codec9::<TiesEven, Tenth>::decode(lane));
+            assert!(rt.to_bits() == ct.to_bits() || (rt.is_nan() && ct.is_nan()), "{code}");
+        }
+    }
+
+    #[test]
+    fn runtime_scale_keeps_the_touched_zero_rule() {
+        let c = Codec9Runtime::<TiesEven>::new(0.5).unwrap();
+        for v in [-0.0, -0.1, -0.249, -0.25] {
+            assert_eq!(c.encode_touched(v).unwrap().codeword(), 0x100, "{v}");
+        }
+        assert_eq!(c.encode_touched(-0.2501).unwrap().codeword(), 0x001);
+        assert_eq!(c.encode_touched(f32::NAN), Err(Codec9Error::NonFinite));
+    }
+
+    #[test]
+    fn runtime_scale_refuses_an_unusable_scale() {
+        for s in [0.0, -1.0, f32::NAN, f32::INFINITY, f32::NEG_INFINITY] {
+            assert_eq!(Codec9Runtime::<TiesEven>::new(s).err(), Some(Codec9Error::InvalidScale), "{s}");
+        }
+        assert!(Codec9Runtime::<TiesEven>::new(f32::MIN_POSITIVE).is_ok());
+    }
+
+    #[test]
+    fn a_different_scale_gives_different_codewords() {
+        // Can-fire half: the scale is used, not ignored.
+        let a = Codec9Runtime::<TiesEven>::new(0.1).unwrap();
+        let b = Codec9Runtime::<TiesEven>::new(0.2).unwrap();
+        assert_ne!(a.encode_touched(1.0), b.encode_touched(1.0));
+    }
+
+    #[test]
+    fn batch_encode_matches_per_value_encode_and_writes_the_presence_plane() {
+        let c = Codec9Runtime::<TiesEven>::new(0.0137).unwrap();
+        let values = probe_values();
+        let n = values.len();
+        let mut mags = vec![0xAAu8; n];
+        let mut pres = vec![u64::MAX; n.div_ceil(64)];
+        c.encode_touched_batch(&values, &mut mags, &mut pres)
+            .unwrap();
+        let mut positives = 0;
+        for (i, &v) in values.iter().enumerate() {
+            let one = c.encode_touched(v).unwrap();
+            let bit = pres[i / 64] >> (i % 64) & 1 == 1;
+            assert_eq!((mags[i], bit), (one.magnitude(), one.presence()), "lane {i} value {v}");
+            positives += bit as usize;
+        }
+        // Anti-vacuity: both polarities occur, and no touched lane is the sentinel.
+        assert!(positives > 0 && positives < n);
+        assert!((0..n).all(|i| mags[i] != 0 || pres[i / 64] >> (i % 64) & 1 == 1));
+        // Bits past the last lane are cleared: those lanes were never touched.
+        let tail = n % 64;
+        if tail != 0 {
+            assert_eq!(pres[n / 64] >> tail, 0);
+        }
+    }
+
+    #[test]
+    fn batch_round_trip_decodes_the_same_floats() {
+        let c = Codec9Runtime::<TiesAway>::new(0.25).unwrap();
+        let values: Vec<f32> = (-600..600).map(|i| i as f32 * 0.11).collect();
+        let n = values.len();
+        let (mut mags, mut pres) = (vec![0u8; n], vec![0u64; n.div_ceil(64)]);
+        c.encode_touched_batch(&values, &mut mags, &mut pres)
+            .unwrap();
+        let mut out = vec![0f32; n];
+        c.decode_batch(&mags, &pres, &mut out).unwrap();
+        for i in 0..n {
+            assert_eq!(out[i], c.decode(c.encode_touched(values[i]).unwrap()), "lane {i}");
+        }
+    }
+
+    #[test]
+    fn batch_decode_turns_never_touched_lanes_into_nan() {
+        let c = Codec9Runtime::<TiesEven>::new(1.0).unwrap();
+        let mags = [0u8, 0, 7, 7];
+        let pres = [0b0110u64];
+        let mut out = [0f32; 4];
+        c.decode_batch(&mags, &pres, &mut out).unwrap();
+        assert!(out[0].is_nan());
+        assert_eq!(&out[1..], &[0.0, 7.0, -7.0]);
+    }
+
+    #[test]
+    fn batch_encode_refuses_non_finite_input_and_bad_lengths() {
+        let c = Codec9Runtime::<TiesEven>::new(1.0).unwrap();
+        let (mut m, mut p) = ([0u8; 3], [0u64; 1]);
+        assert_eq!(c.encode_touched_batch(&[1.0, f32::INFINITY, 2.0], &mut m, &mut p), Err(Codec9Error::NonFinite));
+        assert_eq!((m, p), ([0u8; 3], [0u64; 1]), "a refused batch must not write");
+        assert_eq!(c.encode_touched_batch(&[1.0; 3], &mut [0u8; 2], &mut p), Err(Codec9Error::LengthMismatch));
+        assert_eq!(c.encode_touched_batch(&[1.0; 65], &mut [0u8; 65], &mut p), Err(Codec9Error::LengthMismatch));
+        let mut out = [0f32; 3];
+        assert_eq!(c.decode_batch(&m, &[], &mut out), Err(Codec9Error::LengthMismatch));
+    }
+
     #[test]
     fn decode_and_reencode_touched_values_preserve_the_entire_numeric_domain() {
         for code in 1..512u16 {