Merge pull request #146 from CodeWithJuber/fix/hermetic-test-suite

feat: TypeSafe System One (Jev) typed proposers + hermetic test suite

Author: CodeWithJuber

.github/workflows/ci.yml

@@ -48,7 +48,7 @@ jobs:
         shell: bash
         run: |
           set +e
-          node --test test/*.test.js 2>&1 | tee /tmp/win-test.log
+          node --test --import ./test/_setup.js test/*.test.js 2>&1 | tee /tmp/win-test.log
           ec=${PIPESTATUS[0]}
           echo "===== FAILING TESTS (name + file) ====="
           grep -nE '^not ok ' /tmp/win-test.log | head -80

ARCHITECTURE.md

@@ -280,6 +280,22 @@ fails safe to the stock ID on no gateway / unreachable `/v1/models` / no family
 resolved `tier→model` mapping for verification. The `MODELS` export shape is unchanged: this is a
 resolution-time layer, not a table edit.
 
+**Typed proposers via TypeSafe System One (`src/jev.js`).** Two of the substrate's proposer
+judgments are not text-generation tasks at all: `route`'s complexity band is a classification
+(cheap/mid/premium), and preflight's assumption gate is four independent yes/no readings (one
+per rubric dimension). When `TYPESAFE_API_KEY` is set (same `FORGE_LLM=1` opt-in), those two
+faculties ask Jev instead of a text model — one batched `POST /v1/systemone` returning typed
+`choice`/`noul` answers with probability distributions and confidence in ~150ms, versus seconds
+of text plus JSON parsing. The module reuses the adjudicate contract verbatim: opt-in, fail-safe
+(null → text-LLM fallback → deterministic rubric; a null never moves a verdict), zero-dependency
+(the `llm.js` spawned-child pattern, key in child env as `_FORGE_JEV_KEY`), and secret-refusing
+on the outgoing state. Jev answers are validated against the questions asked — a choice naming
+an option we never offered is garble and fails safe. The reconciles are untouched: `BAND_FLOOR`
+still floors the routing band, the assumption gate still bounds completeness to ±band, and
+clarifying free-text questions stay with the deterministic rubric, because a System One model
+judges but does not author prose. Provenance records which proposer answered
+(`llm.provider: "jev"` in `forge route --json`, `assumption.provenance.provider` in preflight).
+
 **Intent cards (`src/intent.js`).** Prompt → intent by the same exemplar k-NN math as
 model routing — a labeled bank (English + Hinglish rows) under overlap similarity with a
 confidence gate, NOT a keyword DFA. Note `intentGrams` ≠ `contentGrams`: route.js stops
@@ -581,18 +597,20 @@ from the tree it describes.
 ```mermaid
 %%{init: {'theme':'base','themeVariables':{'primaryColor':'#201a15','primaryTextColor':'#f2ede7','primaryBorderColor':'#372c22','lineColor':'#f26430','secondaryColor':'#272019','tertiaryColor':'#171310','edgeLabelBackground':'#201a15','clusterBkg':'#171310','clusterBorder':'#4a3b2e','fontFamily':'ui-sans-serif, system-ui, sans-serif','fontSize':'14px'},'flowchart':{'curve':'basis','padding':10,'nodeSpacing':36,'rankSpacing':44}}}%%
 flowchart LR
+  test["test<br/>105 files"]
   test["test<br/>106 files"]
   src["src<br/>97 files"]
   landing["landing<br/>61 files"]
   research["research<br/>35 files"]
   global["global<br/>3 files"]
   bench["bench<br/>2 files"]
   scripts["scripts<br/>2 files"]
+  _remember[".remember<br/>1 file"]
   docs["docs<br/>1 file"]
+  test -- 201 --> src
   examples["examples<br/>1 file"]
   test -- 206 --> src
   bench -- 7 --> src
-  examples -- 4 --> src
   test -- 2 --> scripts
   scripts --> src
   src --> global

CHANGELOG.md

@@ -8,6 +8,43 @@ to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 ### Added
 
+- **TypeSafe System One (Jev) as the fast proposer.** Where forge's LLM layer asked a text
+  model for a judgment that is really a classification or a yes/no — `route`'s complexity band
+  and preflight's assumption gate — it can now ask Jev instead: typed `choice`/`noul` answers
+  with real probability distributions and confidence in ~150ms, rather than seconds of text
+  generation followed by JSON parsing. The new `src/jev.js` client follows the existing
+  proposer contract exactly: opt-in (`FORGE_LLM=1` plus `TYPESAFE_API_KEY`, overridable via
+  `TYPESAFE_BASE_URL`), fail-safe (any error → null → text-LLM fallback → deterministic
+  rubric, and a null never changes a verdict), zero-dependency (one raw HTTPS POST through
+  the child-process-fetch pattern, the key travelling via child env — never argv, never
+  logged), and secret-refusing on the way out. Routing keeps its `BAND_FLOOR` reconcile and
+  gains `llm.provider: "jev"` plus confidence in `forge route --json`; the assumption gate
+  scores all four rubric dimensions in one batched call (free-text clarifying questions stay
+  with the deterministic rubric — a System One model judges, it does not author prose).
+  `test/_setup.js` now scrubs `TYPESAFE_*` so the suite stays hermetic with the key exported.
+
+### Fixed
+
+- **The test suite is hermetic.** It inherited the developer's environment, so it was green
+  in CI and red on any machine where forge was actually installed and enabled — the two
+  things a maintainer does. An exported `FORGE_LLM=1` both flipped the "llm off by default"
+  assertion in `test/substrate.test.js` and made the faculties fire real model calls, and a
+  real `~/.forge` reached `doctor()`'s machine-scoped install check through
+  `test/doctor.test.js`. Wall time was 593s with two failures. A new `test/_setup.js`,
+  preloaded via `--import` into every test process, scrubs `FORGE_*`/provider env by prefix,
+  sandboxes `$HOME` to a throwaway tmpdir, and forces the keyless HTTP runner instead of
+  shelling out to a real `claude` binary: **0 failures in ~40s**. `test/hermetic.test.js`
+  pins the scrub list against `envVarsRead()` so the two cannot drift, and fails loudly if
+  anyone drops the `--import` wiring. Two assertions were wrong rather than merely leaky and
+  were corrected: `doctor` asserted a global `failed === 0` to prove a local property about
+  `na` rows, and a comment in `substrate` claimed no runner reaches the real CLI — the
+  opposite of the truth, and the reason that file spent 85s on live calls.
+
+### Documentation
+
+- `CLAUDE.md`: Biome 2.5.2 → 2.5.5 (matching the pin), "600+ tests" → "1000+", and the lint
+  command `npx biome check` → `npm run check` — the documented command fails outright, since
+  the npx package is `@biomejs/biome`, not `biome`.
 - **OpenClaw is a first-class emit target — the compiler's tenth tool.** Instructions need
   no new file: OpenClaw appends the execution folder's `AGENTS.md` after its configured
   agent-workspace files as project context, so the canonical source reaches it the same way
@@ -198,7 +235,6 @@ to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
   every PreToolUse hook and recompiled the same trigger-glob RegExp each time; compiled
   globs are now cached in a module-level map bounded by the distinct globs in the
   lesson set.
-
 ## [0.27.4] - 2026-08-04
 
 ### Fixed

CLAUDE.md

@@ -3,14 +3,14 @@
 ## Stack
 
 - Node.js >=20, pure ESM (`"type": "module"`), zero runtime dependencies.
-- Linter/formatter: Biome 2.5.2 (dev dependency).
+- Linter/formatter: Biome 2.5.5 (dev dependency).
 - Types: TypeScript via JSDoc annotations — no `.ts` files, checked by `tsc`.
 
 ## Commands
 
 - Install: `npm ci`
-- Test: `npm test` (node:test, 600+ tests)
-- Lint + format: `npx biome check` (or `npm run check`)
+- Test: `npm test` (node:test, 1000+ tests)
+- Lint + format: `npm run check` (the npx package is `@biomejs/biome`, not `biome`)
 - Typecheck: `npm run typecheck`
 - Build pages: `npm run pages:build`
 

docs/GUIDE.md

@@ -1452,6 +1452,17 @@ exposes `llm.provenance` per faculty (`llm-cleared` / `llm-tightened` / `llm-rai
 conservative tighten-/raise-only mode. Each faculty pairs a pure `*LLM` proposer with a
 `reconcile` step — extend by adding both, never by trusting the model's answer directly.
 
+**TypeSafe System One (Jev) is the preferred proposer when configured.** Where the judgment
+is already a classification or a yes/no — `route`'s complexity band (a `choice` over
+cheap/mid/premium) and preflight's assumption gate (one batched `noul` per rubric dimension) —
+`src/jev.js` asks Jev instead of a text model: typed answers with real probability
+distributions and confidence in ~150ms, rather than seconds of generation followed by JSON
+parsing. Set `TYPESAFE_API_KEY` (plus the same `FORGE_LLM=1` opt-in) and the two proposers
+prefer it automatically; the text-LLM runner remains the fallback on any failure, and the
+deterministic rubrics still judge. Free-text clarifying questions stay with the rubric — a
+System One model judges, it does not author prose. `forge route --json` shows which proposer
+answered under `llm.provider` (`jev` / `text`) with Jev's confidence.
+
 ### Support a new tool
 
 Add an emitter module in `src/emit/<tool>.js` (mirror an existing one like
@@ -1478,6 +1489,8 @@ code reads but this table misses fails CI on the forge repo):
 | `OPENROUTER_API_KEY`                                           | OpenRouter provider                                                                                                                                                                                                                     |
 | `OPENAI_API_KEY`                                               | OpenAI provider (OpenAI-compatible chat/completions); low-configuration auto-detect fallback after Anthropic                                                                                                                            |
 | `GEMINI_API_KEY` / `GOOGLE_API_KEY`                            | Google Gemini provider via its OpenAI-compatible endpoint; low-configuration auto-detect fallback after Anthropic                                                                                                                       |
+| `TYPESAFE_API_KEY`                                             | TypeSafe System One (Jev) — with `FORGE_LLM=1`, the route/assumption proposers prefer typed ~150ms judgments over a text round-trip; unset = text-LLM proposer only                                                                     |
+| `TYPESAFE_BASE_URL`                                            | override the Jev endpoint (default `https://api.typesafe.ai`) — staging/self-hosted                                                                                                                                                   |
 | `FORGE_LLM`                                                    | `1` enables the LLM proposer layer (off = fully deterministic)                                                                                                                                                                          |
 | `FORGE_LLM_AMBIENT`                                            | `1` lets the ambient hook use the proposer too                                                                                                                                                                                          |
 | `FORGE_LLM_HTTP`                                               | `1` forces direct HTTP (Anthropic Messages or OpenAI-compatible, per the resolved provider) instead of the `claude` CLI; automatic when the CLI is absent                                                                               |

mintlify/concepts/model-routing.mdx

@@ -24,6 +24,22 @@ under an overlap-similarity metric with a confidence gate — not a keyword look
   genuinely needs it.
 </Note>
 
+## Optional typed proposer — TypeSafe Jev
+
+The rubric is the judge; an optional **proposer** layer can refine it. With `FORGE_LLM=1`
+plus `TYPESAFE_API_KEY` set, `route` asks TypeSafe's System One model (Jev) for the
+complexity band as a typed `choice` with probabilities and confidence in ~150ms — instead
+of a multi-second text-LLM round-trip. The same applies to preflight's assumption gate,
+scored as one batched `noul` per rubric dimension.
+
+<Note>
+  Fail-safe by construction: any Jev error falls back to the text-LLM proposer, then to
+  the deterministic rubric — and a miss never changes a verdict. Without the key, behavior
+  is byte-identical. `forge route --json` shows which proposer answered under
+  `llm.provider` (`jev` / `text`), with Jev's confidence. `TYPESAFE_BASE_URL` overrides
+  the endpoint for staging or self-hosted deployments.
+</Note>
+
 ## Intent, then tier
 
 Routing shares its math with intent detection (`src/intent.js`): a prompt maps to an

mintlify/concepts/pre-action-gate.mdx

@@ -48,6 +48,15 @@ flowchart TD
   </Step>
 </Steps>
 
+## Optional model proposers
+
+Every phase above is a deterministic rubric by default. `FORGE_LLM=1` adds a thin
+**proposer** layer that can refine — never decide — the gate. With `TYPESAFE_API_KEY`
+also set, the route and assumption proposers prefer TypeSafe's System One (Jev): typed
+`choice` / `noul` answers with probabilities in ~150ms instead of a text round-trip.
+Any failure falls back to the deterministic path, so the flags are safe to leave off or
+on.
+
 ## Blast radius
 
 **Blast radius** — the set of files an edit is predicted to impact, read from the code

package.json

@@ -52,14 +52,14 @@
     "scripts"
   ],
   "scripts": {
-    "test": "node --test test/*.test.js",
+    "test": "node --test --import ./test/_setup.js test/*.test.js",
     "bench": "node bench/bench.mjs",
     "lint": "biome lint .",
     "format": "biome format --write .",
     "check": "biome check .",
     "check:fix": "biome check --write .",
     "typecheck": "tsc -p tsconfig.json",
-    "coverage": "node --test --experimental-test-coverage test/*.test.js",
+    "coverage": "node --test --experimental-test-coverage --import ./test/_setup.js test/*.test.js",
     "bump": "node scripts/bump.mjs",
     "forge": "node src/cli.js",
     "pages:build": "node scripts/build-pages.mjs",

src/docs_check.js

@@ -20,6 +20,7 @@ const DOC_FILES = ["README.md", "docs/GUIDE.md", "ARCHITECTURE.md", "ROADMAP.md"
 // values injected by host tools rather than set by users.
 const INTERNAL_ENV = new Set([
   "_FORGE_LLM_KEY",
+  "_FORGE_JEV_KEY",
   "FORGE_EMBED_KEY",
   // Test-only override of the settings.json path `forge init` targets — plumbing for
   // exercising merge/remove/exit-code behavior without touching the real ~/.claude.
@@ -31,7 +32,8 @@ const INTERNAL_ENV = new Set([
 
 // Prefixes that mark an env var as OURS to document. A doc may freely mention other
 // tools' vars (GITHUB_TOKEN, PATH) — those aren't claims about forge's own surface.
-const ENV_PREFIX_RE = /\b((?:FORGE|ANTHROPIC|LITELLM|OPENROUTER|ENABLE_CORTEX)_[A-Z0-9_]+)\b/g;
+const ENV_PREFIX_RE =
+  /\b((?:FORGE|ANTHROPIC|LITELLM|OPENROUTER|ENABLE_CORTEX|TYPESAFE)_[A-Z0-9_]+)\b/g;
 
 function readDoc(root, rel) {
   const p = join(root, rel);