Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ jobs:
shell: bash
run: |
set +e
node --test test/*.test.js 2>&1 | tee /tmp/win-test.log
node --test --import ./test/_setup.js test/*.test.js 2>&1 | tee /tmp/win-test.log
ec=${PIPESTATUS[0]}
echo "===== FAILING TESTS (name + file) ====="
grep -nE '^not ok ' /tmp/win-test.log | head -80
Expand Down
20 changes: 19 additions & 1 deletion ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -280,6 +280,22 @@ fails safe to the stock ID on no gateway / unreachable `/v1/models` / no family
resolved `tier→model` mapping for verification. The `MODELS` export shape is unchanged: this is a
resolution-time layer, not a table edit.

**Typed proposers via TypeSafe System One (`src/jev.js`).** Two of the substrate's proposer
judgments are not text-generation tasks at all: `route`'s complexity band is a classification
(cheap/mid/premium), and preflight's assumption gate is four independent yes/no readings (one
per rubric dimension). When `TYPESAFE_API_KEY` is set (same `FORGE_LLM=1` opt-in), those two
faculties ask Jev instead of a text model — one batched `POST /v1/systemone` returning typed
`choice`/`noul` answers with probability distributions and confidence in ~150ms, versus seconds
of text plus JSON parsing. The module reuses the adjudicate contract verbatim: opt-in, fail-safe
(null → text-LLM fallback → deterministic rubric; a null never moves a verdict), zero-dependency
(the `llm.js` spawned-child pattern, key in child env as `_FORGE_JEV_KEY`), and secret-refusing
on the outgoing state. Jev answers are validated against the questions asked — a choice naming
an option we never offered is garble and fails safe. The reconciles are untouched: `BAND_FLOOR`
still floors the routing band, the assumption gate still bounds completeness to ±band, and
clarifying free-text questions stay with the deterministic rubric, because a System One model
judges but does not author prose. Provenance records which proposer answered
(`llm.provider: "jev"` in `forge route --json`, `assumption.provenance.provider` in preflight).

**Intent cards (`src/intent.js`).** Prompt → intent by the same exemplar k-NN math as
model routing — a labeled bank (English + Hinglish rows) under overlap similarity with a
confidence gate, NOT a keyword DFA. Note `intentGrams` ≠ `contentGrams`: route.js stops
Expand Down Expand Up @@ -581,18 +597,20 @@ from the tree it describes.
```mermaid
%%{init: {'theme':'base','themeVariables':{'primaryColor':'#201a15','primaryTextColor':'#f2ede7','primaryBorderColor':'#372c22','lineColor':'#f26430','secondaryColor':'#272019','tertiaryColor':'#171310','edgeLabelBackground':'#201a15','clusterBkg':'#171310','clusterBorder':'#4a3b2e','fontFamily':'ui-sans-serif, system-ui, sans-serif','fontSize':'14px'},'flowchart':{'curve':'basis','padding':10,'nodeSpacing':36,'rankSpacing':44}}}%%
flowchart LR
test["test<br/>105 files"]
test["test<br/>106 files"]
src["src<br/>97 files"]
landing["landing<br/>61 files"]
research["research<br/>35 files"]
global["global<br/>3 files"]
bench["bench<br/>2 files"]
scripts["scripts<br/>2 files"]
_remember[".remember<br/>1 file"]
docs["docs<br/>1 file"]
test -- 201 --> src
examples["examples<br/>1 file"]
test -- 206 --> src
bench -- 7 --> src
examples -- 4 --> src
test -- 2 --> scripts
scripts --> src
src --> global
Expand Down
38 changes: 37 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,43 @@ to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

### Added

- **TypeSafe System One (Jev) as the fast proposer.** Where forge's LLM layer asked a text
model for a judgment that is really a classification or a yes/no — `route`'s complexity band
and preflight's assumption gate — it can now ask Jev instead: typed `choice`/`noul` answers
with real probability distributions and confidence in ~150ms, rather than seconds of text
generation followed by JSON parsing. The new `src/jev.js` client follows the existing
proposer contract exactly: opt-in (`FORGE_LLM=1` plus `TYPESAFE_API_KEY`, overridable via
`TYPESAFE_BASE_URL`), fail-safe (any error → null → text-LLM fallback → deterministic
rubric, and a null never changes a verdict), zero-dependency (one raw HTTPS POST through
the child-process-fetch pattern, the key travelling via child env — never argv, never
logged), and secret-refusing on the way out. Routing keeps its `BAND_FLOOR` reconcile and
gains `llm.provider: "jev"` plus confidence in `forge route --json`; the assumption gate
scores all four rubric dimensions in one batched call (free-text clarifying questions stay
with the deterministic rubric — a System One model judges, it does not author prose).
`test/_setup.js` now scrubs `TYPESAFE_*` so the suite stays hermetic with the key exported.

### Fixed

- **The test suite is hermetic.** It inherited the developer's environment, so it was green
in CI and red on any machine where forge was actually installed and enabled — the two
things a maintainer does. An exported `FORGE_LLM=1` both flipped the "llm off by default"
assertion in `test/substrate.test.js` and made the faculties fire real model calls, and a
real `~/.forge` reached `doctor()`'s machine-scoped install check through
`test/doctor.test.js`. Wall time was 593s with two failures. A new `test/_setup.js`,
preloaded via `--import` into every test process, scrubs `FORGE_*`/provider env by prefix,
sandboxes `$HOME` to a throwaway tmpdir, and forces the keyless HTTP runner instead of
shelling out to a real `claude` binary: **0 failures in ~40s**. `test/hermetic.test.js`
pins the scrub list against `envVarsRead()` so the two cannot drift, and fails loudly if
anyone drops the `--import` wiring. Two assertions were wrong rather than merely leaky and
were corrected: `doctor` asserted a global `failed === 0` to prove a local property about
`na` rows, and a comment in `substrate` claimed no runner reaches the real CLI — the
opposite of the truth, and the reason that file spent 85s on live calls.

### Documentation

- `CLAUDE.md`: Biome 2.5.2 → 2.5.5 (matching the pin), "600+ tests" → "1000+", and the lint
command `npx biome check` → `npm run check` — the documented command fails outright, since
the npx package is `@biomejs/biome`, not `biome`.
- **OpenClaw is a first-class emit target — the compiler's tenth tool.** Instructions need
no new file: OpenClaw appends the execution folder's `AGENTS.md` after its configured
agent-workspace files as project context, so the canonical source reaches it the same way
Expand Down Expand Up @@ -198,7 +235,6 @@ to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
every PreToolUse hook and recompiled the same trigger-glob RegExp each time; compiled
globs are now cached in a module-level map bounded by the distinct globs in the
lesson set.

## [0.27.4] - 2026-08-04

### Fixed
Expand Down
6 changes: 3 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,14 @@
## Stack

- Node.js >=20, pure ESM (`"type": "module"`), zero runtime dependencies.
- Linter/formatter: Biome 2.5.2 (dev dependency).
- Linter/formatter: Biome 2.5.5 (dev dependency).
- Types: TypeScript via JSDoc annotations — no `.ts` files, checked by `tsc`.

## Commands

- Install: `npm ci`
- Test: `npm test` (node:test, 600+ tests)
- Lint + format: `npx biome check` (or `npm run check`)
- Test: `npm test` (node:test, 1000+ tests)
- Lint + format: `npm run check` (the npx package is `@biomejs/biome`, not `biome`)
- Typecheck: `npm run typecheck`
- Build pages: `npm run pages:build`

Expand Down
13 changes: 13 additions & 0 deletions docs/GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -1452,6 +1452,17 @@ exposes `llm.provenance` per faculty (`llm-cleared` / `llm-tightened` / `llm-rai
conservative tighten-/raise-only mode. Each faculty pairs a pure `*LLM` proposer with a
`reconcile` step — extend by adding both, never by trusting the model's answer directly.

**TypeSafe System One (Jev) is the preferred proposer when configured.** Where the judgment
is already a classification or a yes/no — `route`'s complexity band (a `choice` over
cheap/mid/premium) and preflight's assumption gate (one batched `noul` per rubric dimension) —
`src/jev.js` asks Jev instead of a text model: typed answers with real probability
distributions and confidence in ~150ms, rather than seconds of generation followed by JSON
parsing. Set `TYPESAFE_API_KEY` (plus the same `FORGE_LLM=1` opt-in) and the two proposers
prefer it automatically; the text-LLM runner remains the fallback on any failure, and the
deterministic rubrics still judge. Free-text clarifying questions stay with the rubric — a
System One model judges, it does not author prose. `forge route --json` shows which proposer
answered under `llm.provider` (`jev` / `text`) with Jev's confidence.

### Support a new tool

Add an emitter module in `src/emit/<tool>.js` (mirror an existing one like
Expand All @@ -1478,6 +1489,8 @@ code reads but this table misses fails CI on the forge repo):
| `OPENROUTER_API_KEY` | OpenRouter provider |
| `OPENAI_API_KEY` | OpenAI provider (OpenAI-compatible chat/completions); low-configuration auto-detect fallback after Anthropic |
| `GEMINI_API_KEY` / `GOOGLE_API_KEY` | Google Gemini provider via its OpenAI-compatible endpoint; low-configuration auto-detect fallback after Anthropic |
| `TYPESAFE_API_KEY` | TypeSafe System One (Jev) — with `FORGE_LLM=1`, the route/assumption proposers prefer typed ~150ms judgments over a text round-trip; unset = text-LLM proposer only |
| `TYPESAFE_BASE_URL` | override the Jev endpoint (default `https://api.typesafe.ai`) — staging/self-hosted |
| `FORGE_LLM` | `1` enables the LLM proposer layer (off = fully deterministic) |
| `FORGE_LLM_AMBIENT` | `1` lets the ambient hook use the proposer too |
| `FORGE_LLM_HTTP` | `1` forces direct HTTP (Anthropic Messages or OpenAI-compatible, per the resolved provider) instead of the `claude` CLI; automatic when the CLI is absent |
Expand Down
16 changes: 16 additions & 0 deletions mintlify/concepts/model-routing.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,22 @@ under an overlap-similarity metric with a confidence gate — not a keyword look
genuinely needs it.
</Note>

## Optional typed proposer — TypeSafe Jev

The rubric is the judge; an optional **proposer** layer can refine it. With `FORGE_LLM=1`
plus `TYPESAFE_API_KEY` set, `route` asks TypeSafe's System One model (Jev) for the
complexity band as a typed `choice` with probabilities and confidence in ~150ms — instead
of a multi-second text-LLM round-trip. The same applies to preflight's assumption gate,
scored as one batched `noul` per rubric dimension.

<Note>
Fail-safe by construction: any Jev error falls back to the text-LLM proposer, then to
the deterministic rubric — and a miss never changes a verdict. Without the key, behavior
is byte-identical. `forge route --json` shows which proposer answered under
`llm.provider` (`jev` / `text`), with Jev's confidence. `TYPESAFE_BASE_URL` overrides
the endpoint for staging or self-hosted deployments.
</Note>

## Intent, then tier

Routing shares its math with intent detection (`src/intent.js`): a prompt maps to an
Expand Down
9 changes: 9 additions & 0 deletions mintlify/concepts/pre-action-gate.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,15 @@ flowchart TD
</Step>
</Steps>

## Optional model proposers

Every phase above is a deterministic rubric by default. `FORGE_LLM=1` adds a thin
**proposer** layer that can refine — never decide — the gate. With `TYPESAFE_API_KEY`
also set, the route and assumption proposers prefer TypeSafe's System One (Jev): typed
`choice` / `noul` answers with probabilities in ~150ms instead of a text round-trip.
Any failure falls back to the deterministic path, so the flags are safe to leave off or
on.

## Blast radius

**Blast radius** — the set of files an edit is predicted to impact, read from the code
Expand Down
4 changes: 2 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -52,14 +52,14 @@
"scripts"
],
"scripts": {
"test": "node --test test/*.test.js",
"test": "node --test --import ./test/_setup.js test/*.test.js",
"bench": "node bench/bench.mjs",
"lint": "biome lint .",
"format": "biome format --write .",
"check": "biome check .",
"check:fix": "biome check --write .",
"typecheck": "tsc -p tsconfig.json",
"coverage": "node --test --experimental-test-coverage test/*.test.js",
"coverage": "node --test --experimental-test-coverage --import ./test/_setup.js test/*.test.js",
"bump": "node scripts/bump.mjs",
"forge": "node src/cli.js",
"pages:build": "node scripts/build-pages.mjs",
Expand Down
4 changes: 3 additions & 1 deletion src/docs_check.js
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ const DOC_FILES = ["README.md", "docs/GUIDE.md", "ARCHITECTURE.md", "ROADMAP.md"
// values injected by host tools rather than set by users.
const INTERNAL_ENV = new Set([
"_FORGE_LLM_KEY",
"_FORGE_JEV_KEY",
"FORGE_EMBED_KEY",
// Test-only override of the settings.json path `forge init` targets — plumbing for
// exercising merge/remove/exit-code behavior without touching the real ~/.claude.
Expand All @@ -31,7 +32,8 @@ const INTERNAL_ENV = new Set([

// Prefixes that mark an env var as OURS to document. A doc may freely mention other
// tools' vars (GITHUB_TOKEN, PATH) — those aren't claims about forge's own surface.
const ENV_PREFIX_RE = /\b((?:FORGE|ANTHROPIC|LITELLM|OPENROUTER|ENABLE_CORTEX)_[A-Z0-9_]+)\b/g;
const ENV_PREFIX_RE =
/\b((?:FORGE|ANTHROPIC|LITELLM|OPENROUTER|ENABLE_CORTEX|TYPESAFE)_[A-Z0-9_]+)\b/g;

function readDoc(root, rel) {
const p = join(root, rel);
Expand Down
Loading
Loading