fix: resolve the deep review's findings across impact, guards, memory, routing, primitives and research - #151
Open
CodeWithJuber wants to merge 70 commits into
Open
CodeWithJuber wants to merge 70 commits into
CodeWithJuber wants to merge 70 commits into
Conversation
forge doctor's plugin-hook check access(X_OK)s every script a hook names. run.mjs landed in #140 as 100644, so the check warned on Linux and test/doctor.test.js failed on Node 20/22 for every push since. Windows ignores X_OK, which is why the Windows job stayed green. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The committed block carried both sides of a merge (test at 105 and 106 files, two test->src edges) and a local-only .remember node. Regenerated with `forge docs render` from a clean LF checkout. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…tegrity digests - B1: bound every quantifier in the key-assigned branch; the unbounded `\b[\w-]*KEY[\w-]*` was cubic (12 KB of `token-` took 40 s, past the hook timeout). 40 KB of every pathological shape now takes < 5 ms. - B2: detect and mask URL userinfo credentials, AWS ASIA keys, AUTH=/ CREDENTIALS= env values, Authorization headers, glpat- and TypeSafe keys; mask assigned values whole (past `/`, and below 8 chars in env form). - B4: npm/yarn/SRI sha*- and go.sum h1: digests are exempt from the entropy leg, so a lockfile commit is no longer refused as a leaked secret. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The silent-miss residual multiplied (1 - w_j) over every lens that "ran": precision-style weights were used as catch probabilities, lenses aimed at disjoint defect classes were treated as independent tries at one defect, and lenses that ran over nothing (empty diff) still claimed coverage. With tests never run and an empty diff it reported 0.042. Each lens now carries a target class and an assumed catch probability in its own column. Same-class lenses combine as nested checks (1 - c_max, the dependence-free bound from review F2), a lens that examined no input catches nothing, and the residual is the worst class, with residualByClass in the provenance. That case now reports 1. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- Every ledger log append terminates a torn final line first, so a record appended after a crash mid-write is no longer glued onto the fragment and silently lost while append reports ok:true. - canonicalize() normalizes object keys to NFC before sorting, so an NFD key no longer produces bytes that fail the claim's own address check on reload. - cortex-mcp answers a throwing tool handler with a JSON-RPC -32603 error instead of swallowing it and leaving the client hanging. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The secret scan read `git diff --cached` with the default 1 MiB buffer, so any leak staged next to a >1 MiB file became an empty diff and passed; a `-diff`/`binary` attribute or a textconv driver hid added lines too. Diff with --text --no-ext-diff --no-textconv and a 256 MiB buffer, retry per file on failure, and refuse any file git still cannot diff (B3). computeCodeState had the same blind spot (a stale verify PASS survived a later edit when the pending diff exceeded 1 MiB): hash a --binary diff with the same buffer and report "cannot bind" when git fails. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… AUC-PR and sound kill criteria The pre-edit hook passed the predictor only the file path, so four of its seven features were pinned to 0 and risk topped out at sigmoid(-1.0) = 0.27, below the 0.66 "high" band: the advisory could never fire. The hook's risk block now computes caller fan-out and test coverage from one bounded git grep of the module name, whether the edit rewrites an existing declaration, and whether any caller is in the working diff, and names the reasons. A hot file with ten importers, no test and a rewritten signature scores 0.91. predictor.js: aucPr ranks tied scores as one threshold (the same data gave 1.0 or 0.333 depending on input order). The kill criteria no longer decide on a held-out split under 10 samples / 2 per class, and compare AUC-PR with the exact expected AP of a random ranking instead of a fixed 0.6 floor. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…, never raise The M1 reconcile compared the proposer's band floor (0.15/0.40/0.65) with the deterministic point score, so agreement moved the score (fable 0.887 + a Jev "premium" vote -> 0.688 opus, logged "llm-lowered"; 0.431 + "mid" -> 0.400), Jev confidence was ignored (0.34 and 1.00 routed identically), a proposer's self-assessment could raise the tier (contradicting whitepaper 5.1), and the 0.2 point bound blocked confident down-routes from the top of a band. reconcileRoute() maps the score to recommend()'s bands first: same band keeps the score; a lower band moves to its ceiling only when p(band) >= minConfidence (ROUTE_MIN_CONFIDENCE 0.8, an a-priori default, configurable via llm.minConfidence; must be chosen on fresh data) and never below the strong-signal floor; a higher band is recorded as llm.escalateTo and deferred to a verifier failure. routingBand is gone. recommend() now routes a non-finite score to the default tier instead of fable. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…g risk The ring score was a weighted mean in which clean signals (deprecated:false at weight 1.0, "no advisories" at 0.9) counted as zeros and diluted the rest: a dependency 4 majors behind and 3 years stale scored 0.221 (adopt), currency risk was capped at 0.255 so "assess" was unreachable from the score, and a high-severity advisory alone scored 0.247 (adopt). The score is now 1 - prod(1 - w_k * s_k), the same shape as the lesson and consensus scores. The same dependency scores 0.485 (trial); maximal currency risk (0.545) and a high advisory (0.630) reach assess. Absent evidence still lands in assess via the evidence-count gate. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ities only floor clearing With the LLM layer on, reconcileAssumption put hasUnresolved in the ask condition itself, so a task the rubric proceeded on and the model judged complete was still asked (a grounded rename with a background URL -> path "llm-tightened"); tighten-only mode ignored it. It is now a floor on clearing a rubric ask only, identically in both modes. The reconcile also clipped the proposer's reading to det +/- 0.25, but the rubric's logistic saturates (median 0.983 on the held-out tasks) while Jev's mean noul centres on 0.5, so a stub at Jev's median pinned 74/80 values at det - 0.25. Each reading is now judged against its own threshold and only the verdicts are compared; the proposer flips the rubric only at p >= minConfidence (GATE_MIN_CONFIDENCE 0.8, a-priori, llm.minConfidence). The rubric's completeness is reported unchanged, the proposer's rides in provenance.proposalCompleteness, and the substrate `band` key is removed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`(node "$entry" stop &)` in a non-interactive shell gets /dev/null as stdin, so every Stop payload was empty, the session id fell back to "default", and the real session was never processed: no episodes, no lessons, no deja summary, and the session log was never cleared. Read the payload before detaching and pipe it into the background node. Add a shim-level test that drives `node run.mjs cortex.sh ...` the way Claude Code does (the existing tests piped into cortex_hook_main.js directly and could not see this). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…nd MCP never confirm Security fixes from review C2: - refStrength: "resolved" now means forge re-derived the pointer — a git object id (resolved at append/import, re-resolved by verify) or a bridge pointer on its own bridge oracle (episode:/cortex.episode, legacy:/legacy.import). Untyped and unknown-prefix refs, ci:, human:, test:, file: and symbolic git:HEAD count at format strength and are capped below the serving floor. An agent: identity never supplies human-family evidence at resolved strength. - reuseQuery no longer appends a graph.reval confirm on every serve; only a failed revalidation is written back. mintArtifact reports `serves` from the val the proof actually earns. - MCP forge_ledger_ratify / forge_ledger_retract are stamped agent:mcp and only propose: ratify mints a distinct agent-proposed decision (no val change); retract requires one exact 64-char id and records a pending retraction proposal (claim stays live) surfaced by ledger query, stats and show. `forge ledger retract` (human, CLI) now also requires the full id. - getClaimByPrefix refuses an ambiguous prefix instead of returning the first sorted match. BREAKING CHANGE: evidence citing untyped/ci:/human:/session: refs no longer lifts a claim past 0.55; cite a git object id for full-strength evidence. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…tions research/recompute_corrections.py re-derives every number used in the 2026-09-21 corrections to the research papers: Theorem D and Eq. 5 arithmetic, the repo-cluster bootstrap (seed 1234, 20,000 resamples), the paired held-out comparison, cost per judged-correct output, the calibration bins and the label agreement. It is standard-library Python with a minimal Parquet reader, so it runs without numpy, scipy or pyarrow against the extracted replication package. The ARCHITECTURE.md repo-map block is re-rendered for the new file. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…definitions An external deep review (2026-09-21, findings F1-F4, F6) found the formal synthesis's central theorem circular as stated and its Eq. 5 dependent on an independence the design contradicts. The HTML edition now: - states the residual as (1-p)·P(no check fires | miss) with Frechet bounds, and Theorem D as residual <= eps over an explicit (p, q) region with the task-dependence assumption stated (union bound); - corrects the lattice of gates: one classifier at Stop, pre-commit and CI gives (1-p)(1-c_max) = 0.015, not the product's 3.75e-5; - shows c_j depends on the agent (a STATE.md touch passes the gate; c = 0.95(1-h), residual 0.27 at h = 0.9; c = 0 after the one block); - fixes lfp(X -> X u N(X)) = empty set, downgrades the oracle-is-Delta* identity, and corrects T3-T6, A1's type error, A3's definitional I1, the B = 150 lines vs 8 KB cap, and the use of Rice's theorem; - reconciles the faculty table with the whitepaper and Eq. 1 with the amnesia equation, concedes priority, marks the section 10 prototype results as refuted, says the convergence was not independent, and fixes the Appendix A tally (9 confirmed). Every change is marked in place and listed with the original wording in a dated Corrections section. crosswalk.json/.md and the README follow; the audit report gets a dated editorial note. The PDF predates this. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The external review (2026-09-21, finding F5 and parts of F1-F4, F6) recomputed the refutation from its replication package. The counts reproduce exactly; some inferences did not. The LaTeX source and the extended preprint now: - report repository-cluster bootstrap intervals, because every ground-truth pair is mirrored (20,144 of 20,144) and files cluster in nine repositories: oracle precision [0.15, 0.91], recall [0.0005, 0.052] (seed 1234, 20,000 resamples); the refutation survives (oracle F1 [0.001, 0.093] vs grep [0.381, 0.539]); - stop claiming the repaired oracle beats grep: paired dF1 +0.044, 3/3 held-out repositories (sign test p = 0.125), pytest 71% of pairs, and the relation choice was made on all nine repositories; - say the gold labels and the second pass are one model (kappa measures prompt robustness), add cost per judged-correct output ($1.06 vs $1.76, from 6 and 3 of 64), and qualify the 96.8% ceiling; - fix the calibration paragraph (bins 27/5/28/4/16, ECE 0.103 or 0.078, p = 0.028) and explain 801 labelled vs 759 evaluated files. The extended preprint also receives the synthesis's theory corrections (its theory sections are shared) and drops its remaining "this is why the prototype achieves perfect recall". Both papers end with a dated Corrections section; paper.pdf and extended_preprint.pdf predate it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The cognitive-substrate whitepaper was written before its prototypes were tested on real data. Following the external review (2026-09-21), its HTML edition (and the identical docs/ copy) now carries a status banner, inline markers and a dated Corrections section for: - the impact oracle's "never misses an affected file" / "guaranteed recall" (recall 0.022 on 759 files in nine repositories), including the "sound over-approximation" wording in section 7.1; - the 62.1% routing saving (tuning set only; -20.2% on 80 held-out tasks, $1.06 vs $1.76 per judged-correct output); - M1's worst-case cost, which is the sum over every tier escalated through, not a cheap plus a premium attempt; - Eq. 1 (deterministic) vs its gloss (sampled) vs M2 (argmax), now one statement: y ~ p_theta(.|x), greedy decoding as the argmax case; - the misquoted Faros figure: "31.3% more PRs merged with no review". Its faculty table is kept as the canonical one. The executive summary's banner no longer calls the repair a win; docs/cognitive-substrate/ deliverable-package.md, which had no banner at all, gets one; the Python impact oracle's README no longer claims perfect recall. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Documentation side of review items A7 and E4 (2026-09-21): - The impact graph under-approximates, so README, GUIDE, the Mintlify intro, the capability map, the substrate-v2 plan and source/substrate.json's limits no longer call it conservative; an empty impact set is "unknown", not "safe". - The README's impact-quality row (precision 0.90, F1 0.92) does not reproduce: evalImpact on the same six cases gives precision 0.34, recall 0.97, F1 0.50 at 1a82388. The README and a note beside the generated block in reports/benchmarks.md say so and carry TODO(impact-numbers) markers for the post-fix re-measurement. - The prototype rows (recall 1.000; 62.1% saved) now sit beside their real-data refutations (recall 0.022; total spend 20.2% higher), and the README, GUIDE, reports/cost-eval.md and the plan stop presenting 62% as a result. The README keeps the "**62.1% cost saved**" label the status-page generator parses, now marked as refuted. - research/README.md gains a Corrections section (what changed, how to recompute it, which PDFs are stale) and stops calling the repaired oracle "a real but narrow win". Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… and 62% claims The ccusage-less spend estimate ignored cache_creation_input_tokens and cache_read_input_tokens (most of Claude Code's input) and summed every log line, although one response is logged on several lines sharing a message id: a one-message fixture logged three times estimated $0.038 against $0.228. Cache writes are now priced at 1.25x input (2x for 1-hour writes), reads at 0.1x (Anthropic's caching multipliers; the price table carries base rates only), and each message id counts once across all log files. forge cost --stages called its composition a lower bound that can only grow, but the route factor goes negative when routing prices above the always-premium baseline; it is now "measured stages only, not a bound". The paper's 62% routing saving is printed only as refuted, next to the measured -20.2% on total spend. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…acts on it A deferred higher-band vote is reported as llm.escalateTo, but no code path consumes it: nothing escalates automatically after a verifier failure. The route reason text, reconcileRoute JSDoc, substrate guarantees, GUIDE, cognitive-substrate README, substrate.json note and CHANGELOG now call it an advisory recommendation instead of implying a wired escalation path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The prompt and capture hooks wrote raw prompts and Bash commands to .forge/sessions/<sid>.jsonl, so a pasted GITHUB_TOKEN=... or a curl with an Authorization: Bearer header sat on disk verbatim, and `forge init` never gitignored the directory. Redact every string of an event before it is appended, and have init write a nested .forge/.gitignore listing sessions/ (the user's root .gitignore is left alone, so a committed ledger stays committable). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A ref-mode push replaced state.json with the pushing replica's verified state, so records it had to quarantine (a file: proof only a teammate has, an unfetched commit) vanished from refs/forge/ledger for everyone. Push now writes mergeStates(remoteRaw, localVerified); verification happens only on read, so the shared ref only ever grows. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
shadowFact v1 -> v2 -> v1 re-used v1's permanently tombstoned id, leaving no live fact. A retired value is re-asserted as the lowest non-tombstoned rev (deterministic, so teammates converge), and reconcileFacts matches store and ledger by content instead of by rev-0 id. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…s 4th optional item rank.history summed val() over lesson claims AND every deja session summary naming a file. A summary is minted for every session, first-try successes included, and a session whose own tests passed carries a confirm outcome: every edit became an "incident", five ordinary sessions added 2.5 to a file's history, and a tested, passing session added 0.64 against an untested one's 0.5. Only lesson claims (recorded mistakes) count now. context.assemble checked its per-source diminishing-returns cut after taking the item, against a 0.2 floor, so it took six items from one source before stopping — and stopped with a `break`, ending the fill for every other source too. The check now runs before taking an item, at the floor the comment describes (the 4th item, delta^3 ~ 0.34), and skips only that source: 10 candidate facts yield 3. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…confirm-only rec Review C5 + C8 + the future-dated half of C11: - Scope moves into the linear term (s = 0.10) instead of multiplying σ from outside, where it was a strict priority over relevance. - rel = max(4-token-shingle Jaccard, query-term coverage), so a 2-3 word query finds its fact instead of scoring 0. - Tokenization is Unicode-aware and an empty token set no longer "agrees" with every other empty set (any two non-ASCII texts scored Jaccard 1). - rec keys on confirmations (or the mint), so a contradiction no longer raises a stale claim's score, and evidence ages by its distance from now, so a future-dated record cannot pin rec at 1. - retrieve picks one similarity backend per ranking — cosine only when every candidate is embedded — instead of mixing cosine with Jaccard. - deja gates on rel (DEJA_REL_FLOOR) instead of the total score, which fired an unrelated symbol-scoped lesson on every prompt. - EQ3_WEIGHTS no longer claims a P8 calibration that was never run; the PCM spec §4 is updated to match. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…rences
The entity scanner paired a code fence's third backtick with the next inline
backtick, so every word inside a fence became an identifier (a broker-URL log
line yielded "Setting", "up", "for", "broker") and each one reached the
substring git grep behind routing fan-out. URLs, markdown links/images, "N/A"
and "and/or" counted as files. The concreteness anchors fired on URLs, image
links, contractions ("'t break it, it'") and versions ("since v2.3:"), while a
named identifier counted for nothing, so "Rename getUser to fetchUser
everywhere" was hard-flagged as having no concrete anchor.
Fences are stripped before the inline-code scan and an inline span needs a
closing run of equal length; stripUrls removes addresses before every scan; a
bare slash token must look like a path; the quoted/filename/worked-value
anchors are tightened and a named identifier is now an anchor; the
success-criteria cue matches \btest, not "latest".
Diagnostic on the spent 80-task held-out set: entities on the 64
well-specified tasks fall from 210 files / 1,414 symbols to 42 / 388, and the
filename anchor's firing rate on gold-ask tasks from 0.69 to 0.31.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…cosine
slug() kept only [a-z0-9], so every non-Latin name ("مفتاح الواجهة",
"数据库地址") slugged to "" and fell back to the callers' shared literal:
two facts with different names overwrote each other under one "fact" slug.
It is Unicode-aware now (NFKC, letters/marks/digits of any script), with a
short content hash for names carrying no letter or digit at all. ASCII
slugs are unchanged; knowledge_router's factName("???") now gets that
hash instead of the shared "fact".
clamp01(NaN) returned NaN and poisoned every score it fed; it fails to 0,
as does any non-numeric input.
cosine() promised "never NaN" but squared components before dividing:
components above ~1e154 overflowed to Infinity/Infinity = NaN and
components below ~1e-162 underflowed to a false zero vector. It now scales
each vector by its largest component, rejects non-finite components, and
clamps the result to [-1, 1].
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The landing page's evidence ledger and the generated status page both showed "62.1% cost saved" as a measured outcome. That figure comes from the 30-task Python demonstration whose thresholds were tuned on those same tasks; the pre-registered 80-task held-out run spent 20.2% MORE than always-premium. Both surfaces now carry the held-out number, and the README snapshot label build-pages.mjs lifts changed with them. Per judged-correct output ($1.06 vs $1.76) is stated in the README where it can carry its boundary. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ning - claimToLesson derives `active` with hysteresis (on at 0.6, off below 0.55), so one confirm no longer flips to inactive after one day of decay. - isDormant latches: once val drops below the floor at an evidence event, only a later confirmation clears it — decay alone stops reviving claims a human revert refuted. - pruneLedger implements the spec's forgetting rule (tombstoned or dormant with nothing new for 2·T → attic) and is wired into the Stop-path ledger write, `ledger merge` and sync's import. putClaim/appendRecord are attic-aware: a re-import never un-prunes, new evidence does. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…thon package roots
scope.js resolved only six regex shapes against a candidate list that could not
express TypeScript NodeNext (`./x.js` → `x.ts`) and gave Python zero edges: its
`from .y import` pattern fed a dotted module name to a path resolver. Imports written
inside comments or strings counted as imports.
It now owns the ONE import parser the rest of the graph uses:
- `maskCode()` blanks comments and string/regex CONTENTS (offsets, lengths and line
structure preserved, delimiters kept) for JS/TS, Python, Ruby, PHP, Go and the
C-family, including template-literal interpolation and regex-vs-division.
- `jsImports()` reads static imports with multi-line clauses, `export * from`,
`export { x as y } from`, dynamic `import()` and `require()`, with their named
bindings; `pyImports()` reads relative, parenthesised, backslash-continued and
aliased dotted imports, one statement at a time.
- `resolveSpec()` (now exported and shared with atlas.js) resolves exact, NodeNext
twin, extensionless and `<dir>/index.*` specifiers; `pyModuleIndex()` /
`resolvePyImport()` resolve Python modules by PACKAGE ROOT, so `src/mypkg/core.py`
is `mypkg.core` in a src layout exactly as in a flat one, with a prefix-stripped
fallback only for dotted names and only when unique (so `import json` cannot bind
to a local `tools/json.py`).
Measured on the fixtures: the file graph goes from 0 to 7 edges on the Python
fixture and from 0 to 2 on the TypeScript NodeNext one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…s, walk siblings
The deep review's findings A1–A6 on the impact graph, all reproduced first on the
unmodified code and each pinned by a regression test in test/atlas_resolve.test.js
and test/atlas_relations.test.js (fixtures: test/fixtures/impact_repos.mjs).
A1/A3/A4/A5 — imports are resolved STRUCTURALLY through scope.js instead of being
stored as raw strings and matched by their last dotted segment. On this repo 3 of
502 relative import statements resolved and all three were spurious ("../scripts/
build-pages.mjs" → `mjs` → `const mjs` in doctor.js); now 679 of 679 relative
statements (of 1,392 seen, up from 1,196 — multi-line, `export … from`, dynamic and
`require` forms were invisible) resolve to the exact file the specifier names, none
wrong. Python modules resolve by package root, so a src layout answers like a flat
one, and an unresolvable import stays unresolved instead of being pinned to a
same-named symbol — in particular never across languages (a Python
`from impact_oracle.oracle import …` used to land on the JS `const oracle`).
A6 — every structural regex now runs on masked source, so a comment saying
`class Parser` no longer defines a phantom that makes the name ambiguous and erases
a real edge; a call is attributed to its innermost enclosing function instead of the
nearest preceding local `const`, restoring transitive callers; ambiguous references
are counted (3,136 → 877 here) and surfaced with the cap and unresolved-import counts
in `impact()` and `forge impact`; the file cap counts source files only; and
extraction is linear — 16k lines of JS plus 16k of Python build in 0.3 s, down from
4.9 s. ATLAS-Q: `query()` ranks exact definitions above path-only qname matches.
A2 — the reverse-only traversal is extended with the empirical refutation's SIBLING
and FORWARD relations at the replication package's FROZEN parameters (sibling: one
forward hop to the shared dependency's file, one reverse hop back, weight 0.7, bridge
in-degree cap 100; forward: ≤2 hops, weight 0.5), both terminal. `impact(serializer)`
now reports the `deserializer` that shares `wire_format` with it; every item carries
its `relation`, and `relations: ["reverse"]` reproduces the old answer exactly.
The atlas artifact is version 3: an older `.forge/atlas.json` or per-file cache is
rebuilt rather than trusted, since it holds unresolved specifiers.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…on fix Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ked files, doom sig Review C10: - The preflight hook measures goal drift against the PERSISTED goal instead of the current prompt, and makes no drift claim when no goal is set. - The CUSUM series gets per-checkpoint increments (what moved since the last prompt) instead of the cumulative off-goal ratio, so one static off-goal file no longer alarms by itself after three idle prompts. - M5 lean counts untracked files — where over-engineering actually lives. - The doom-loop signature hashes stdout AND stderr over the whole normalized output (head+tail above 64 KB) instead of the first 800 chars of stdout, and the advisory no longer claims edits were made when none were. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Without jq every guard fell back to a regex that cut the value at the first escaped quote, so `echo "x"; cat .env` and `git diff -- ".env"` walked past the deny (the two protect-paths tests that fail on a jq-less machine), and the status line lost every segment but the dir. `printf | grep -q` under pipefail also lost matches to SIGPIPE on large commands, and internal errors exited 1 — a NON-blocking hook error. - protect-paths.sh is now a thin launcher over protect-paths.mjs (the secret-redact split): one parser, pure rule set, deny on any internal error or unparsable payload. - _guardlib.sh + statusline.sh read fields through guards/hookfield.mjs (jq when installed, node otherwise); no regex path remains. - rules: git reset --hard, git clean -f, find -delete/-exec rm, chmod -R, dd of=, lowercase SQL; --force-with-lease allowed again; .aws/credentials, .netrc, .npmrc, .git-credentials protected. - both manifests run protect-paths on Read (plugin installs ship no permissions.deny), and the template denies the credential stores. - resolveBash skips …\Microsoft\WindowsApps\bash.exe (the WSL alias that made every guard exit 127). - settings allowlist: drop Bash(fd:*) (fd -x runs anything), narrow git branch to its read-only forms, deny git branch -d/-D and git diff --output (B8). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…(B7)
The provenance stamp and the block-once marker are files under .forge/,
so a hand-written {"tests":{"status":"PASS"}} satisfied the strong leg
and a pre-written <sid>.blocked switched the gate off. Both now carry a
MAC over what they claim, keyed by a machine-local key outside the repo
(mode 0600, created on first use); unsigned evidence is ignored. Not a
boundary — an agent with shell access can read the key — but forging is
no longer a side effect of writing one file in the project.
Also: a comment-only touch to a test file is no longer test evidence
(the added lines must contain code); a package.json test script that
masks failures (`node --test || true`) reports INCOMPLETE, not PASS;
skillgate runs the heuristic ALONGSIDE the external scanner instead of
letting a clean scanner exit replace it; imagine reads the runner's own
(last) TAP summary and cross-checks the exit code.
verify: classifySuiteFailure is now pure, so the "killed by a signal ⇒
INCOMPLETE" case is checked on every OS — the old fixture needed a
POSIX shebang script that Windows cannot run, which is why that test
failed there (ME-02).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ny list (B8) The cost governor only wrote to stderr and exited 0, which a PreToolUse hook shows to nobody: past the FORGE_COST_CEILING it now asks the human (permissionDecision: "ask") and the nudges ride along as additionalContext. `forge harden` emitted a `credentials.deny` key that Claude Code reads nowhere; it now writes permissions.deny with real Read(<glob>) rules. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
forge_field spawned a parser per field, and node costs ~1s to start on Windows — four fields in doom-loop meant four spawns. Load the seven common fields once per guard process and serve the rest from that cache (jq, when installed, stays per-call: it starts in milliseconds). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The shell prefilter decides whether the Node redactor runs at all, so a shape it misses is never scanned. A URL credential has no 20+ char token run and no key-ish assignment, so `postgres://app:pw@host` skipped the redactor entirely; a SCREAMING_SNAKE `DB_PASSWORD=` missed the case-sensitive branch. Match case-insensitively and add the URL-userinfo, glpat-, ASIA, apikey_ and auth/credentials shapes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…mmand `git\s.*push\s.*--force` matched prose too: a commit message mentioning "push -f" was blocked. Anchor the push/reset/clean rules the way the reader rules are anchored (optional env/command/sudo prefix, a path, then git's global options), and cover `sudo git push --force`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…annel (B8) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ed UI verdicts Review C11 (the remaining items): - LEDGER.md rows are stable (no per-row val, which changed with the clock and with each replica) and the ledger ships a nested .gitattributes marking the generated index merge=union — two teammates' facts merge cleanly now. - loadState keeps a derived snapshot beside the ledger, validated by a stat-only fingerprint of every file's (path, size, mtime), so the per-prompt hooks stop re-reading 900 files three times: one loadClaims 619 ms → 112 ms and the hook path 3026 ms → 517 ms on a 300-claim ledger. Any external edit rebuilds it from the files; the cache is gitignored and disposable. - uiinteract records a verdict with today's day instead of epoch day 0, so five failing UI runs move the fingerprint's val (0.5000 → 0.3636) instead of decaying to nothing on arrival. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
# Conflicts: # reports/cost-eval.md
# Conflicts: # CHANGELOG.md
# Conflicts: # CHANGELOG.md
…ember - The brain index inlined into AGENTS.md never consulted the ledger, so a fact three CI runs had contradicted (dormant) was still shipped to every AGENTS.md-reading tool. Dormant facts are withheld, with the count and the reason stated in the block — never a silent drop. - MCP forge_remember answered "Remembered" even when the store refused the write (a secret-shaped fact); it now reports the refusal. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The guards read their hook payload through node now (guards/hookfield.mjs),
and the grep fallback is gone — it mis-parsed any command containing an
escaped quote, which is the B6 finding. doctor reported jq's absence as a
warning ("falls back to grep parsing (still enforced)"), which was wrong
twice over: nothing falls back, and nothing is missing. The row is now
"na — not needed", and README no longer lists jq as an assumed tool.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…st run `echo "run npm test later"` and `grep -r 'jest' package.json` marked a session "tested" (minting a test.run confirm for a session that ran no tests), and `npm test || true` counted as a pass. A command must now BE a test run — at the start or after a shell separator — and keep its exit code. The same grammar is used by the cortex S1 signal. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
# Conflicts: # CHANGELOG.md # test/cortex_hook.test.js
The repo map is generated from the live import graph, which every area changed; each agent was asked to leave it alone so six branches wouldn't conflict on the same generated block. Rendered once here instead. test/brain.test.js exceeded the line width after the merge. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…merge Two areas edited this file: research rewrote the 62% status paragraph and primitives corrected the composed-reduction claim (E4 — a measured stage can be negative, so the figure is not a lower bound). Resolving the conflict in favour of research's paragraph silently took primitives' correction with it. Restored; both changes now stand. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ng it The Unicode-aware slug (9e24b92) changed the key of any fact whose name stripped to "" under the old rule — those live on disk as the shared "fact.md" fallback. Re-adding such a fact wrote the new key and left the old file in place, so list() and MEMORY.md served a stale twin forever, with no migration step. add() now drops any other file holding the same fact NAME before writing. Found by the independent review of the merged diff; only the legacy file store (FORGE_LEDGER_ONLY=0) was affected — the ledger supersedes by (name, text) and was never at risk. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The P2 read-flip made the ledger the default and only store (util.js
ledgerOnly() defaults true) — legacy lessons/recall/brain files are no
longer written or read unless FORGE_LEDGER_ONLY=0. ARCHITECTURE.md still
described the pre-flip state ("legacy stores remain the read path") and
the repo-layout comments for ledger_bridge.js/ledger_read.js still framed
the bridge as active by default; both now match the code.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
route.js's reconcileRoute() never applies a higher-band vote (whitepaper §5.1) — it only lowers the tier, confidence-gated and bounded, and records what it would have picked as the advisory `llm.escalateTo` field, which nothing acts on automatically. The old routeTask() this replaced did apply a free raise, which is what the README bullet was still describing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…n request The sibling and forward relations ported from the empirical refutation's repaired oracle do what the paper says — they are why a sibling that shares a dependency is no longer missed — but they are a recall instrument. On this repo they take the median answer from 15 files to 78 of ~450 (max 196): recall 1.00, precision 0.09, and the substrate's 25-file blast threshold would trip on nearly every edit. impact() now defaults to DEFAULT_IMPACT_RELATIONS (reverse only). Callers pass `relations` for more; `forge impact --all-relations` is the CLI door, and predictImpact threads it through. The rules and their frozen parameters are untouched — only the default changed. The A2 tests now request the wide walk explicitly, and a new test pins the default so it cannot drift back. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…git exits 0 CI caught what a Windows-only run could not: `git diff --cached` over a staged blob whose object is missing prints `error: unable to read …` and still exits 0 on the Linux runner's git, so the scan saw an EMPTY diff and read it as "this file added no lines" — the exact fail-open B3 closed for oversized diffs. Which git versions exit non-zero for this differs by platform, so gitStrict now trusts the message, not the status: an `error:`/`fatal:` line on stderr throws, which routes the file into `unscanned` and refuses the commit. Warnings (CRLF conversion on a Windows checkout) stay routine and never block. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…-closed case fails The assertion said only 'expected true'. On Linux and macOS the diff of a staged blob whose object was deleted evidently still succeeds, and the message gave no way to tell why. It now carries git's exit status, stderr and the gate's findings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… missing object On the Linux and macOS runners git answered `diff --cached` for the corrupted blob from the worktree file (status 0, empty stderr, full diff), so the staged file was scanned normally and the fail-closed path never ran — the test passed on Windows and failed everywhere else. Removing the worktree copy forces git to read the object it no longer has. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The scan ran `gitleaks detect --source .` with no --config, so the repo's allowlist (synthetic PEM/token fixtures, bibliography citation keys) may never have been applied, and --redact without --verbose printed only "leaks found: 20" — a blocking gate that names nothing is unactionable. The job now loads the config explicitly and lists each finding's path, line and rule, with the secret still redacted. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ing the secret gate All 20 gitleaks findings — on master as well as here — were the same false positive: `"citation_key": "graves2014ntm"` in the two literature datasets, where the field name beside a quoted alphanumeric value trips generic-api-key. Two sibling bibliography files were already allowlisted by path; these were missed, so the gate has been red and unactionable. Allowlisted by SHAPE (an author+year BibTeX key on the matched line), not by file: a real credential added to references.json or gap_map.json is still caught. All 32 keys in those files match the pattern. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the findings of an external deep review of this repo (~134 findings across the impact graph, secrets/guards/gates, memory/ledger, routing/assumption gate, numeric primitives and the research mathematics).
Six areas, developed in parallel on their own branches, then merged here. Every finding was reproduced on unmodified code before it was fixed, and each fix carries a regression test that fails without it.
Gate results on this branch
npm testnpx biome checknpm run typecheck/docs checkfec7950)The acceptance harness was written from the review and the reviewer's repro scripts before any fix landed, and no authoring agent had access to it.
Headline fixes
hasSecretwas cubic (40 KB unfinished after 15 s → 9 ms). URL credentials, AWSASIA…,AUTH=/DATABASE_URL=,Authorization:headers and TypeSafe keys were undetected — now caught and fully redacted. Session logs no longer store raw tokens. Lockfile integrity digests are no longer mistaken for secrets..gitattributes -diff; a leak hidden either way now exits 1.N/Aand code fences as code references.verify --deepreported a 0.042 residual with tests never run and an empty diff (now 1.0); AUC-PR depended on input order; the high-risk advisory could never fire; the cost report ignored cache tokens ($0.038 vs a measured $0.228).Behaviour changes worth knowing
forge impactwalks reverse dependencies by default;--all-relationsadds the paper's sibling/forward rules (median 15 → 78 files on this repo, recall 1.00, precision 0.09).jqis no longer required anywhere; guards parse hook JSON with node and fail closed.val.Not fixed — 14 open items
Recorded with reasons, including: the router's premium recall is still 0/17 (needs new labelled data; the 80-task set is spent on tuning), both confidence thresholds are a-priori, completion-gate evidence is signable by an agent that can read the key file, 5 PDFs are stale (no WeasyPrint/TeX available), and
bench/impact_cases.mjslabels need relabelling before any precision claim is republished.🤖 Generated with Claude Code