Skip to content

fix: credit a workspace run only to the tool itself; mark non-exact reuse as needing review (Q01, Q02) - #170

Merged
CodeWithJuber merged 3 commits into
masterfrom
claude/forgekit-deep-review-issues-ww6y0h
Sep 27, 2026
Merged

CodeWithJuber merged 3 commits into
masterfrom
claude/forgekit-deep-review-issues-ww6y0h

Conversation

@CodeWithJuber

Copy link
Copy Markdown
Owner

What & why

The 2026-09-27 recheck of v1.7.3 (6d98121) confirmed that all eight N01–N08 reproduction cases are closed. It reported two adjacent findings, and this PR fixes both. It also records the recheck's confirmation in the claims registry. All three of the review package's scripts pass with --assert-fixed on this branch:

  • reproduce_remaining.mjs: N01–N08 stay closed.
  • original-reproduce.mjs: F01–F16 stay closed.
  • edge-probes.mjs: Q01 and Q02 no longer reproduce.

Q02 (P1): a local program named like a package manager covered the workspaces.

  • The problem: a root test script of ./tools/npm test --workspaces was credited as npm's recursive run, because recognition read only the file's basename. The fixture's stub exits 0, so packages/bad never ran and forge verify said PASS.
  • Now: a tool is recognized only by its bare name (the program the script's PATH finds) or as its own install under node_modules/.bin. Any other path (./tools/npm, ./scripts/pnpm.js, /usr/bin/env) is an unknown program. The review's fixture now runs the member and reports FAIL, with every basis measured. forge verify prints the root run as "not credited" and names the program.
  • The same conservatism now covers PATH overrides and wrappers:
    • Variables on the command line are allowlisted. PATH, LD_PRELOAD, HOME and a NODE_OPTIONS preload are refused.
    • npx -p/--package, env -i and env -C/--chdir are refused.
    • A bare name must not be shadowed:
      • the first node_modules/.bin entry on the script's PATH (the package's own, then each parent directory's) must be the tool's own package's binary, and a symlink must resolve into that package;
      • a package manager or system program there is a shim;
      • a same-named Windows executable in the root (npm.cmd) counts too, because cmd.exe runs it first.
    • yarn must be a release: a yarnPath outside .yarn/releases/, or a packageManager fetched from a URL, is refused.
    • node-options must be inert, a script-shell that is a project file is not a shell, and nx plugins (which can define test) run the members on their own.
  • Unchanged: standard invocations keep working (npm test --workspaces, turbo run test --force, ./node_modules/.bin/turbo, yarn turbo, CI=1 …), and verify.workspaces: "root" stays labelled declared.

Q01 (P2): opposite requirements were labelled a near "reworded match".

  • The problem: "Deny admins and allow guests…" near-hit an artifact verified for "Allow admins and deny guests…" (MinHash 0.828).
  • Now, the contract: every hit states what it establishes. semanticEquivalence is "identical" only for exact and "unverified" for near and adapt, and requiresReview is true for every non-exact hit. Both fields are in forge reuse query --json and the gate's reuse summary. The CLI, gate and docs no longer call a near hit reworded.
  • Now, known reversals: the semantic guard has a binding kind. Each polarity, negation or direction word binds the next content word of its clause. A word bound to opposite relations, or the same words in another order, is a conflict. Permission-subject, source/destination, into, negation-scope, quoted-literal and possessive role swaps are held at adapt, with the conflict named. Consolidation lists such pairs as conflicts rather than proposals.
  • Limits: the binding check is a token heuristic. It does not prove equivalence, which is why the contract applies to every non-exact hit. A reorder of the same words that changes nothing is also held at adapt. Both tiers require review now, so the cost is a note, never a refusal.

Claims check without release tags. The reviewer's first full run failed once because the clone lacked the newer release tags. Now a claim naming a release newer than every local tag, but not newer than package.json's version, is reported unchecked with a git fetch --tags hint instead of failing. I reproduced the reviewer's condition (a clone with only v1.4.0 and v1.4.3) and verified the fix there.

Claims (second commit):

  • verify-pass-binding and similar-rules-never-merged are re-assessed on cf89040.
  • A new claim, reuse-review-contract, links Q01.
  • The recheck's confirmation of each N01–N08 fixture is recorded on its claim.

Docs are updated in the same change: GUIDE, the Mintlify verify, reuse and memory pages, the reuse-cache plan, and CHANGELOG with the rendered changelog page.

Checklist

  • npm test passes: 1,778 tests, 1,774 pass, 0 fail, 4 platform-gated skips (Node 22)
  • npm run check passes (Biome lint + format; the same 14 pre-existing warnings as master)
  • New public functions have a test: compareVersions, the bins field of recursiveTestInvocation, the review contract fields, and the guard's binding kind
  • Conventional commit messages
  • CHANGELOG.md updated under ## [Unreleased]
  • No new runtime dependency
  • Substrate/docs updated: the gate's reuse summary and forge verify/forge reuse changed, and their docs changed with them

Risk & rollback

  • Risk level: low–medium. Every change is conservative:
    • root runs that were credited before may now run their members on their own, for example a command line that sets a variable outside the allowlist, a yarnPath outside .yarn/releases/, or an nx workspace with plugins;
    • some near hits become adapt.
    • A failure shows up as a slower verify, a named refusal or an adapt note, never as a false PASS.
  • New JSON fields: semanticEquivalence and requiresReview on reuse hits, and bins on recursiveTestInvocation's result. No existing field changed.
  • Rollback plan: revert the two commits. No data migration.

Extra checks (tick if applicable)

  • npm run typecheck passes
  • Input validated at boundaries; errors handled (no swallowing)
  • Authorization/ownership checked (if it touches access): n/a
  • Logs contain no secrets/PII
  • If AI-assisted: I understand it, verified the package APIs, and it has tests

🤖 Generated with Claude Code

https://claude.ai/code/session_01GVVG2VDETWsDxMu6MBWPz2


Generated by Claude Code

…euse as needing review

The 2026-09-27 recheck of v1.7.3 confirmed N01-N08 closed and found two
adjacent cases. Both are fixed conservatively: what cannot be established
runs the members on their own, or is served as a candidate to review.

Q02 (P1), workspace coverage: `./tools/npm test --workspaces` was credited
as npm's recursive run because recognition read only the file's basename,
so a stub that exited 0 hid a failing workspace behind a PASS.
- a tool is recognized by its bare name or its node_modules/.bin install;
  any other path is an unknown program, and the refusal is reported
- variables on the command line are allowlisted (PATH, LD_PRELOAD, HOME
  and a NODE_OPTIONS preload are refused); npx -p and env -i/-C refused
- shadows are refused: a package manager or system program in any
  node_modules/.bin on the script PATH, another package's binary (or a
  link out of the package) under a tool's name, a same-named Windows
  executable in the root, a yarnPath outside .yarn/releases, and a
  packageManager fetched from a URL
- node-options must be inert, a script-shell inside the project is not a
  shell, and nx plugins run the members on their own
- a verify.workspaces "root" declaration stays labelled declared

Q01 (P2), near reuse: "Deny admins and allow guests..." near-hit an
artifact verified for "Allow admins and deny guests..." (MinHash 0.83) and
was called a "reworded match".
- every hit carries semanticEquivalence ("identical" only for exact,
  "unverified" otherwise) and requiresReview (true unless exact), in
  `forge reuse query --json` and the gate's reuse summary
- the semantic guard's new `binding` kind holds opposite bindings
  (allow->admins vs deny->admins, from/to swaps) and same-word
  rearrangements at adapt; consolidation lists such pairs as conflicts
- the CLI, the gate and the docs no longer call a near hit reworded

The claims check no longer fails in a checkout that lacks newer release
tags: such a claim is reported unchecked (fetch the tags), not failed.

Docs (GUIDE, Mintlify, the reuse plan), CHANGELOG and the rendered
changelog page are updated in the same change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GVVG2VDETWsDxMu6MBWPz2
The 2026-09-27 recheck of 1.7.3 (6d98121) independently confirmed the
N01-N08 fixtures closed, and found Q01 and Q02, fixed in cf89040.

- verify-pass-binding: re-assessed on cf89040 with Q02 as a regression
  test; the scope now says forge trusts installed tools only past the
  identity and shadow checks
- similar-rules-never-merged: re-assessed on cf89040, since the guard's
  binding kind now lists the N02 pair as a conflict rather than a proposal
- reuse-review-contract (new): non-exact hits carry semanticEquivalence
  "unverified" and requiresReview; known relational reversals are held at
  adapt (Q01)
- route-outcome-provenance, phase-p3-reuse-cache, reuse-exact-identity,
  phase-p4-context-assembly, context-completeness: the recheck's
  confirmation recorded; their assessment is unchanged

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GVVG2VDETWsDxMu6MBWPz2
Creating a symlink needs elevation on Windows; the repo's other symlink
tests skip there for the same reason. The check moves into its own test
with the same skip, so a non-elevated Windows checkout runs the rest of
the Q02 shadow tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GVVG2VDETWsDxMu6MBWPz2
@CodeWithJuber
CodeWithJuber marked this pull request as ready for review September 27, 2026 05:17
@CodeWithJuber
CodeWithJuber merged commit e04e529 into master Sep 27, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants