Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
2d88d19
test(coverage): specify nested npm metadata lock validation
seonghobae Aug 6, 2026
fe9b9ac
ci: add nested npm materializer trigger
seonghobae Aug 6, 2026
5818e1d
ci: trigger nested npm metadata materializer
seonghobae Aug 6, 2026
252ee50
ci: add PR 807 materializer blank-line repair
seonghobae Aug 7, 2026
f6d882d
ci: trigger PR 807 materializer repair
seonghobae Aug 7, 2026
58be8a5
chore(coverage): remove npm metadata materializer trigger
seonghobae Aug 7, 2026
acbd31d
chore(coverage): remove PR 807 repair trigger
seonghobae Aug 7, 2026
cad3701
chore(coverage): remove npm metadata materializer workflow
seonghobae Aug 7, 2026
e08ec3c
chore(coverage): remove PR 807 repair workflow
seonghobae Aug 7, 2026
bf03df1
chore(coverage): remove npm metadata trigger workflow
seonghobae Aug 7, 2026
b0b1bad
test(coverage): define canonical-pin contract for nested npm metadata
seonghobae Aug 7, 2026
d530b96
ci(coverage): add permanent nested npm metadata quality gate
seonghobae Aug 7, 2026
6f3b265
chore(coverage): implement nested npm metadata pins once
seonghobae Aug 7, 2026
ceec6d7
chore(coverage): remove PR 807 implementation writer
seonghobae Aug 7, 2026
59edad3
chore(coverage): stage reviewed nested metadata implementation
seonghobae Aug 7, 2026
27abdd8
chore(coverage): apply reviewed nested metadata implementation
seonghobae Aug 7, 2026
53ba3a9
ci(coverage): run bounded nested-metadata finalizer
seonghobae Aug 7, 2026
4f5bd6b
chore(coverage): remove PR-controlled npm metadata writer
seonghobae Aug 7, 2026
f12600f
chore(coverage): remove PR-controlled npm metadata patcher
seonghobae Aug 7, 2026
2e228b6
fix(coverage): resolve nested npm metadata through canonical pins
seonghobae Aug 7, 2026
c7e55b5
docs(coverage): define canonical npm metadata pin inheritance
seonghobae Aug 7, 2026
1cffd3f
chore(changelog): record canonical npm metadata pin validation
seonghobae Aug 7, 2026
37c134c
fix(coverage): preserve legacy npm diagnostics before version gate
seonghobae Aug 7, 2026
ebfe966
test(coverage): complete npm validator branch evidence
seonghobae Aug 7, 2026
b4ada29
docs(coverage): align npm explicit-port policy
seonghobae Aug 7, 2026
d046966
test(security): reject symlinked materialization parents
seonghobae Aug 7, 2026
f36c686
fix(security): reject symlinked materialization paths
seonghobae Aug 7, 2026
c556945
fix(security): preserve symlink rejection contract
seonghobae Aug 7, 2026
0ca2653
test(coverage): exercise non-directory output component rejection
seonghobae Aug 7, 2026
1ba1a01
chore(npm): stage descriptor-safe output part 00
seonghobae Aug 7, 2026
583f864
chore(npm): stage descriptor-safe output part 01
seonghobae Aug 7, 2026
c7b3adc
chore(npm): stage descriptor-safe output part 02
seonghobae Aug 7, 2026
0bf64ad
chore(npm): stage descriptor-safe output part 03
seonghobae Aug 7, 2026
4c745d1
chore(npm): stage descriptor-safe output part 04
seonghobae Aug 7, 2026
2d36a26
test(coverage): reproduce JavaScript materializer output races
seonghobae Aug 7, 2026
30acf12
test(coverage): run JavaScript output-race regressions
seonghobae Aug 7, 2026
b7ce2bf
fix(coverage): pin JavaScript materializer output descriptors
seonghobae Aug 7, 2026
7865258
test(coverage): close JavaScript materializer security branches
seonghobae Aug 7, 2026
9247a9c
test(security): prove descriptor ancestry and cleanup races
seonghobae Aug 7, 2026
899500d
ci(security): execute descriptor ancestry regressions
seonghobae Aug 7, 2026
e951b01
fix(coverage): harden JavaScript materializer descriptor ancestry
seonghobae Aug 7, 2026
4c9fcd1
test(coverage): preserve dir-fd capability under race instrumentation
seonghobae Aug 7, 2026
5fef3cb
test(coverage): preserve bounded descriptor-open failure
seonghobae Aug 7, 2026
f65f345
test(coverage): cover descriptor cleanup edge branches
seonghobae Aug 7, 2026
813edc9
test(coverage): execute descriptor edge regressions
seonghobae Aug 7, 2026
44608d7
test(security): prove ancestor creation and complete rollback
seonghobae Aug 7, 2026
6b94070
test(coverage): exercise existing relative directory
seonghobae Aug 7, 2026
42961d0
test(coverage): execute existing-directory regression
seonghobae Aug 7, 2026
562bdda
test(security): remove capability-masking fixture
seonghobae Aug 7, 2026
9d8a57f
fix(security): separate dir-fd capability identity from instrumentation
seonghobae Aug 7, 2026
e63b307
fix(materializer): pin output creation and rollback
seonghobae Aug 11, 2026
8d25ba0
docs(coverage): cite RFC 3986 for npm registry origin pins
seonghobae Aug 13, 2026
cb4596c
fix(coverage): accept only bounded relative requirement includes
seonghobae Aug 13, 2026
362479d
fix: harden Python lock materialization writes
seonghobae Aug 20, 2026
cf4abba
Merge branch 'main' into fix/npm-nested-metadata-lock-validation
opencode-agent[bot] Aug 20, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
136 changes: 136 additions & 0 deletions .github/workflows/npm-nested-metadata-validation-quality-ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,136 @@
name: npm Nested Metadata Validation Quality CI

on:
pull_request:
branches: [main]
paths:
- ".github/workflows/npm-nested-metadata-validation-quality-ci.yml"
- "scripts/ci/materialize_base_javascript_packages.py"
- "tests/conftest.py"
- "tests/test_javascript_materializer_creation_cleanup_security.py"
- "tests/test_javascript_materializer_descriptor_ancestry.py"
- "tests/test_javascript_materializer_existing_directory_coverage.py"
- "tests/test_javascript_materializer_output_edge_coverage.py"
- "tests/test_javascript_materializer_output_security.py"
- "tests/test_materialize_base_javascript_packages.py"
- "tests/test_npm_nested_metadata_lock_validation.py"
- "docs/doctoring/npm-nested-metadata-canonical-pins.md"
- "requirements-opencode-review-ci-hashes.txt"
- "CHANGELOG.md"
push:
branches: [main]
paths:
- ".github/workflows/npm-nested-metadata-validation-quality-ci.yml"
- "scripts/ci/materialize_base_javascript_packages.py"
- "tests/conftest.py"
- "tests/test_javascript_materializer_creation_cleanup_security.py"
- "tests/test_javascript_materializer_descriptor_ancestry.py"
- "tests/test_javascript_materializer_existing_directory_coverage.py"
- "tests/test_javascript_materializer_output_edge_coverage.py"
- "tests/test_javascript_materializer_output_security.py"
- "tests/test_materialize_base_javascript_packages.py"
- "tests/test_npm_nested_metadata_lock_validation.py"
- "docs/doctoring/npm-nested-metadata-canonical-pins.md"
- "requirements-opencode-review-ci-hashes.txt"
- "CHANGELOG.md"

concurrency:
group: npm-nested-metadata-quality-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

permissions:
contents: read

env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true

jobs:
python-310-compatibility:
name: Python 3.10 compatibility
runs-on: ubuntu-24.04
timeout-minutes: 10
steps:
- name: Harden runner
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
- name: Checkout exact source revision
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
persist-credentials: false
- name: Set up Python 3.10
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.10"
- name: Compile implementation and contracts
run: |
python -m compileall -q \
scripts/ci/materialize_base_javascript_packages.py \
tests/conftest.py \
tests/test_javascript_materializer_descriptor_ancestry.py \
tests/test_javascript_materializer_existing_directory_coverage.py \
tests/test_javascript_materializer_output_edge_coverage.py \
tests/test_javascript_materializer_output_security.py \
tests/test_materialize_base_javascript_packages.py \
tests/test_npm_nested_metadata_lock_validation.py

python-314-quality:
name: Python 3.14 complete quality
runs-on: ubuntu-24.04
timeout-minutes: 30
steps:
- name: Harden runner
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
- name: Checkout exact source revision
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
persist-credentials: false
- name: Set up Python 3.14
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.14"
cache: pip
cache-dependency-path: requirements-opencode-review-ci-hashes.txt
- name: Install exact hash-locked quality tooling
run: >-
python -m pip install --disable-pip-version-check --require-hashes
-r requirements-opencode-review-ci-hashes.txt
- name: Run focused tests with complete production branch coverage
run: |
python -m coverage erase
python -m coverage run --branch -m pytest -q \
tests/test_javascript_materializer_creation_cleanup_security.py \
tests/test_javascript_materializer_descriptor_ancestry.py \
tests/test_javascript_materializer_existing_directory_coverage.py \
tests/test_javascript_materializer_output_edge_coverage.py \
tests/test_javascript_materializer_output_security.py \
tests/test_materialize_base_javascript_packages.py \
tests/test_npm_nested_metadata_lock_validation.py
python -m coverage report \
--include=scripts/ci/materialize_base_javascript_packages.py \
--show-missing \
--fail-under=100
- name: Enforce complete production docstrings and compilation
run: |
python -m interrogate --fail-under 100 scripts/ci/materialize_base_javascript_packages.py
python -m compileall -q \
scripts/ci/materialize_base_javascript_packages.py \
tests/conftest.py \
tests/test_javascript_materializer_creation_cleanup_security.py \
tests/test_javascript_materializer_descriptor_ancestry.py \
tests/test_javascript_materializer_existing_directory_coverage.py \
tests/test_javascript_materializer_output_edge_coverage.py \
tests/test_javascript_materializer_output_security.py \
tests/test_materialize_base_javascript_packages.py \
tests/test_npm_nested_metadata_lock_validation.py
- name: Run complete central regression suite
run: |
python -m coverage erase
python -m coverage run -m pytest tests -q
python -m coverage report --show-missing --fail-under=100
- name: Verify clean patches
run: git diff --check
1 change: 1 addition & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,4 @@ Materialize accepts only exact SHA-256 pins or a bounded relative `-r` include (
Conflict-scope roots fail closed when the immediate parent directory is a symbolic link.
OriginWeave hourly NVIDIA NIM repair is a thin caller at minute 10. See [`docs/doctoring/originweave-hourly-review-caller.md`](docs/doctoring/originweave-hourly-review-caller.md).
nonnest2 hourly NVIDIA NIM repair is a thin caller at minute 16. See [`docs/doctoring/nonnest2-hourly-review-caller.md`](docs/doctoring/nonnest2-hourly-review-caller.md).
Npm nested metadata pin rules are recorded in [`docs/doctoring/npm-nested-metadata-canonical-pins.md`](docs/doctoring/npm-nested-metadata-canonical-pins.md).
28 changes: 27 additions & 1 deletion ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,30 @@ Product callers stagger Clearfolio at minute 23, DiskSage at minute 37, and
fast-mlsirm at minute 49. Each caller is read-only, dispatches at most one
repair, and delegates all privileged logic to the same sealed scheduler.

## Nested npm metadata pins

```mermaid
flowchart TD
Entry["packages map entry"]
Link{"workspace link?"}
Fields{"resolved and integrity?"}
Root{"canonical root?"}
Accept["Accept after HTTPS SHA-512 pin"]
Reject["Fail closed"]

Entry --> Link
Link -->|"yes"| Accept
Link -->|"no"| Fields
Fields -->|"both"| Accept
Fields -->|"one"| Reject
Fields -->|"neither"| Root
Root -->|"yes"| Reject
Root -->|"no"| Accept
```

An explicit port, userinfo, query, or fragment is not the default npm
registry origin. Publication uses no-follow, descriptor-relative opens.

## Control-plane data flow

```mermaid
Expand Down Expand Up @@ -125,4 +149,6 @@ trusted `uv` exporter is downloaded from the literal GitHub Releases URL for
- [`docs/doctoring/hourly-nvidia-nim-autofix.md`](docs/doctoring/hourly-nvidia-nim-autofix.md)
— current increment's repair-worker decision and APA 7th citations.
- [`docs/doctoring/fast-mlsirm-hourly-review-caller.md`](docs/doctoring/fast-mlsirm-hourly-review-caller.md)
— product-specific psychometric repair heartbeat and scientific gates.
— product-specific psychometric repair heartbeat and scientific gates.
- [`docs/doctoring/npm-nested-metadata-canonical-pins.md`](docs/doctoring/npm-nested-metadata-canonical-pins.md)
— current increment's lockfile decision and APA 7th citations.
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,10 @@ Semantic Versioning where the repository publishes a release.
- Compared the trusted `uv` executable's post-install `--version` output against the real GitHub Releases build's full string, `uv 0.12.1 (x86_64-unknown-linux-gnu)`, instead of the bare `uv 0.12.1` the prior check required; the genuine release binary always prints the target triple, so every installation was failing the pin check immediately after the archive download itself was fixed (ContextualWisdomLab/.github#1109).
- Excluded relative `-r` and `--requirement` referrers from generated flat base-lock publication while retaining bounded include syntax diagnostics and discovering independently complete direct `.txt` children of `requirements` directories.
- Refused a conflict-scope repository root whose immediate parent is a symbolic link, so a swapped parent cannot redirect the canonical worktree after the last-component check (CWE-367).
- Accepted npm v2/v3 metadata-only nested workspace and peer locations only when one exact scoped or unscoped canonical root package carries the same version, HTTPS npm-registry tarball, and canonical SHA-512 integrity, while continuing to reject malformed identities, partial pins, metadata-only roots, alternate origins, and version drift. The decision record now cites RFC 3986 so an explicit port, userinfo, query, or fragment cannot masquerade as the default npm registry origin.
- Made Python lock evidence publication use the same descriptor-relative, no-follow, exclusive-write and inode-binding boundary as JavaScript lock evidence, closing TOCTOU paths that could redirect generated requirements or manifests during materialization.
- Recorded the org control-plane architecture, including nested npm metadata pins, so agents reconstruct the lockfile trust boundary from the repo instead of private memory.
- Made JavaScript lock evidence publication fail before mutation without descriptor/no-follow capabilities, create every output component from pinned directory descriptors, and roll back only inode-matched entries from a freshly owned project subtree after late write failure.
- Bounded the Strix quality self-test's deterministic timeout fixtures to 3-second process and 5-second fake-sleep budgets so exact-head policy evidence completes inside the existing job limit without changing production Strix scanner timeouts, providers, credentials, or review semantics.
- Allowed commas and ASCII parentheses in the bounded Strix changed-file path policy so legal tracked Packrat fixtures can receive exact-head security analysis, while rejecting raw `..` components before normalization and keeping controls, backslashes, whitespace ambiguity, and shell punctuation fail-closed.
- Bound each review-agent invocation key to the wrapper's complete canonical payload, including the base branch and requesting actor; altered fields with a valid-format key now fail before durable-leader election or forwarding, and wrapper write permission is job-scoped.
Expand Down
3 changes: 2 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,8 @@ Details: `docs/pr-review-and-merge-procedure.md` and `PR_GOVERNANCE_AUDIT.md`.
- `docs/` — master context, Project protocol, `org-required-workflow-rollout.md`,
`scorecard-governance.md`, SBOM inventory. Doctoring records live under
`docs/doctoring/`. [`ARCHITECTURE.md`](ARCHITECTURE.md) is the control-plane
diagram for review, hourly NVIDIA NIM repair, and merge trust boundaries.
diagram for review, hourly NVIDIA NIM repair, nested npm metadata pins, and
merge trust boundaries.
- `.jules/` — recorded performance (`bolt.md`) and security (`sentinel.md`) learnings from past work
on `scripts/ci/`; worth scanning before optimizing or hardening those scripts.

Expand Down
101 changes: 101 additions & 0 deletions docs/doctoring/npm-nested-metadata-canonical-pins.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
# npm nested metadata canonical pins

## Decision

Changed npm lockfiles remain untrusted pull-request inputs. The central JavaScript dependency materializer accepts npm lockfile versions 2 and 3 only after validating the complete `packages` map. Every non-link package location under a `node_modules` segment must declare a nonempty exact `version`.

npm can serialize a nested workspace or peer location with version and classification metadata while the canonical root location carries the registry tarball and integrity fields. The validator therefore distinguishes two safe forms:

1. **Complete pin** — the location declares both `resolved` and `integrity`. The URL must be an HTTPS tarball on `registry.npmjs.org` with no user information, explicit port, query, or fragment, and the integrity value must be one canonical SHA-512 SRI value.
2. **Metadata-only nested location** — the location declares neither field. It is accepted only when `node_modules/<package identity>` contains one complete pin for the same scoped or unscoped package identity and the exact same version.

A metadata-only canonical root entry is forbidden. A nested location that declares only one of `resolved` or `integrity` is also forbidden. Independently complete nested pins remain valid and may carry a different version because their bytes and integrity are self-contained.

## Package identity

Identity is derived from the path segments after the final `node_modules` component:

- unscoped: exactly one segment, such as `react`;
- scoped: exactly two segments, such as `@types/react-dom`.

Incomplete scopes, additional identity segments, absolute paths, backslashes, and parent traversal fail closed. Workspace links retain their separate bounded relative-link validation and never inherit registry metadata.

```mermaid
flowchart TD
A[npm packages map entry] --> B{link is true?}
B -->|yes| C[Validate bounded relative workspace target]
B -->|no| D[Derive exact package identity and require version]
D --> E{resolved and integrity}
E -->|both present| F[Validate exact npm registry tarball and SHA-512 SRI]
E -->|one present| G[Reject partial pin]
E -->|both absent| H{canonical root?}
H -->|yes| I[Reject metadata-only root]
H -->|no| J[Require same identity and version at complete root pin]
```

## Security and compatibility boundary

The policy does not repair, synthesize, or mutate lockfile metadata. It consumes the validated lock unchanged. It preserves the existing lockfile version, path, link, URL, origin, tarball suffix, and SHA-512 controls while admitting npm's location-keyed metadata representation.

RFC 3986 treats userinfo, port, query, and fragment as distinct URI
components that change origin identity (Berners-Lee et al., 2005). An
explicit `:443` is therefore not the same pin as the default HTTPS origin
`registry.npmjs.org`. The validator rejects every explicit port, not only
non-default ones.

The canonical root pin is a provenance anchor for metadata-only locations, not a claim that all nested locations share one physical installation. A complete nested record is validated independently and does not depend on the root. Missing roots, version drift, malformed identity, partial fields, alternate registries, malformed URLs, and invalid integrity remain blocking.

### Filesystem publication boundary

Materialized JavaScript and Python evidence is published only when the runtime supports descriptor-relative directory operations, descriptor-backed enumeration, `O_DIRECTORY`, `O_NOFOLLOW`, and no-follow `stat`. The capability gate runs before any output path is created. Missing output components are then created and opened one component at a time from a held filesystem-root descriptor; each name is inspected without following links, opened relative to its pinned parent, and matched to the observed device/inode identity. The final absolute pathname must still identify the pinned output directory before any project file is written.

Generated files use exclusive, no-follow descriptor-relative creation, forward-progress-checked writes, file and directory synchronization, and post-write identity and link-count validation. A project directory is fresh and owned exclusively by one attempt. If a later write fails, cleanup walks only that held project descriptor, removes only inode-matched regular files and directories in reverse publication order, and never follows links. A raced, replaced, symlink, or special entry is retained for forensic inspection; cleanup never masks the original fail-closed error or removes pre-existing operator entries outside the owned project directory.

## Verification

The permanent regression suite includes:

- the BandScope `apps/desktop/node_modules/@types/react-dom` peer-location shape;
- unscoped metadata-only locations;
- independently pinned nested versions;
- missing canonical pins;
- canonical-version mismatch;
- metadata-only canonical roots;
- partial `resolved` or `integrity` declarations;
- malformed scoped identities;
- nonempty-version enforcement;
- alternate origins and invalid SHA-512 SRI values; and
- all pre-existing npm path, link, lockfile, URL, and integrity cases;
- missing descriptor/no-follow capabilities before mutation;
- missing-ancestor and intermediate-ancestor replacement races;
- nested-directory and generated-file identity replacement; and
- late-write rollback that preserves pre-existing operator data.

The dedicated quality workflow runs Python 3.10 compilation, Python 3.14 focused tests with 100% production statement and branch coverage, 100% production docstrings, the complete central test suite, and a clean-patch check. The Python materializer regression tests additionally cover capability refusal, ancestor and output rebinding, exclusive creation, non-regular targets, partial-write cleanup, and final identity validation.

## Incident recovery and rollback

1. Preserve the exact pull-request head SHA, lockfile blob SHA, validation error, and quality-run ID.
2. Determine whether the changed lock is malformed or whether npm produced a supported metadata-only nested location.
3. Never add missing tarball or integrity values by hand. Regenerate the lock with the repository's pinned npm version when the lock is invalid.
4. Preserve any raced or unexpected filesystem entry for forensic inspection. Do not replace descriptor-relative cleanup with recursive pathname deletion.
5. Roll back only by restoring the prior fail-closed validator or another reviewed implementation that keeps the same identity, version, origin, integrity, no-follow publication, and owned-object cleanup controls.
6. Rerun the complete exact-head quality, security, and supply-chain matrix after any repair.

## References

Berners-Lee, T., Fielding, R., & Masinter, L. (2005). *Uniform Resource
Identifier (URI): Generic syntax* (RFC 3986). Internet Engineering Task
Force. https://doi.org/10.17487/RFC3986

npm, Inc. (2026). *package-lock.json*. npm Docs. https://docs.npmjs.com/cli/v11/configuring-npm/package-lock-json

npm, Inc. (2026). *npm install*. npm Docs. https://docs.npmjs.com/cli/v11/commands/npm-install

World Wide Web Consortium. (2016). *Subresource Integrity*. https://www.w3.org/TR/SRI/

Institute of Electrical and Electronics Engineers, & The Open Group. (2024). *The Open Group Base Specifications Issue 8: IEEE Std 1003.1-2024*. https://pubs.opengroup.org/onlinepubs/9799919799/

MITRE Corporation. (2026). *CWE-59: Improper link resolution before file access ('link following')* (Version 4.20). https://cwe.mitre.org/data/definitions/59.html

MITRE Corporation. (2026). *CWE-367: Time-of-check time-of-use (TOCTOU) race condition* (Version 4.20). https://cwe.mitre.org/data/definitions/367.html
Loading
Loading