Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,22 @@ jobs:
exit 1
fi
python -I "$RUNNER_TEMP/comment-policy-runner.py" --native-tests
- name: Enforce pinned notice drift
env:
BASE_REF: ${{ github.event.pull_request.base.sha }}
run: |
set -euo pipefail
: "${BASE_REF:?BASE_REF is required}"
if git diff --quiet "$BASE_REF" HEAD -- LICENSE COPYRIGHT.md quality/comment-cleanup-scope.json; then
echo "No pinned notice inputs changed; drift check not needed."
exit 0
fi
if ! git cat-file -e "${BASE_REF}:scripts/check_comment_cleanup_scope.py" 2>/dev/null; then
echo "Trusted pinned-notice checker is missing" >&2
exit 1
fi
git show "${BASE_REF}:scripts/check_comment_cleanup_scope.py" > "$RUNNER_TEMP/trusted_check_scope.py"
python -I "$RUNNER_TEMP/trusted_check_scope.py" --check-drift --base "$BASE_REF"

base-guard:
name: PR base branch is an integration branch
Expand Down
8 changes: 8 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,14 @@ repos:
stages: [pre-commit]
files: ^results-data/bundles/

- id: audit-raw
name: dependency audit raw inventory drift check
entry: uv run -- python _project/scripts/dependency_audit/parse_deps.py --check
language: system
pass_filenames: false
stages: [pre-commit]
files: ^(pyproject\.toml|_project/scripts/dependency_audit/parse_deps\.py|docs/development/dependency-audit-raw\.md)$

- id: uv-lock-revision-guard
name: uv.lock revision downgrade guard
entry: python3 _project/scripts/check_uv_lock_revision.py
Expand Down
6 changes: 5 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -447,7 +447,11 @@ windows-antipatterns-check:

.PHONY: comment-policy-check
comment-policy-check:
uv run -- python scripts/run_comment_policy.py --native-tests
uv run -- python scripts/run_comment_policy.py --native-tests && uv run -- python scripts/check_comment_cleanup_scope.py --check-drift $(if $(BASE_REF),--base "$(BASE_REF)",)

.PHONY: check-comment-scope
check-comment-scope:
uv run -- python scripts/check_comment_cleanup_scope.py --check-drift $(if $(BASE_REF),--base "$(BASE_REF)",)

.PHONY: comment-policy-strict
comment-policy-strict:
Expand Down
71 changes: 67 additions & 4 deletions _project/audits/remediation-contract-evidence.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@
develop_sha: 2eb03f3e67ec8f7f1347738f29696fa39ed5f526
measured_at_sha: dd20aed2d53a87a4c34ae74c73a187cfa9bf3c5e
checked_sha: dd20aed2d53a87a4c34ae74c73a187cfa9bf3c5e
replay_sha: 12acfd365b313374e057d97cb7191f4a98a5dc7d
replay_scope: PR 2888 workflow and pipeline/publication/full-fast replay at 50183db50c15b46a1dbd474d3bf7cd297f1c2975; PR 2821 D20-D26 scanner fixes, focused and full scanner verification at 12acfd365b313374e057d97cb7191f4a98a5dc7d
replay_sha: 94b2da915a78d525d8adf8ea1efa554914b28984
replay_scope: PR 2888 workflow and pipeline/publication/full-fast replay at 50183db50c15b46a1dbd474d3bf7cd297f1c2975; PR 2821 D20-D26 scanner fixes, focused and full scanner verification at 12acfd365b313374e057d97cb7191f4a98a5dc7d; PR 2829 drift-guard remediation, prescribed pipeline/publication replay at 94b2da915a78d525d8adf8ea1efa554914b28984
---

# Remediation contract evidence
Expand Down Expand Up @@ -111,10 +111,10 @@ uv run -- python -m pytest \
tests/unit/scripts/publication/test_journal.py -q
```

The clean detached-worktree result was `111 passed`. The command and pinned
The clean detached-worktree result was `112 passed`. The command and pinned
SHA define the replay; the aggregate count is summary evidence only.

The prescribed pipeline and publication seam replay passed 111 tests after
The prescribed pipeline and publication seam replay passed 112 tests after
the end-to-end cohort consumer control was added. Re-run the cited nodes after
any source or consumer change; a passing aggregate count is not acceptance of
the collective claim.
Expand Down Expand Up @@ -644,3 +644,66 @@ format passed for both changed Python files. Full-project `uv run ty check`
reported 674 diagnostics with exit status 0. `make comment-policy-check`
passed 5,779 source files with zero violations, zero enforced failures and all
19 native syntax tests passing.

## PR #2829 pinned notice drift

`quality/comment-cleanup-scope.json` records 19 whole-file notice pins.
`LICENSE` and `COPYRIGHT.md` are the two non-external pins in scope; the other
17 EULA, NOTICE and PATCHES files are already checked by the base-trusted
comment-policy gate. `PINNED_NOTICE_DRIFT_PATHS` limits SCOPE008 to those two
paths and avoids duplicating external-notice enforcement.

`pinned_notice_drift_findings` compares each applicable pin with the file bytes
at the immutable base. It rejects a changed or deleted notice without a
matching pin update and rejects removal of the corresponding policy entry.
Required CI checks whether `LICENSE`, `COPYRIGHT.md` or
`quality/comment-cleanup-scope.json` changed. It skips the drift step when none
changed; otherwise it runs only `scripts/check_comment_cleanup_scope.py`
extracted from `BASE_REF`. A missing trusted checker or unsupported
`--check-drift` option fails closed. The step never executes the PR's candidate
checker, so this PR can land without executing its new code in the gate.

- Both non-external paths reject changes without a pin update in
`test_pinned_notice_drift_rejects_a_modification_without_a_pin_update`.
- Both paths accept a matching pin update in
`test_pinned_notice_drift_accepts_a_modification_with_a_pin_update`.
- Deletion and policy-entry removal are rejected; external notice changes are
left to the existing base-trusted comment-policy gate.
- `test_check_drift_main_reports_zero_for_a_clean_tree_and_one_for_drift`
verifies CLI success and failure.
- `test_pinned_notice_gate_fails_closed_without_running_candidate_checker`
executes the workflow step with a missing or unsupported base checker; both
cases fail without executing the candidate checker.

An isolated smoke also confirmed the no-notice-change path skips successfully
with an older base checker without executing the candidate helper.

The focused drift selection passed 9 tests. The combined scope, CI parity,
Make timing and hook-ledger selection passed 137 tests:

```bash
uv run -- python -m pytest tests/unit/scripts/test_comment_cleanup_scope.py \
tests/system/test_ci_lint_parity.py \
tests/unit/scripts/test_make_timing.py::test_timed_targets_are_prerequisite_free_single_command_recipes \
tests/unit/test_ledger_coverage.py::test_ledger_lists_every_pre_commit_hook -q -n0
```

The direct drift check reported zero findings:

```bash
uv run -- python scripts/check_comment_cleanup_scope.py --check-drift --base "$(git merge-base origin/develop HEAD)"
```

Ruff check, Ruff format check and `py_compile` passed for the changed Python
files. `uv run -- ty check` completed with 674 diagnostics.
`make comment-policy-check` failed before its drift command: the trusted
transition scan reported 10 enforced errors with this diagnostic:
`CP coverage-error: shell command source requires an executable-payload adapter`.
They occur across seven workflow and script files that this PR does not change.
The drift check passed independently; the full policy gate remains blocked by
those coverage errors.
The final CI parity test file passed a scoped report-only policy scan with
zero violations.

The checker, tests and workflow are registered soundness paths and require
independent review.
36 changes: 17 additions & 19 deletions docs/internal/comment-policy.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,16 +127,16 @@ isolated from project configuration, import and installer overrides, candidate
`node_modules` and candidate Python environments. The candidate's own checker is used only to bootstrap a
base that contains the rollout commit `ed5c263c513ba65499f4918d3a7de607f280c65b`
and holds no trusted checker files, launcher or policy registry; on any other
base, a missing checker fails. The job log is informational and the exit status is what decides the result. The
checker escapes untrusted text it prints, and the native JavaScript tests run
inside a `stop-commands` fence with a random token. Every tool the trusted
launcher uses (Python, uv and Node) is set up before the pull request is checked
out, so no candidate file or configuration, such as a `.yarnrc` that names a
script, can run before the checker or change the interpreter it uses. The pull
request's own code still runs in the job through those native tests, and its
text can reach the log through other actions, so a log can be made to read
differently from what the checker found. It cannot change the exit status. `.github/soundness-paths.txt` protects these files, and changes to this wiring need the
repository's independent soundness review.
base, a missing checker fails. The job log is informational, and its exit
status decides the result. The checker escapes untrusted text. Native
JavaScript tests run inside a random `stop-commands` fence. The trusted
launcher's tools (Python, uv, and Node) are set up before checkout, so candidate
configuration cannot execute before the checker or change its interpreter.
The separate pinned-notice drift step runs
`scripts/check_comment_cleanup_scope.py` from the candidate revision and can
affect the job result. `.github/soundness-paths.txt` protects the workflow,
checker, manifest, and tests; changes require independent soundness review.
The drift step does not replace or modify the trusted base-sourced parser.

## Coverage and limits

Expand Down Expand Up @@ -199,12 +199,10 @@ Known gaps, each a place where a comment can pass unreported:
such comments where they are part of the product's output, for example
maintenance SQL headers, dry-run DDL previews, stream-file headers, the Trino
tuning note and the AWS Glue job script.
- A program piped to an interpreter on stdin from a producer other than `echo`
or `printf` (such as `curl ... | sh`) is not scanned.
- Whole-file pinned notices in `quality/comment-cleanup-scope.json` are checked
against their recorded digest, but changed files do not yet require a pin
update on the same branch.
- Unsupported `printf` formats or `echo -e` escapes in a recognized pipeline
produce a coverage error.
- JSX `{// ...}` line-comment expressions in MDX prose are not extracted, so
comments in that form outside Markdown code fences are not inspected.
- Pipelines to unmodeled stdin interpreters (such as `perl` or `ruby`) are not scanned.
- Whole-file pins for the non-external `LICENSE` and `COPYRIGHT.md` notices are
enforced by required CI; changing either file requires updating its pin in
the same pull request. External notices are checked against base-trusted
pins by the comment-policy gate.
- Unsupported `printf` formats or `echo -e` escapes in a recognized pipeline produce a coverage error.
- JSX `{// ...}` line-comment expressions in MDX prose are not extracted, so comments in that form outside Markdown code fences are not inspected.
1 change: 1 addition & 0 deletions docs/internal/dev-loop-property-ledger.md
Original file line number Diff line number Diff line change
Expand Up @@ -405,6 +405,7 @@ Each is `product-safety` or `tooling`, never `pure-process`:
| `uat-loc-table` | tooling | UAT table hygiene |
| `timing-policy-check` | product-safety | Monotonic-clock policy |
| `comment-policy` | tooling | Comments, docstrings, parser coverage and completed-scope enforcement |
| `audit-raw` | tooling | Dependency inventory drift against `dependency-audit-raw.md` |
| `timing-policy-fast-lane` | pure-process | Fast-lane mechanics |
| `pr-preflight` | pure-process | PR-loop mechanics |
| `blind-spot-validate` | product-safety | Blind-spot coverage |
Expand Down
117 changes: 115 additions & 2 deletions scripts/check_comment_cleanup_scope.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
import hashlib
import io
import json
import os
import re
import subprocess
import sys
Expand All @@ -17,6 +18,7 @@
from typing import Any

POLICY_PATH = "quality/comment-cleanup-scope.json"
PINNED_NOTICE_DRIFT_PATHS = frozenset({"LICENSE", "COPYRIGHT.md"})

COMMENT_POLICY_ENFORCEMENT_MODES = {"advisory", "blocking"}

Expand Down Expand Up @@ -1196,14 +1198,125 @@ def output_path(root: Path, value: str | None) -> Path | None:
return path


def resolve_drift_base(root: Path, requested: str | None) -> str:
if requested:
if len(requested) != 40 or any(char not in "0123456789abcdef" for char in requested):
raise PolicyError("base must be a full lowercase commit SHA")
return requested
base = subprocess.check_output(
["git", "-C", str(root), "rev-parse", "--verify", "origin/develop^{commit}"],
stderr=subprocess.PIPE,
text=True,
).strip()
merged = subprocess.check_output(
["git", "-C", str(root), "merge-base", base, "HEAD"], stderr=subprocess.PIPE, text=True
).strip()
if len(merged) != 40 or any(char not in "0123456789abcdef" for char in merged):
raise PolicyError("cannot resolve a drift base commit")
return merged


def pinned_notice_drift_findings(root: Path, base: str) -> list[Finding]:
try:
policy = load_policy(root / POLICY_PATH)
except (OSError, UnicodeError, json.JSONDecodeError, PolicyError) as error:
return [Finding("SCOPE008", POLICY_PATH, f"cannot load scope policy for drift check: {error}")]
try:
base_policy_bytes = base_blob(root, base, POLICY_PATH)
base_policy = load_policy_bytes(base_policy_bytes)
except (OSError, UnicodeError, json.JSONDecodeError, PolicyError, subprocess.CalledProcessError) as error:
return [Finding("SCOPE008", POLICY_PATH, f"cannot load base scope policy for drift check: {error}")]
findings = []
base_notices = {
entry.get("path"): entry
for entry in base_policy.get("notices", [])
if isinstance(entry, dict) and entry.get("path") in PINNED_NOTICE_DRIFT_PATHS
}
candidate_notices = {
entry.get("path"): entry
for entry in policy.get("notices", [])
if isinstance(entry, dict) and entry.get("path") in PINNED_NOTICE_DRIFT_PATHS
}
for path in base_notices:
if path not in candidate_notices:
findings.append(
Finding("SCOPE008", path, "pinned notice was removed from quality/comment-cleanup-scope.json")
)
for entry in policy["notices"]:
if not isinstance(entry, dict):
findings.append(Finding("SCOPE008", POLICY_PATH, "notice entry must be an object"))
continue
path = entry.get("path")
if not isinstance(path, str) or not valid_path(path):
findings.append(Finding("SCOPE008", POLICY_PATH, f"notice has an invalid path: {path!r}"))
continue
if path not in PINNED_NOTICE_DRIFT_PATHS:
continue
try:
frozen = base_blob(root, base, path)
except subprocess.CalledProcessError:
findings.append(Finding("SCOPE008", path, "pinned notice is not tracked at the base"))
continue
try:
current = (root / path).read_bytes()
except OSError:
findings.append(
Finding("SCOPE008", path, "pinned notice is missing from the working tree without a pin update")
)
continue
start = entry.get("byte_start")
end = entry.get("byte_end")
retained = entry.get("retained_sha256")
if (
entry.get("blob_sha256") != hashlib.sha256(current).hexdigest()
or not isinstance(start, int)
or isinstance(start, bool)
or not isinstance(end, int)
or isinstance(end, bool)
or not isinstance(retained, str)
or start < 0
or end <= start
or end > len(current)
or hashlib.sha256(current[start:end]).hexdigest() != retained
):
if current == frozen:
detail = "digest pin does not match the pinned file content"
else:
detail = "pinned notice modified without updating the digest pin in quality/comment-cleanup-scope.json"
findings.append(Finding("SCOPE008", path, detail))
return findings


def run_drift_check(root: Path, base: str | None, policy_path: str) -> int:
try:
if policy_path != POLICY_PATH:
raise PolicyError("policy location is fixed")
resolved = resolve_drift_base(root, base)
findings = pinned_notice_drift_findings(root, resolved)
except (OSError, UnicodeError, PolicyError, subprocess.CalledProcessError) as error:
print(f"comment-cleanup-scope: configuration failure: {error}", file=sys.stderr)
return 2
for finding in findings:
print(finding, file=sys.stderr)
print(f"comment-cleanup-scope drift: {len(findings)} pinned notice findings")
return int(bool(findings))


def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--root", type=Path, default=Path.cwd())
parser.add_argument("--base", required=True)
parser.add_argument("--task-set", required=True)
parser.add_argument("--base", required=False, default=os.environ.get("BASE_REF"))
parser.add_argument("--task-set", required=False, default=None)
parser.add_argument("--policy", default=POLICY_PATH)
parser.add_argument("--output")
parser.add_argument("--check-drift", action="store_true")
args = parser.parse_args(argv)
if args.check_drift:
return run_drift_check(args.root.resolve(), args.base, args.policy)
if not args.base:
parser.error("--base is required")
if not args.task_set:
parser.error("--task-set is required")
try:
root = args.root.resolve()
policy_path = root / args.policy
Expand Down
Loading
Loading