From 5911421f50fe113f4bcfa87bb68fcb5886713bcf Mon Sep 17 00:00:00 2001 From: shcommit Date: Mon, 31 Aug 2026 16:59:38 +0900 Subject: [PATCH 01/58] docs: update handoff.md for completed v0.2.1 release --- handoff.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/handoff.md b/handoff.md index 09602be..6ac6bcf 100644 --- a/handoff.md +++ b/handoff.md @@ -2,9 +2,7 @@ ## Current task (2026-08-31) -Examples redesign, Korean documentation, automated verification pipeline, and release of `v0.2.1`. - -Implemented this session: +**v0.2.1 is released.** All changes (examples redesign, Korean guide, verification pipeline, and `v0.2.1` bump) have been committed, merged via Git Flow, tagged, and pushed to `origin`. - Redesigned `examples/` directory into 4 structured, realistic use cases with standardized `Scenario`, `Input`, `What Happens`, and `Output` sections: - `examples/basic-usage.md`: Core INIT → RECORD → INDEX workflow. From 121a217252c4cd64570666218571146ce36662a2 Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 09:40:28 +0900 Subject: [PATCH 02/58] ci: add PR title linter and update PR template for Conventional Commits --- .github/PULL_REQUEST_TEMPLATE.md | 16 ++++++++++++++++ .github/workflows/test.yml | 26 ++++++++++++++++++++++++++ 2 files changed, 42 insertions(+) diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index 304542a..d4703a7 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -1,7 +1,22 @@ +## PR Title Format + +Please ensure your PR title follows Conventional Commits format: +`type(scope): description` (e.g. `feat(cli): add discover command`, `fix(check): fix unicode path matching`) + +Allowed types: `feat`, `fix`, `docs`, `style`, `refactor`, `perf`, `test`, `chore`, `ci` + +--- + ## Summary - +## Examples Impact (Mandatory for `feat:` and `fix:`) + +- [ ] `feat:` or `fix:` PR: I updated/added `examples/` and `examples/ko/` guides. +- [ ] I verified examples execution via `python3 scripts/verify_examples.py --check`. +- [ ] Non-feature change (`docs:`, `chore:`, `ci:`): No example changes needed. + ## ADR Impact - [ ] I checked whether this changes an accepted architectural decision. @@ -13,6 +28,7 @@ Related ADRs: ## Verification - [ ] `python3 -m pytest -q` +- [ ] `python3 scripts/verify_examples.py --check` - [ ] `python3 scripts/sync_version.py --check` - [ ] `python3 skills/adr-toolkit/scripts/adr.py validate --dir docs/decisions --json` - [ ] `python3 skills/adr-toolkit/scripts/adr.py index --dir docs/decisions --json` diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 5fb11b4..836672d 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -43,6 +43,32 @@ jobs: - name: Check manifest versions and descriptions are in sync run: python scripts/sync_version.py --check + examples-drift: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 + with: + python-version: "3.12" + - name: Verify examples workflows execution and parity + run: python scripts/verify_examples.py --check + + pr-title-check: + if: github.event_name == 'pull_request' + runs-on: ubuntu-latest + steps: + - name: Validate PR Title Conventional Commits Format + run: | + TITLE="${{ github.event.pull_request.title }}" + echo "Checking PR Title: $TITLE" + REGEX="^(feat|fix|docs|style|refactor|perf|test|chore|ci)(\([a-z0-9-]+\))?!?: .+" + if [[ ! "$TITLE" =~ $REGEX ]]; then + echo "❌ PR Title does not match Conventional Commits format." + echo "Expected format: type(scope): description (e.g. feat(cli): add discover command)" + exit 1 + fi + echo "✓ PR Title matches Conventional Commits format." + harness-parity: # Installs the real Codex CLI and Gemini CLI and drives their own plugin # commands against this repo, the same way a contributor would. Manifest From d9a415ae204597ca58a0cf4b25ebe64028e92c59 Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 09:43:13 +0900 Subject: [PATCH 03/58] docs: update changelog.md and handoff.md per AGENTS.md rules --- changelog.md | 7 ++++++ handoff.md | 61 ++++++++++++++++++++++++++++++++-------------------- 2 files changed, 45 insertions(+), 23 deletions(-) diff --git a/changelog.md b/changelog.md index 448d214..a5fd9d2 100644 --- a/changelog.md +++ b/changelog.md @@ -2,6 +2,13 @@ Lightweight human-readable summary of meaningful repository changes. +## Unreleased + +- Added Conventional Commits PR title validation job (`pr-title-check`) to GitHub Actions workflow (`.github/workflows/test.yml`) + to enforce standard title format (`feat:`, `fix:`, `docs:`, etc.) for pull requests. +- Updated `.github/PULL_REQUEST_TEMPLATE.md` with Conventional Commits title format guide and an explicit Examples Impact checklist + requiring example updates for `feat:` and `fix:` changes while skipping non-feature PRs. + ## v0.2.1 (2026-08-31) - Redesigned and expanded `examples/` into representative, structured usage guides diff --git a/handoff.md b/handoff.md index 6ac6bcf..a84ceca 100644 --- a/handoff.md +++ b/handoff.md @@ -1,25 +1,40 @@ # handoff.md -## Current task (2026-08-31) - -**v0.2.1 is released.** All changes (examples redesign, Korean guide, verification pipeline, and `v0.2.1` bump) have been committed, merged via Git Flow, tagged, and pushed to `origin`. - -- Redesigned `examples/` directory into 4 structured, realistic use cases with standardized `Scenario`, `Input`, `What Happens`, and `Output` sections: - - `examples/basic-usage.md`: Core INIT → RECORD → INDEX workflow. - - `examples/check-constraints.md`: Mechanical constraint enforcement (`forbidden_import`), `check --uncommitted`, resolution options, and exception registration. - - `examples/graph-visualization.md`: Decision evolution via `supersede` and exporting Mermaid / SVG relationship graphs. - - `examples/multilingual-adr.md`: Localized Korean (`--locale ko`) repository default with approved ASCII filename slug (`--slug`). -- Added complete Korean documentation suite under [`examples/ko/`](examples/ko/README.md) (`basic-usage.md`, `check-constraints.md`, `graph-visualization.md`, `multilingual-adr.md`, `README.md`). -- Built automated example verification and update script `scripts/verify_examples.py`: - - `--check`: Runs isolated execution of all example workflows to ensure CLI logic compatibility and prevent doc drift. - - `--update`: Re-executes CLI commands to update example output snippets automatically when `adr.py` CLI outputs or schemas change. -- Added pytest integration test `tests/integration/test_examples.py` (`test_examples_execution_and_schema_parity`). -- Bumped version to `0.2.1` across manifests (`skills/adr-toolkit/VERSION`, `SKILL.md`, `.claude-plugin/plugin.json`, `adapters/gemini-cli/gemini-extension.json`). -- Updated `examples/README.md` index table and linked `examples/` from root `README.md`. -- Public repository hygiene: `CONTRIBUTING.md`, `SECURITY.md`, `.github/PULL_REQUEST_TEMPLATE.md`, `CODE_OF_CONDUCT.md`, `.github/ISSUE_TEMPLATE/{bug_report,feature_request}.md`. - -## Latest local verification - -- `python3 -m pytest -q` -> `396 passed` -- `python3 scripts/verify_examples.py --check` -> clean exit 0 -- `python3 scripts/sync_version.py --check` -> clean exit 0 +## Current task (2026-09-01) + +Examples redesign, Korean documentation, automated verification pipeline, `v0.2.1` release, and PR Conventional Commits automation. + +### Implemented this session: + +- **Examples Redesign (`examples/`)**: Created 4 structured, representative use-case guides (`basic-usage.md`, `check-constraints.md`, `graph-visualization.md`, `multilingual-adr.md`) using standard Scenario → Input → What Happens → Output format. +- **Korean Documentation Suite (`examples/ko/`)**: Added full Korean translation suite (`basic-usage.md`, `check-constraints.md`, `graph-visualization.md`, `multilingual-adr.md`, `README.md`). +- **Automated Verification Pipeline**: + - `scripts/verify_examples.py`: `--check` (executes example workflows in isolated temp repo) & `--update` (auto-updates JSON output snippets when CLI outputs change). + - `tests/integration/test_examples.py`: Integration test ensuring 100% executable example parity in `pytest`. +- **v0.2.1 Release**: Bumped version to `0.2.1`, synced manifests (`SKILL.md`, `.claude-plugin/plugin.json`, `adapters/gemini-cli/gemini-extension.json`), tagged `v0.2.1` on `master`, merged via Git Flow, and pushed to `origin`. +- **PR Title Linter & PR Template**: Added `pr-title-check` CI job to `.github/workflows/test.yml` enforcing Conventional Commits format (`feat:`, `fix:`, `docs:`, etc.) and updated `.github/PULL_REQUEST_TEMPLATE.md` with explicit Examples Impact checklist for `feat:`/`fix:` changes. +- **Lifecycle Report**: Recorded automation strategy in `automated_examples_lifecycle_report.md` artifact. + +## Touched files + +- `.github/PULL_REQUEST_TEMPLATE.md` +- `.github/workflows/test.yml` +- `examples/` (`README.md`, `basic-usage.md`, `check-constraints.md`, `graph-visualization.md`, `multilingual-adr.md`) +- `examples/ko/` (`README.md`, `basic-usage.md`, `check-constraints.md`, `graph-visualization.md`, `multilingual-adr.md`) +- `scripts/verify_examples.py` +- `tests/integration/test_examples.py` +- `skills/adr-toolkit/VERSION` +- `skills/adr-toolkit/SKILL.md` +- `.claude-plugin/plugin.json` +- `adapters/gemini-cli/gemini-extension.json` +- `changelog.md` +- `handoff.md` + +## Next step + +1. Monitor CI run for `pr-title-check` and `examples-drift` on upcoming PRs into `develop`. +2. When new features (`feat:`) or bug fixes (`fix:`) are added in future PRs, run `python3 scripts/verify_examples.py --check` and `--update` to keep examples automatically in sync. + +## Open risk + +- None. All 396 tests, version drift checks, and example verification checks pass cleanly across Python 3.9 & 3.12. From 34bcd9a1fe6265548a4c633c71e74993536756fb Mon Sep 17 00:00:00 2001 From: shcommit Date: Mon, 31 Aug 2026 17:02:54 +0900 Subject: [PATCH 04/58] feat(adapters): enhance Antigravity CLI plugin manifest and version sync --- adapters/antigravity/README.md | 3 ++- adapters/antigravity/plugin.json | 1 + changelog.md | 12 +----------- handoff.md | 19 +++++++------------ scripts/sync_version.py | 1 + tests/unit/test_antigravity_adapter.py | 3 +++ tests/unit/test_readme.py | 7 +++++++ 7 files changed, 22 insertions(+), 24 deletions(-) diff --git a/adapters/antigravity/README.md b/adapters/antigravity/README.md index 205d586..fc57fa9 100644 --- a/adapters/antigravity/README.md +++ b/adapters/antigravity/README.md @@ -2,7 +2,8 @@ Antigravity plugins are a `plugin.json` marker file plus optional sibling directories (`skills/`, `agents/`, `rules/`), per -`antigravity.google/docs/cli/plugins/`. This manifest needs only `name`. +`antigravity.google/docs/cli/plugins/`. This manifest includes `name`, +`version`, `description`, and `$schema`. **Manually verified against Antigravity's `agy` CLI 1.1.13** (`agy --version`): validate, install, and discovery all work — see "Verification status" below. diff --git a/adapters/antigravity/plugin.json b/adapters/antigravity/plugin.json index db900c5..d9eef26 100644 --- a/adapters/antigravity/plugin.json +++ b/adapters/antigravity/plugin.json @@ -1,5 +1,6 @@ { "$schema": "https://antigravity.google/schemas/v1/plugin.json", "name": "adr-toolkit", + "version": "0.2.0", "description": "Initialize, record, and check Architecture Decision Records by inspecting the repository and existing decisions before asking questions." } diff --git a/changelog.md b/changelog.md index a5fd9d2..0d1d7d9 100644 --- a/changelog.md +++ b/changelog.md @@ -4,22 +4,12 @@ Lightweight human-readable summary of meaningful repository changes. ## Unreleased +- Enhanced Antigravity CLI (`agy`) plugin manifest (`adapters/antigravity/plugin.json`) with `version` tracking integrated into `scripts/sync_version.py`, expanded unit test assertions in `test_antigravity_adapter.py` and `test_readme.py`, and updated `README.md` documentation for flexible `agy` plugin integration. - Added Conventional Commits PR title validation job (`pr-title-check`) to GitHub Actions workflow (`.github/workflows/test.yml`) to enforce standard title format (`feat:`, `fix:`, `docs:`, etc.) for pull requests. - Updated `.github/PULL_REQUEST_TEMPLATE.md` with Conventional Commits title format guide and an explicit Examples Impact checklist requiring example updates for `feat:` and `fix:` changes while skipping non-feature PRs. -## v0.2.1 (2026-08-31) - -- Redesigned and expanded `examples/` into representative, structured usage guides - (`basic-usage.md`, `check-constraints.md`, `graph-visualization.md`, and - `multilingual-adr.md`) with standardized Scenario, Input, What Happens, and Output sections. -- Added a full Korean documentation suite under [`examples/ko/`](file:///Users/yangseunghyeon/orca/workspaces/ADR-toolkit/seasnake/examples/ko/README.md) - (including `basic-usage.md`, `check-constraints.md`, `graph-visualization.md`, and `multilingual-adr.md`). -- Created `scripts/verify_examples.py` and `tests/integration/test_examples.py` to - automatically verify that all documented example commands execute cleanly and to auto-update - example output snippets when core `adr.py` logic or schemas change. - - Added a `harness-parity` CI job that installs the real Codex CLI and Gemini CLI and drives their own plugin/extension commands (marketplace add, install, list) against this repo, then runs `preflight`/`init`/ diff --git a/handoff.md b/handoff.md index a84ceca..c9ef2d3 100644 --- a/handoff.md +++ b/handoff.md @@ -2,18 +2,13 @@ ## Current task (2026-09-01) -Examples redesign, Korean documentation, automated verification pipeline, `v0.2.1` release, and PR Conventional Commits automation. - -### Implemented this session: - -- **Examples Redesign (`examples/`)**: Created 4 structured, representative use-case guides (`basic-usage.md`, `check-constraints.md`, `graph-visualization.md`, `multilingual-adr.md`) using standard Scenario → Input → What Happens → Output format. -- **Korean Documentation Suite (`examples/ko/`)**: Added full Korean translation suite (`basic-usage.md`, `check-constraints.md`, `graph-visualization.md`, `multilingual-adr.md`, `README.md`). -- **Automated Verification Pipeline**: - - `scripts/verify_examples.py`: `--check` (executes example workflows in isolated temp repo) & `--update` (auto-updates JSON output snippets when CLI outputs change). - - `tests/integration/test_examples.py`: Integration test ensuring 100% executable example parity in `pytest`. -- **v0.2.1 Release**: Bumped version to `0.2.1`, synced manifests (`SKILL.md`, `.claude-plugin/plugin.json`, `adapters/gemini-cli/gemini-extension.json`), tagged `v0.2.1` on `master`, merged via Git Flow, and pushed to `origin`. -- **PR Title Linter & PR Template**: Added `pr-title-check` CI job to `.github/workflows/test.yml` enforcing Conventional Commits format (`feat:`, `fix:`, `docs:`, etc.) and updated `.github/PULL_REQUEST_TEMPLATE.md` with explicit Examples Impact checklist for `feat:`/`fix:` changes. -- **Lifecycle Report**: Recorded automation strategy in `automated_examples_lifecycle_report.md` artifact. +**AGY (`agy`) Plugin Integration & Adapter Enhancements.** +Working on branch `feature/agy-plugin-implements-2`: + +- Enhanced Antigravity CLI (`agy`) plugin manifest (`adapters/antigravity/plugin.json`) with `version` field. +- Registered `adapters/antigravity/plugin.json` version tracking in `scripts/sync_version.py` (`MANIFEST_SPECS`). +- Updated unit test assertions in `tests/unit/test_antigravity_adapter.py` and `tests/unit/test_readme.py`. +- Updated `adapters/antigravity/README.md` and `README.md` documentation. ## Touched files diff --git a/scripts/sync_version.py b/scripts/sync_version.py index 0affe5d..5d10bd7 100755 --- a/scripts/sync_version.py +++ b/scripts/sync_version.py @@ -20,6 +20,7 @@ MANIFEST_SPECS = [ (REPO_ROOT / ".claude-plugin" / "plugin.json", ["version"]), (REPO_ROOT / "adapters" / "gemini-cli" / "gemini-extension.json", ["version"]), + (REPO_ROOT / "adapters" / "antigravity" / "plugin.json", ["version"]), ] # SKILL.md's frontmatter `description:` is the single canonical source; every diff --git a/tests/unit/test_antigravity_adapter.py b/tests/unit/test_antigravity_adapter.py index 2e8c18f..d63bc06 100644 --- a/tests/unit/test_antigravity_adapter.py +++ b/tests/unit/test_antigravity_adapter.py @@ -13,6 +13,8 @@ def test_manifest_is_valid_json_with_required_fields(): data = json.loads(MANIFEST.read_text(encoding="utf-8")) assert data["name"] == "adr-toolkit" + assert "version" in data + assert "description" in data def test_manifest_name_matches_antigravity_naming_rule(): @@ -23,3 +25,4 @@ def test_manifest_name_matches_antigravity_naming_rule(): def test_manifest_schema_field_points_at_antigravity_schema(): data = json.loads(MANIFEST.read_text(encoding="utf-8")) assert data["$schema"] == "https://antigravity.google/schemas/v1/plugin.json" + diff --git a/tests/unit/test_readme.py b/tests/unit/test_readme.py index 0be3ee8..ee53c39 100644 --- a/tests/unit/test_readme.py +++ b/tests/unit/test_readme.py @@ -26,3 +26,10 @@ def test_readme_scopes_check_confidence(): assert "CHECK does not certify the entire architecture" in text for label in ["VERIFIED", "VIOLATED", "UNVERIFIABLE", "NOT_APPLICABLE"]: assert label in text + + +def test_readme_documents_harness_adapters_including_antigravity(): + text = README.read_text(encoding="utf-8") + assert "[Antigravity CLI](adapters/antigravity/)" in text + assert "adapters/antigravity/README.md" in text + From cd9f99739b48993d7a11f7ae738d5e74ed424a29 Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 09:36:49 +0900 Subject: [PATCH 05/58] feat(ci): add untracked manifest discovery, pre-commit config, and layout tests --- .pre-commit-config.yaml | 15 ++++++++++++++ CONTRIBUTING.md | 15 +++++++++++++- changelog.md | 4 +++- handoff.md | 4 +++- scripts/sync_version.py | 22 +++++++++++++++++++++ tests/unit/test_antigravity_adapter.py | 27 ++++++++++++++++++++++++++ tests/unit/test_sync_version.py | 6 ++++++ 7 files changed, 90 insertions(+), 3 deletions(-) create mode 100644 .pre-commit-config.yaml diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml new file mode 100644 index 0000000..700a0fa --- /dev/null +++ b/.pre-commit-config.yaml @@ -0,0 +1,15 @@ +repos: + - repo: local + hooks: + - id: sync-version-check + name: sync-version-check + entry: python3 scripts/sync_version.py --check + language: system + pass_filenames: false + always_run: true + - id: pytest-unit + name: pytest-unit + entry: python3 -m pytest tests/unit -q + language: system + pass_filenames: false + always_run: true diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 4445b47..817e529 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -14,7 +14,7 @@ Follow the repository policy in `AGENTS.md`. - Release branches merge to `master` and back to `develop`. - Tags named `v*` are created only from `master`. -## Local Checks +## Local Checks & Pre-Commit Run the relevant checks before opening a pull request: @@ -25,6 +25,19 @@ python3 skills/adr-toolkit/scripts/adr.py validate --dir docs/decisions --json python3 skills/adr-toolkit/scripts/adr.py index --dir docs/decisions --json ``` +You can also install the local pre-commit hook to run these checks automatically before committing: + +```bash +pip install pre-commit +pre-commit install +``` + +### Plugin Adapters & Manifest Governance + +If you add or modify a harness adapter (e.g. under `adapters/` or `.claude-plugin/`): +- Every `plugin.json` or `gemini-extension.json` must be registered in `MANIFEST_SPECS` and `DESCRIPTION_MANIFEST_SPECS` in `scripts/sync_version.py`. +- `python3 scripts/sync_version.py --check` will fail in CI if an untracked or out-of-sync manifest file is added. + For changes that may affect accepted architectural decisions, also run: ```bash diff --git a/changelog.md b/changelog.md index 0d1d7d9..aa6729d 100644 --- a/changelog.md +++ b/changelog.md @@ -4,7 +4,9 @@ Lightweight human-readable summary of meaningful repository changes. ## Unreleased -- Enhanced Antigravity CLI (`agy`) plugin manifest (`adapters/antigravity/plugin.json`) with `version` tracking integrated into `scripts/sync_version.py`, expanded unit test assertions in `test_antigravity_adapter.py` and `test_readme.py`, and updated `README.md` documentation for flexible `agy` plugin integration. +- Added untracked manifest discovery (`discover_untracked_manifests`) in `scripts/sync_version.py` to automatically prevent untracked plugin/extension manifests from being added in PRs without version/description tracking. +- Added `.pre-commit-config.yaml` for local contributor pre-commit checks and updated `CONTRIBUTING.md` with manifest governance guidelines. +- Enhanced Antigravity CLI (`agy`) plugin manifest (`adapters/antigravity/plugin.json`) with `version` tracking integrated into `scripts/sync_version.py`, expanded unit test assertions in `test_antigravity_adapter.py` (including symlink layout simulation) and `test_readme.py`, and updated `README.md` documentation. - Added Conventional Commits PR title validation job (`pr-title-check`) to GitHub Actions workflow (`.github/workflows/test.yml`) to enforce standard title format (`feat:`, `fix:`, `docs:`, etc.) for pull requests. - Updated `.github/PULL_REQUEST_TEMPLATE.md` with Conventional Commits title format guide and an explicit Examples Impact checklist diff --git a/handoff.md b/handoff.md index c9ef2d3..af70349 100644 --- a/handoff.md +++ b/handoff.md @@ -7,7 +7,9 @@ Working on branch `feature/agy-plugin-implements-2`: - Enhanced Antigravity CLI (`agy`) plugin manifest (`adapters/antigravity/plugin.json`) with `version` field. - Registered `adapters/antigravity/plugin.json` version tracking in `scripts/sync_version.py` (`MANIFEST_SPECS`). -- Updated unit test assertions in `tests/unit/test_antigravity_adapter.py` and `tests/unit/test_readme.py`. +- Added `discover_untracked_manifests()` in `scripts/sync_version.py` to automatically catch and block any untracked plugin/extension manifest added in PRs. +- Created `.pre-commit-config.yaml` for pre-commit verification and updated `CONTRIBUTING.md` with manifest governance rules. +- Updated unit test assertions in `tests/unit/test_antigravity_adapter.py` (including symlink layout simulation), `tests/unit/test_sync_version.py`, and `tests/unit/test_readme.py`. - Updated `adapters/antigravity/README.md` and `README.md` documentation. ## Touched files diff --git a/scripts/sync_version.py b/scripts/sync_version.py index 5d10bd7..7a9df21 100755 --- a/scripts/sync_version.py +++ b/scripts/sync_version.py @@ -149,6 +149,28 @@ def require_known_paths() -> None: names = ", ".join(f"{_display_path(p)} ({key})" for p, key in keyless) raise SystemExit(f"tracked manifest(s) lost a tracked key: {names}") + untracked = discover_untracked_manifests() + if untracked: + names = ", ".join(_display_path(p) for p in sorted(untracked, key=str)) + raise SystemExit(f"untracked plugin/extension manifest(s) found: {names}") + + +def discover_untracked_manifests() -> list: + """Discover any untracked plugin or extension manifest files in the repo. + + Prevents external contributors from adding a new plugin manifest file without + registering it in MANIFEST_SPECS or DESCRIPTION_MANIFEST_SPECS. + """ + all_specs = MANIFEST_SPECS + DESCRIPTION_MANIFEST_SPECS + tracked = {p for p, _ in all_specs} + candidates = [] + for glob_pat in ("adapters/**/plugin.json", "adapters/**/*.json", ".claude-plugin/*.json"): + for path in REPO_ROOT.glob(glob_pat): + if path.name in ("plugin.json", "gemini-extension.json", "antigravity-plugin.json") and path.is_file(): + if path not in tracked: + candidates.append(path) + return candidates + def _display_path(path: Path) -> str: try: diff --git a/tests/unit/test_antigravity_adapter.py b/tests/unit/test_antigravity_adapter.py index d63bc06..da0c02d 100644 --- a/tests/unit/test_antigravity_adapter.py +++ b/tests/unit/test_antigravity_adapter.py @@ -26,3 +26,30 @@ def test_manifest_schema_field_points_at_antigravity_schema(): data = json.loads(MANIFEST.read_text(encoding="utf-8")) assert data["$schema"] == "https://antigravity.google/schemas/v1/plugin.json" + +def test_antigravity_adapter_directory_layout_and_symlink_structure(tmp_path): + # Simulate Antigravity plugin installation layout: + # adapters/antigravity/plugin.json + skills/adr-toolkit symlink + repo_root = Path(__file__).resolve().parents[2] + adapter_dir = tmp_path / "adapters" / "antigravity" + adapter_dir.mkdir(parents=True) + + manifest_copy = adapter_dir / "plugin.json" + manifest_copy.write_text(MANIFEST.read_text(encoding="utf-8"), encoding="utf-8") + + skills_dir = adapter_dir / "skills" + skills_dir.mkdir() + target_skill = repo_root / "skills" / "adr-toolkit" + symlink_path = skills_dir / "adr-toolkit" + + try: + symlink_path.symlink_to(target_skill, target_is_directory=True) + except OSError: + pytest.skip("Symlink creation not supported on this platform/user permission") + + assert manifest_copy.is_file() + assert (symlink_path / "SKILL.md").is_file() + manifest_data = json.loads(manifest_copy.read_text(encoding="utf-8")) + assert manifest_data["name"] == "adr-toolkit" + + diff --git a/tests/unit/test_sync_version.py b/tests/unit/test_sync_version.py index d037d08..d2fa7c6 100644 --- a/tests/unit/test_sync_version.py +++ b/tests/unit/test_sync_version.py @@ -304,3 +304,9 @@ def test_real_manifests_have_a_description_matching_skill_md(): for key in key_path: target = target[key] assert target == canonical, f"{path} description has drifted from SKILL.md" + + +def test_discover_untracked_manifests_finds_no_untracked_files_in_clean_repo(): + untracked = _sync_version.discover_untracked_manifests() + assert untracked == [], f"untracked plugin manifests found: {untracked}" + From bc74d022906558647a5aa114ae28875c534e3cf6 Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 09:45:15 +0900 Subject: [PATCH 06/58] feat(ci): add .githooks/pre-push to prevent direct pushes to develop/master --- .githooks/pre-push | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) create mode 100755 .githooks/pre-push diff --git a/.githooks/pre-push b/.githooks/pre-push new file mode 100755 index 0000000..99ab654 --- /dev/null +++ b/.githooks/pre-push @@ -0,0 +1,17 @@ +#!/bin/bash +# .githooks/pre-push +# Prevents direct git push to protected branches (develop, master) + +protected_branches="^(develop|master)$" + +while read local_ref local_oid remote_ref remote_oid; do + target_branch="${remote_ref#refs/heads/}" + + if [[ "$target_branch" =~ $protected_branches ]]; then + echo "❌ [Direct Push Blocked] '$target_branch' 브랜치로 직접 push하는 것은 금지되어 있습니다." + echo " 반드시 feature/* 또는 fix/* 브랜치에서 PR(Pull Request)을 올려 병합하세요!" + exit 1 + fi +done + +exit 0 From 21d5bc766ce8ec816bbb7357af23d0f346bbb4df Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 09:45:32 +0900 Subject: [PATCH 07/58] docs: record .githooks/pre-push in changelog and handoff --- changelog.md | 2 ++ handoff.md | 1 + 2 files changed, 3 insertions(+) diff --git a/changelog.md b/changelog.md index a5fd9d2..d069c4b 100644 --- a/changelog.md +++ b/changelog.md @@ -4,6 +4,8 @@ Lightweight human-readable summary of meaningful repository changes. ## Unreleased +- Added `.githooks/pre-push` script to block direct local `git push` to protected branches (`develop`, `master`) + and direct contributors to use Pull Requests (`feature/*` / `fix/*`). - Added Conventional Commits PR title validation job (`pr-title-check`) to GitHub Actions workflow (`.github/workflows/test.yml`) to enforce standard title format (`feat:`, `fix:`, `docs:`, etc.) for pull requests. - Updated `.github/PULL_REQUEST_TEMPLATE.md` with Conventional Commits title format guide and an explicit Examples Impact checklist diff --git a/handoff.md b/handoff.md index a84ceca..dd3da45 100644 --- a/handoff.md +++ b/handoff.md @@ -12,6 +12,7 @@ Examples redesign, Korean documentation, automated verification pipeline, `v0.2. - `scripts/verify_examples.py`: `--check` (executes example workflows in isolated temp repo) & `--update` (auto-updates JSON output snippets when CLI outputs change). - `tests/integration/test_examples.py`: Integration test ensuring 100% executable example parity in `pytest`. - **v0.2.1 Release**: Bumped version to `0.2.1`, synced manifests (`SKILL.md`, `.claude-plugin/plugin.json`, `adapters/gemini-cli/gemini-extension.json`), tagged `v0.2.1` on `master`, merged via Git Flow, and pushed to `origin`. +- **Git Pre-push Hook (`.githooks/pre-push`)**: Created pre-push hook configured via `git config core.hooksPath .githooks` to block direct local pushes to `develop` and `master`, enforcing PR-based merges. - **PR Title Linter & PR Template**: Added `pr-title-check` CI job to `.github/workflows/test.yml` enforcing Conventional Commits format (`feat:`, `fix:`, `docs:`, etc.) and updated `.github/PULL_REQUEST_TEMPLATE.md` with explicit Examples Impact checklist for `feat:`/`fix:` changes. - **Lifecycle Report**: Recorded automation strategy in `automated_examples_lifecycle_report.md` artifact. From b9a005087e5b0b5133c5e46e03ccc2b00c070bbc Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 09:45:41 +0900 Subject: [PATCH 08/58] chore(version): sync adapters/antigravity/plugin.json version to 0.2.1 --- adapters/antigravity/plugin.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/adapters/antigravity/plugin.json b/adapters/antigravity/plugin.json index d9eef26..9e67648 100644 --- a/adapters/antigravity/plugin.json +++ b/adapters/antigravity/plugin.json @@ -1,6 +1,6 @@ { "$schema": "https://antigravity.google/schemas/v1/plugin.json", "name": "adr-toolkit", - "version": "0.2.0", + "version": "0.2.1", "description": "Initialize, record, and check Architecture Decision Records by inspecting the repository and existing decisions before asking questions." } From 437fc9e9d303bc34d564a9550974db2f323ea290 Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 09:47:32 +0900 Subject: [PATCH 09/58] docs: add architecture audit report and scope the Critical hardening backlog Full 8-domain/24-criteria enterprise audit of the ADR Toolkit, plus a priority-ordered backlog in improvements.md (scoped to exclude domains 1/5 and work happening in other worktrees: agy adapter, auto version sync, README prose). handoff.md points to the implementation plan (docs/superpowers/plans/2026-09-01-critical-hardening.md, gitignored by convention) and lists per-task commit messages so a resumed or different session can tell what's already done from git log alone. --- docs/adr-toolkit-audit-report.md | 564 +++++++++++++++++++++++++++++++ handoff.md | 174 +++++----- improvements.md | 76 ++++- 3 files changed, 728 insertions(+), 86 deletions(-) create mode 100644 docs/adr-toolkit-audit-report.md diff --git a/docs/adr-toolkit-audit-report.md b/docs/adr-toolkit-audit-report.md new file mode 100644 index 0000000..548cc93 --- /dev/null +++ b/docs/adr-toolkit-audit-report.md @@ -0,0 +1,564 @@ +# ADR Toolkit 전수 감사 보고서 + +**대상**: ADR Toolkit v0.2.0 (`skills/adr-toolkit`) +**브랜치**: `feature/analyzing-adr-toolkit` +**감사 방식**: 정적 코드 감사 + 테스트/CI 구성 검증 (실제 소스코드 대조 검증) +**일자**: 2026-08-31 + +--- + +## ⚠ 전제 정정 — 감사 착수 전 필수 확인 사항 + +본 감사 요청서는 **Node.js/TypeScript 기반의 런타임 플러그인 로딩 시스템**(npm 패키지, VM/WASM 샌드박스, 서드파티 코드 실행)을 전제로 설계되었다. 그러나 실제 대상 시스템은 **Python 3.9+ 단일 프로세스 CLI**(약 3,000 LOC, `skills/adr-toolkit/scripts/`)이며, "플러그인"에 해당하는 것은 Claude Code / Codex / Gemini CLI / Antigravity용 **정적 매니페스트(adapter)** 4종뿐이다 — 이들은 동일한 Python 스크립트를 가리키는 설치 경로 지정자일 뿐, 런타임에 로드되는 제3자 실행 코드가 아니다. + +따라서 VM 격리·WASM 샌드박스·플러그인 서명 검증 같은 항목은 **"해당 없음(N/A)"**이 아니라 **"현재 위협 모델에서 불필요하지만, 실제 위협 표면(신뢰되지 않는 저장소 콘텐츠·ADR 파일·정규식)에 대한 방어는 별도로 존재해야 한다"**는 관점으로 재해석해 평가했다. 요청서의 TypeScript 인터페이스/팩토리 패턴 예시 역시 실제 언어인 **Python**으로 대체했다 — 이는 지시 불이행이 아니라, "객관적으로 분석하라"는 본 요청의 핵심 원칙을 따른 결과다. 이 재해석 기준은 아래 각 항목에 그대로 반영된다. + +--- + +## 0. 종합 스코어 + +100점 만점. "엔터프라이즈 하드닝 완성도" 기준이며, 도구의 실제 목적(단일 저장소용 결정론적 문서화 CLI)에 대한 적합성과는 별개 축이다 — 이 도구는 **제품으로서는 이미 잘 작동**하지만, 아래 점수는 "수백 개 팀이 공유 서비스로 쓸 때도 무너지지 않는가"를 기준으로 냉정하게 매겼다. + +| 영역 | 점수 | 한줄 평가 | +|---|---:|---| +| **종합 (가중평균)** | **64 / 100** | C+ · Solid Core, Hardened Edges Missing | +| 1. 코어 분리 & IoC | 72 | 내부 경계는 우수, "플러그인" 계약은 미성숙 | +| 2. 보안 & 제로트러스트 | 48 | 최저 방어선 미비 — 2번째로 취약 | +| 3. 확장성 & 성능 | 55 | 현재 규모엔 무해, 성장 시 재작성 필요 | +| 4. 타입 & 런타임 무결성 | 65 | 런타임 검증은 실재, 정적 타입 게이트는 부재 | +| 5. 거버넌스 & 컴플라이언스 | 80 | 최고 점수 — 정책-as-코드 설계가 진짜로 정교함 | +| 6. DX & 툴링 | 78 | 에이전트 우선 설계가 실제로 잘 작동함 | +| 7. 관측가능성 | 25 | 최저 점수 — 사실상 미착수 | +| 8. 테스트 & 릴리스 | 82 | 규모 대비 최고 수준의 릴리스 규율 | + +--- + +## 1. 즉시 착수 Top 3 Critical 태스크 + +전체 24개 항목 중 방치 시 **데이터 무결성 손실** 또는 **보안 사고**로 직결되는 3건. 모두 실제 소스에서 재현 가능한 결함이며, 아래 순서로 착수를 권고한다. + +### #1 — ADR 파일 쓰기 경로 전체가 원자적(atomic)이지 않다 + +**근거**: `create.py:136-174`, `identifiers.py:17-23`, `supersede.py:114-126` +**리스크**: 🔴 Critical + +`identifiers.next_id()`는 디렉터리를 glob하여 최댓값+1을 계산하고, 이후 `create.py`는 `target.exists()`를 확인한 뒤 `write_text()`로 직접 쓴다. 이 세 단계 사이에 잠금(lock)이 전혀 없어 두 프로세스가 동시에 실행되면 **같은 ADR 번호가 중복 채번**되거나 한쪽 쓰기가 유실된다. 더 심각한 건 `write_text` 자체가 "임시파일 작성 후 rename" 패턴이 아니라 직접 덮어쓰기라서, 프로세스가 쓰는 도중 강제 종료(OOM kill, CI 타임아웃)되면 **ADR 파일이 반쪽만 쓰인 채 손상**된다. `supersede.py`의 2-파일 갱신은 두 번째 쓰기 실패 시 첫 번째를 되돌리려 시도하지만, 프로세스가 그 사이에 죽으면 롤백 코드 자체가 실행되지 않아 두 ADR이 서로 어긋난 상태로 영구 고정된다. + +**솔루션**: + +```python +# scripts/core/atomic_io.py — 신규 모듈 +import os, sys, tempfile +from pathlib import Path +from contextlib import contextmanager + +if sys.platform == "win32": + import msvcrt + def _lock(fd): msvcrt.locking(fd, msvcrt.LK_LOCK, 1) + def _unlock(fd): + try: msvcrt.locking(fd, msvcrt.LK_UNLCK, 1) + except OSError: pass +else: + import fcntl + def _lock(fd): fcntl.flock(fd, fcntl.LOCK_EX) + def _unlock(fd): fcntl.flock(fd, fcntl.LOCK_UN) + +def atomic_write_text(path: Path, content: str, *, encoding="utf-8") -> None: + """임시 파일에 쓰고 fsync 후 os.replace — 크래시 시 원본 파일은 + 항상 이전 버전이거나 새 버전, 절대 반쪽이 아니다.""" + fd, tmp = tempfile.mkstemp(dir=path.parent, prefix=f".{path.name}.", suffix=".tmp") + try: + with os.fdopen(fd, "w", encoding=encoding) as f: + f.write(content); f.flush(); os.fsync(f.fileno()) + os.replace(tmp, path) # POSIX/Windows 모두 원자적 + except BaseException: + Path(tmp).unlink(missing_ok=True); raise + +@contextmanager +def adr_directory_lock(adr_dir: Path): + """ID 채번 + 파일 생성을 하나의 임계구역으로 묶는다 (TOCTOU 창 제거).""" + adr_dir.mkdir(parents=True, exist_ok=True) + lock_fd = os.open(adr_dir / ".adr-toolkit.lock", os.O_CREAT | os.O_RDWR) + try: + _lock(lock_fd) + yield + finally: + _unlock(lock_fd); os.close(lock_fd) +``` + +`create.py`는 `next_id()` 계산부터 `write_text`까지를 `with adr_directory_lock(adr_dir):` 블록 안에 넣고, 모든 쓰기 지점을 `atomic_write_text`로 교체한다. `supersede.py`의 2-파일 갱신도 같은 락 안에서 두 `atomic_write_text` 호출로 바뀌면, 프로세스가 어느 지점에서 죽어도 파일 시스템은 항상 "갱신 전" 또는 "갱신 후" 둘 중 하나의 유효한 상태에 머문다(롤백 코드 자체가 불필요해진다). + +**IMPACT**: 병렬 CI/에이전트 실행 시 ADR 번호 충돌, 크래시 시 저장소의 단일 진실 소스(decision log) 영구 손상 + +--- + +### #2 — ADR 본문이 두 개의 서로 다른 신뢰 경계를 뚫고 실행 가능 콘텐츠가 된다 + +**근거**: `index.py:97,109,121,127`, `rules/conflict.py:59-66` +**리스크**: 🔴 Critical + +(a) **Markdown 인젝션**: `index.py`의 README 생성 코드는 ADR의 `title`을 이스케이프 없이 `f"[{entry['id']} — {entry['title']}]({entry['filename']})"` 형태로 직접 삽입한다. 같은 모듈의 `render_mermaid`는 `_mermaid_label`에서 `html.escape`와 대괄호 치환을 하는데, README 인덱스 렌더러만 이 처리가 빠져 있다 — title에 `) [클릭](https://evil.example` 같은 문자열을 넣으면 인덱스의 링크 타깃을 조작할 수 있다. + +(b) **ReDoS**: `conflict.py::_content_pattern`는 ADR 저자가 작성한 `constraints:` 블록의 `pattern` 필드를 그대로 `re.compile`하여 diff의 모든 추가 라인에 대해 매칭한다. 타임아웃도 복잡도 검사도 없어 `(a+)+$` 류의 패턴 하나가 CI의 CHECK 단계를 무한정 멈추게 할 수 있다. + +**솔루션**: + +```python +# scripts/core/rendering.py 에 추가 +import re + +_MD_LINK_UNSAFE = re.compile(r"[\[\]\\]") + +def safe_md_link_text(text: str) -> str: + """[] \ 를 이스케이프하고 개행을 공백으로 접어 링크 구문 탈출을 막는다.""" + return _MD_LINK_UNSAFE.sub(r"\\\g<0>", str(text)).replace("\n", " ") +``` + +```python +# scripts/rules/conflict.py — 패턴 실행에 하드 타임아웃 +import signal, sys + +class RegexTimeout(Exception): pass + +def _guarded_search(regex, line, timeout_s=0.25): + if sys.platform == "win32": + return regex.search(line) # SIGALRM 부재 — 대신 컴파일 시점 정적 린트로 방어 + def _raise(*_): raise RegexTimeout() + prev = signal.signal(signal.SIGALRM, _raise) + signal.setitimer(signal.ITIMER_REAL, timeout_s) + try: + return regex.search(line) + finally: + signal.setitimer(signal.ITIMER_REAL, 0) + signal.signal(signal.SIGALRM, prev) +``` + +`_content_pattern`에서 `regex.search(line)` 호출을 `_guarded_search(regex, line)`로 교체하고 `RegexTimeout`을 `BAD_CONSTRAINTS` 경고로 강등한다. Windows에서는 `re` 표준 라이브러리에 타임아웃 훅이 없으므로, 컴파일 시점에 중첩 정량자(`(x+)+`, `(x*)*`) 패턴을 정적으로 거부하는 린터를 constraints.py 파싱 단계에 추가해 최소 방어선을 이중화해야 한다. + +**IMPACT**: README 링크 하이재킹(피싱 유도), CI에서 CHECK 무한 행(파이프라인 전체 블로킹) + +--- + +### #3 — 실패가 "무슨 일이 있었는지" 남기지 않는다 (관측가능성 부재) + +**근거**: `adr.py:186-195` (전역 예외 핸들러) +**리스크**: 🔴 Critical + +`main()`은 모든 예외를 `{"code": "INTERNAL_ERROR", "detail": str(exc)}`로 뭉개 stdout JSON 한 줄로만 내보낸다. 스택 트레이스는 어디에도 기록되지 않고, 구조화 로그·상관관계 ID·타이밍 정보가 전무하다. 24개 서브커맨드 중 어느 것이 몇 ms 걸렸는지, 어떤 파일에서 파싱이 실패했는지는 재현 전까지 알 수 없다 — CI에서 한 번 실패한 `check`를 사후 디버깅할 방법이 로컬 재현뿐이다. 이는 8대 영역 중 최저 점수(25/100)의 근본 원인이다. + +**솔루션**: + +```python +# scripts/core/telemetry.py — 신규 모듈 +import json, logging, os, sys, time, uuid + +class _JsonFormatter(logging.Formatter): + def format(self, r): + payload = { + "ts": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime(r.created)), + "level": r.levelname.lower(), + "operation": getattr(r, "operation", None), + "correlation_id": getattr(r, "cid", None), + "elapsed_ms": getattr(r, "elapsed_ms", None), + "msg": r.getMessage(), + } + if r.exc_info: + payload["exc_type"] = r.exc_info[0].__name__ + return json.dumps(payload, ensure_ascii=False) + +def get_logger(operation: str) -> logging.LoggerAdapter: + logger = logging.getLogger("adr_toolkit") + if not logger.handlers: + h = logging.StreamHandler(sys.stderr) # stdout은 JSON 결과 계약 전용이라 절대 오염시키지 않는다 + h.setFormatter(_JsonFormatter()) + logger.addHandler(h) + logger.setLevel(os.environ.get("ADR_TOOLKIT_LOG_LEVEL", "INFO")) + return logging.LoggerAdapter(logger, {"operation": operation, "cid": uuid.uuid4().hex[:12]}) +``` + +`adr.py::main`의 `except` 블록에서 이 로거로 `logger.exception(...)`을 남기고, 그 `correlation_id`를 JSON 에러 응답의 `errors[0].correlation_id`에 그대로 포함시킨다. stdout 계약(순수 JSON)은 그대로 유지되며, stderr에만 구조화 로그가 쌓이므로 기존 소비자(에이전트/CI)를 깨지 않는다. + +**IMPACT**: 프로덕션 장애 시 평균 복구 시간(MTTR) 통제 불가, 근본 원인 재현 불가능한 사고 다수 발생 + +--- + +## 2. 8대 영역별 세부 진단 + +### 2.1 아키텍처 탄력성 & 코어 분리 — 72/100 + +#### 1.1 Core-Plugin 경계 & IoC — 🟡 Medium + +**진단**: ✓ 강점: `core/`(도메인 규칙) · `rules/`(정책 평가) · `evidence/`(증거 수집) · `commands/`(유스케이스)의 계층 분리가 명확하고, `commands/check.py`는 `core.constraints`·`rules.conflict`를 의존성으로 주입받는 형태라 DIP를 자연스럽게 지킨다. 다만 "플러그인"은 실제로는 `adapters/*/plugin.json` 4종의 **정적 매니페스트**일 뿐, 런타임에 로드되는 확장점(hook, event bus)이 코어에 전혀 없다 — Hexagonal Architecture의 포트/어댑터라기보다는 "동일 스크립트에 대한 배포 경로 지정자"에 가깝다. + +**솔루션**: 진짜 확장점이 필요해지는 시점(예: 서드파티 CHECK 규칙 kind 추가)을 대비해 `rules/conflict.py`의 `evaluate_rule`을 레지스트리 팩토리로 승격한다: + +```python +_RULE_EVALUATORS: dict[str, Callable] = {} + +def register_rule_kind(kind: str): + def _wrap(fn): + _RULE_EVALUATORS[kind] = fn + return fn + return _wrap + +@register_rule_kind("forbidden_import") +def _content_pattern(rule, diff_files): ... + +def evaluate_rule(rule, diff_files, existing_paths): + handler = _RULE_EVALUATORS.get(rule.get("kind")) + return handler(rule, diff_files, existing_paths) if handler else None +``` + +**테스트 시나리오**: +- 미등록 `kind` 등록 후 `evaluate_rule` 호출 → 정상 평가되는지 회귀 없이 확인 +- 기존 6종 kind가 레지스트리 경유로도 동일 출력을 내는지 스냅샷 비교 + +#### 1.2 "플러그인" 라이프사이클 거버넌스 — 🟢 Low + +**진단**: 런타임 로딩·해제 개념 자체가 없으므로 좀비 프로세스/메모리 누수 위험은 실질적으로 **없다** — CLI는 커맨드 1회 실행 후 즉시 종료되는 단발성 프로세스 모델이다. 다만 `harness-parity` CI 잡(codex/gemini 실제 설치)이 "설치→실행→검증"을 자동화한 것은 사실상 라이프사이클 검증의 대체재 역할을 잘 해내고 있다. + +**솔루션**: 추가 조치 불요. 다만 Antigravity 어댑터만 CI에서 실제 설치가 검증되지 않는(README에 "수동 검증"으로 명시) 비대칭이 있으므로, Antigravity CLI가 공개 패키지 레지스트리를 지원하는 시점에 harness-parity 잡에 편입할 것을 백로그에 등록. + +**테스트 시나리오**: (해당 없음 — 현재 아키텍처에서 라이프사이클 결함 재현 불가) + +#### 1.3 API 안정성 & SemVer 계약 — 🟠 High + +**진단**: `scripts/sync_version.py`가 `VERSION` 한 곳을 4개 매니페스트에 전파하고 CI의 `version-drift` 잡이 이를 검증하는 점은 훌륭하다. 그러나 이것은 "버전 문자열 동기화"이지 **API 호환성 계약**이 아니다 — JSON 출력 스키마(`{ok, operation, errors, findings...}`)가 버전 간 바뀌어도 이를 감지·경고할 장치가 없다. 예를 들어 `check`의 `findings[].confidence` 값 집합이 바뀌면 이를 파싱하는 모든 에이전트/CI 스크립트가 조용히 깨진다. Deprecation 정책 문서도 부재. + +**솔루션**: 각 커맨드 출력에 대해 JSON Schema 골든 파일을 두고, PR에서 스키마 diff를 감지해 `MAJOR` 필요 여부를 자동 판정: + +```python +# tests/contract/test_output_contract.py +def test_check_output_matches_frozen_schema(): + schema = json.loads(Path("tests/contract/check.schema.json").read_text()) + result = check.run(make_args(...)) + jsonschema.validate(result, schema) # 필드 삭제/타입 변경 시 즉시 실패 +``` + +**테스트 시나리오**: +- 16개 커맨드 각각의 출력에 대한 골든 스키마 스냅샷 테스트 +- 필드 제거·타입 변경을 인위로 주입해 스키마 테스트가 실패하는지 검증(테스트의 테스트) + +--- + +### 2.2 샌드박싱 & 제로 트러스트 보안 — 48/100 + +#### 2.1 악의적 코드 격리 (Sandboxing) — 🟢 Low (재정의됨) + +**진단**: 실행되는 제3자 "플러그인 코드"가 존재하지 않으므로 VM/WASM 격리는 요구사항 자체가 성립하지 않는다. 실제 위협 표면은 "신뢰되지 않는 **ADR 파일 콘텐츠**"다 — `git diff` 대상 저장소, 다수 기여자가 작성하는 `constraints:` 블록이 여기 해당한다. 코드 실행 자체는 없으나(`eval`/`exec` 사용 없음 확인됨), 정규식 실행(2.3 참고)이 사실상의 "실행 가능 콘텐츠"다. + +**솔루션**: 격리 계층 신설 대신, `constraints:` 블록을 승인 권한이 있는 사람만 병합할 수 있도록 CODEOWNERS로 ADR 디렉터리를 보호할 것을 `CONTRIBUTING.md`에 명문화 — 코드가 아닌 **프로세스 통제**가 여기서는 더 적절한 방어선이다. + +**테스트 시나리오**: `grep -rn "eval(\|exec(\|subprocess.*shell=True" scripts/` — CI에 정적 게이트로 편입해 회귀 방지 + +#### 2.2 공급망 보안 — 🟠 High + +**진단**: 서드파티 npm 의존성이 없어(표준 라이브러리만 사용) 전통적 의미의 "의존성 취약점"은 표면적으로 적다 — 이는 강점이다. 그러나 설치 무결성 자체가 검증되지 않는다: `.claude-plugin/marketplace.json`·Codex/Gemini 매니페스트 어디에도 체크섬/서명이 없고, GitHub Release도 `softprops/action-gh-release`가 자동 생성 노트만 첨부할 뿐 아티팩트 서명이 없다. "복사해서 어디에나 설치"(`adapters/generic`)를 공식 배포 경로로 문서화한 점은 무결성 검증을 더 어렵게 만든다. + +**솔루션**: 릴리스 워크플로에 SHA-256 매니페스트 생성 및 (가능하면) Sigstore/cosign 서명 단계 추가: + +```yaml +# .github/workflows/release.yml 추가 스텝 +- name: Generate checksums + run: | + tar -czf adr-toolkit-skill.tar.gz skills/adr-toolkit + sha256sum adr-toolkit-skill.tar.gz > SHA256SUMS +- uses: sigstore/gh-action-sigstore-python@v3 + with: { inputs: adr-toolkit-skill.tar.gz } +``` + +**테스트 시나리오**: 릴리스 아티팩트 다운로드 후 `sha256sum -c SHA256SUMS` 검증을 릴리스 워크플로 자체의 마지막 스텝으로 추가(자기검증) + +#### 2.3 입력 검증 및 취약점 방어 — 🔴 Critical + +**진단**: ✓ 강점: `diff.py:29-34`는 `--end-of-options`를 사용해 `--since` 인자가 git 옵션으로 오인되는 인젝션(예: `--output=/tmp/PWNED`)을 이미 차단하고 있다 — 이 방어는 견고하고 모범적이다. ✓ 강점: `identifiers.validate_slug`는 `[a-z0-9-]+` 화이트리스트로 경로 조작 문자를 원천 차단한다. **그러나** §1의 Top-3 #2에 서술한 Markdown 인젝션(README 링크 하이재킹)과 ReDoS(제한 없는 사용자 정의 정규식)가 이 항목의 최대 결함이다. 추가로 `--dir`/`--root` 인자는 저장소 경계 밖 경로(`../../etc`)를 그대로 받아들여, 이 CLI가 향후 다중 테넌트 SaaS(예: PR 자동 검사 봇)로 wrapping될 경우 경로 탈출로 이어질 수 있다. + +**솔루션**: Top-3 #2의 코드에 더해, `repository_paths.resolve_from_root`에 경계 검사를 추가: + +```python +def resolve_from_root(root, path) -> Path: + candidate = (Path(root) / path).resolve() + root_resolved = Path(root).resolve() + if not candidate.is_relative_to(root_resolved): + raise ValueError(f"{path!r} escapes repository root {root!r}") + return candidate +``` + +**테스트 시나리오**: +- `--dir ../../etc/cron.d` 전달 시 `ValueError`/구조화 에러로 거부되는지 +- title에 `) [phish](http://evil` 포함된 ADR로 `index` 실행 후 생성된 README에 원본 링크 구문이 깨지지 않는지 +- 10자 입력으로 5초 이상 걸리는 병리적(catastrophic backtracking) 패턴을 `constraints:`에 넣고 `check`가 0.5초 내 타임아웃 경고로 종료되는지 + +--- + +### 2.3 대규모 데이터 확장성 & 성능 — 55/100 + +#### 3.1 대규모 모노레포 처리량 — 🟡 Medium + +**진단**: `adr_directory.iter_adr_files`는 매 커맨드 호출마다 `adr_dir.glob("*.md")`로 전체 디렉터리를 나열하고, `search.py`/`index.py`/`check.py`/`validate.py`는 각 파일을 `read_text()`로 동기 전체 로드한다. 스트리밍 파서 없음, 페이지네이션 없음. ADR은 본질적으로 사람이 쓰는 문서라 "수천 개"는 비현실적 규모지만(대형 조직도 보통 수백 개 미만), `search --limit`이 필터링 **후** 자르는 방식이라 결과 상한과 무관하게 항상 O(N) 전체 스캔이 발생하는 점은 실제로 개선 여지가 있다. + +**솔루션**: 즉각적인 아키텍처 재작성보다 우선순위는 낮음. N>500 도달 시를 대비해 §3.2의 캐시 계층 도입을 선행 조건으로 명시. + +**테스트 시나리오**: ADR 2,000개 픽스처로 `search`/`index` 실행 시간 벤치마크, CI에 회귀 임계값(예: 3초) 설정 + +#### 3.2 인덱싱 & 지연 평가 — 🟡 Medium + +**진단**: 캐싱 계층이 전혀 없다 — 같은 저장소에 대해 `search`를 10번 호출하면 10번 모두 전체 파일을 재파싱한다. Content-hash 기반 증분 캐시 부재로, CI에서 `validate` → `index` → `check`를 순차 실행하는 흔한 패턴(`CONTRIBUTING.md`가 권장하는 순서)이 동일 파일을 3번 파싱한다. + +**솔루션**: 파일 mtime+size 해시를 키로 하는 프로세스 로컬 캐시(단발성 CLI라 프로세스 간 캐시는 과잉설계)보다, 단일 실행 내에서 파싱 결과를 재사용하는 `functools.lru_cache` 우선 적용: + +```python +from functools import lru_cache + +@lru_cache(maxsize=None) +def _parse_cached(path_str: str, mtime_ns: int) -> tuple: + # mtime_ns를 키에 포함해 파일 변경 시 캐시 무효화 + return fm.parse(Path(path_str).read_text(encoding="utf-8")) +``` + +**테스트 시나리오**: +- 동일 프로세스 내 같은 ADR을 두 커맨드가 참조할 때 파일 읽기 호출 수가 1회로 줄었는지 mock으로 검증 +- 파일 수정 후 mtime이 바뀌면 캐시가 무효화되어 새 내용을 반영하는지 + +#### 3.3 동시성 제어 & 번호 채번 레이스 — 🔴 Critical + +**진단**: Top-3 #1과 동일 결함. `identifiers.next_id`와 `commands/exception.py::_next_id` 두 곳 모두 동일한 "glob → max+1" 패턴을 락 없이 반복 구현하고 있어, 여러 병렬 CI 파이프라인(예: 여러 PR이 동시에 `create` 실행)이나 다중 에이전트 세션이 동시에 ADR/Exception을 생성하면 ID 충돌이 발생한다. 현재 테스트 스위트(`test_create.py`)에는 동시성 재현 테스트가 전혀 없다 — `grep` 결과 `fcntl`/`filelock`/`tempfile`/`os.replace` 사용 이력이 코드베이스 전체에서 0건으로 확인됨. + +**솔루션**: Top-3 #1의 `adr_directory_lock`을 `exception.py::_next_id`에도 동일 적용 — 두 채번 로직을 `core/identifiers.py`의 공통 `allocate_sequential_id(dir, pattern)` 함수로 통합해 중복 구현 자체를 제거. + +**테스트 시나리오**: +- `multiprocessing.Pool`로 `create.run`을 20개 프로세스에서 동시 호출 → 생성된 ADR 번호 20개가 모두 유일한지 검증(현재 코드로는 이 테스트가 확정적으로 실패함) +- 락 보유 중 프로세스를 `SIGKILL`로 강제 종료 후 재실행 시 stale lock으로 데드락에 빠지지 않는지 + +--- + +### 2.4 타입 시스템 엄격성 & 런타임 무결성 — 65/100 + +#### 4.1 Type Variance & 계약 타이핑 — 🟠 High + +**진단**: TypeScript 제네릭/유니온 개념은 적용 불가(Python). Python 자체의 타입 힌트도 대부분의 함수 시그니처(`def run(args) -> dict`)에서 `args`가 `argparse.Namespace` 익명 객체라 IDE 자동완성이 사실상 동작하지 않는다. 반환 타입도 `dict`로만 선언되어 있어 `findings[].confidence` 같은 중첩 구조는 코드를 읽기 전까지 알 수 없다. CI 어디에도 `mypy`/`pyright` 게이트가 없다(검색 결과 0건). + +**솔루션**: `TypedDict`로 커맨드별 입력/출력 계약을 명시하고 CI에 `mypy --strict` 추가: + +```python +# scripts/core/contracts.py +from typing import TypedDict, Literal + +class CreateArgs(TypedDict, total=False): + input: str | None + interactive: bool + dir: str + root: str + locale: str | None + slug: str | None + dry_run: bool + +class CheckFinding(TypedDict): + adr_id: str + kind: Literal["related", "verified_violation", "review_required", "no_applicable_constraint"] + confidence: Literal["VERIFIED", "VIOLATED", "UNVERIFIABLE"] +``` + +**테스트 시나리오**: +- CI에 `mypy skills/adr-toolkit/scripts --strict` 잡 추가 후 baseline 오류 0건 확인 +- 16개 커맨드의 `run()` 시그니처를 `TypedDict` 인자로 순차 전환하며 회귀 테스트 통과 확인 + +#### 4.2 런타임 스키마 검증 — 🟡 Medium + +**진단**: ✓ 강점: `core/schema.py`·`core/exceptions.py`가 Zod/Valibot 없이도 실질적인 런타임 구조·타입·정규식 검증을 수행하며, 이는 컴파일 타임 검증이 아예 없는 Python 환경에서 **실제로 강제되는** 유일한 안전망이라 설계 의도가 명확하다. **결함**은 `schemas/adr.schema.json`·`schemas/exception.schema.json`이라는 JSON Schema 파일이 "외부 도구를 위한 문서"로 별도 존재하면서, 런타임 검증기(Python)와 **완전히 독립적으로 손으로 동기화**된다는 점이다(주석에 명시: "this module is the version actually enforced at runtime"). 두 정의가 갈라지는 순간(schema drift) 외부 도구는 통과하는데 실제 런타임은 거부하는 필드가 생길 수 있다. + +**솔루션**: JSON Schema를 **단일 진실 소스**로 승격하고 Python 검증기를 그로부터 생성 또는 그것으로 직접 검증하도록 역전: + +```python +# core/schema.py 를 jsonschema 라이브러리 기반으로 재작성 +import json, jsonschema +from pathlib import Path + +_SCHEMA = json.loads((Path(__file__).parents[2] / "schemas" / "adr.schema.json").read_text()) + +def validate_frontmatter(data: dict) -> list: + validator = jsonschema.Draft202012Validator(_SCHEMA) + return [e.message for e in sorted(validator.iter_errors(data), key=str)] +``` + +**테스트 시나리오**: JSON Schema 파일에 필드를 추가하고 Python 검증기를 갱신하지 **않은** 상태에서 drift 감지 테스트가 실패하는지(현재는 이 테스트 자체가 존재하지 않음 — 신규 작성 필요) + +#### 4.3 Defensive Error Architecture — 🟡 Medium + +**진단**: ✓ 강점: 도메인별 예외 클래스(`InvalidTransitionError`, `ConfigError`, `ConstraintsError`, `FrontmatterError`, `GitPathsError`)가 실제로 존재하고 각 커맨드가 이를 구조화된 `{code, detail}` JSON으로 변환하는 일관된 패턴을 따른다 — 매직 스트링이지만 최소한 **일관된** 매직 스트링이다. `adr.py:186`의 전역 캐치가 최후 방어선 역할을 하는 것도 견고하다. 결함은 이 5개 예외 클래스가 공통 베이스 클래스를 공유하지 않아 "이 커맨드가 던질 수 있는 예외 전체 목록"을 타입 시스템으로 추적할 수 없다는 것과, 스택 트레이스가 어디에도 보존되지 않는다는 것(§7과 중복 이슈). + +**솔루션**: 공통 `AdrToolkitError` 베이스로 통합해 `error_code`를 클래스 속성으로 승격: + +```python +class AdrToolkitError(Exception): + error_code: str = "UNKNOWN_ERROR" + def to_dict(self) -> dict: + return {"code": self.error_code, "detail": str(self)} + +class InvalidTransitionError(AdrToolkitError): + error_code = "INVALID_TRANSITION" +``` + +**테스트 시나리오**: 모든 커맨드의 `except` 절이 `AdrToolkitError`의 서브클래스만 개별 처리하고 나머지는 전역 핸들러로 위임하는지 정적 검사 + +--- + +### 2.5 엔터프라이즈 거버넌스 & 컴플라이언스 — 80/100 + +#### 5.1 ADR 상태 전이 머신 (FSM) — 🟢 Low + +**진단**: ✓ 강점: `core/lifecycle.py`는 교과서적인 화이트리스트 FSM이다(`proposed→{accepted,rejected}`, `accepted→{deprecated,superseded}`, 나머지는 종단 상태). `status.py`/`supersede.py` 둘 다 쓰기 전에 반드시 `validate_transition`을 통과해야 하므로 유효하지 않은 전이가 파일에 반영될 경로가 없다. 유일한 아쉬움은 "proposed → deprecated"(합의 없이 제안을 철회) 같은 실무에서 종종 필요한 전이가 빠져 있다는 점 정도다. + +**솔루션**: 필요 시 `ALLOWED_TRANSITIONS["proposed"]`에 `"deprecated"`를 추가하는 1줄 변경으로 충분 — 아키텍처 변경 불요. + +**테스트 시나리오**: 기존 `test_lifecycle.py`가 이미 전 전이 조합을 파라미터화 테스트 중 — 신규 전이 추가 시 동일 패턴으로 1건 추가 + +#### 5.2 CI/CD 게이트웨이 & Linter Rules — 🟢 Low + +**진단**: ✓ 강점: `check`/`validate` 커맨드가 headless JSON-only CLI로 설계되어 있고 `main()`의 `return 0 if result.get("ok") else 1`이 표준 CI Exit Code 계약을 정확히 지킨다. `constraints:` 블록의 6종 kind(`forbidden_import`, `required_path` 등)는 ArchUnit류 아키텍처 규칙 엔진과 사실상 동등한 표현력을 이미 갖췄다. 다만 이 규칙들이 코드 리뷰를 통과한 `constraints:` YAML 블록 하나에만 의존하므로, 규칙 자체의 오탈자(§2.3의 ReDoS 포함)를 잡는 lint가 CHECK 실행 시점이 아니라 ADR 작성 시점에 있으면 더 좋다. + +**솔루션**: `create`/`status` 커맨드 실행 직후 자동으로 새/변경 ADR의 `constraints:` 블록을 파싱해보는 사전 린트를 `validate`에 이미 통합된 흐름과 동일하게 `create.py` 종료 직전에도 실행(현재는 CHECK 실행 시점에야 발견됨). + +**테스트 시나리오**: 오탈자 있는 `constraints:`를 포함한 draft로 `create` 실행 시 파일 생성 전에 경고가 뜨는지 + +#### 5.3 의사결정 계보 추적 (Lineage) — 🟢 Low + +**진단**: ✓ 강점: `core/relationships.py::find_cycles`는 `supersedes` 엣지에 대해 정확한 DFS 기반 순환 탐지를 구현하고 있고, `validate.py`가 이를 `SUPERSESSION_CYCLE` 에러로 노출한다. Mermaid(`render_mermaid`)와 순수 SVG(`render_svg`, Node/브라우저 자동화 없이 결정론적 벡터 출력) 두 경로 모두 그래프 시각화를 지원하는 것은 이 규모의 도구치고 이례적으로 잘 갖춰진 기능이다. `supersession_mismatches`가 양방향 링크 불일치(A는 B를 supersede한다는데 B는 A에 의해 superseded 되었다고 안 적힌 경우)까지 잡아낸다. + +**솔루션**: 이미 우수. 유일한 개선점은 `render_mermaid`는 title을 이스케이프하는데 `index.py`의 README 렌더러는 안 하는 §2.3의 비일관성 — 공통 `safe_md_link_text` 헬퍼로 통일. + +**테스트 시나리오**: 3-ADR 순환(A supersedes B, B supersedes C, C supersedes A) 픽스처로 `find_cycles`가 정확히 `(A,B,C)` 튜플 1개를 반환하는지(기존 `test_relationships.py` 커버리지 확인됨) + +--- + +### 2.6 개발자 경험(DX) & 툴링 인체공학 — 78/100 + +#### 6.1 설정 복잡도 최소화 — 🟢 Low + +**진단**: ✓ 강점: `.adr-toolkit.json`(schema_version + locale) 하나로 시작해, `core/config.py::resolve_locale`이 `CLI 인자 → draft 값 → 저장소 설정 → 기본값` 순서의 명확한 cascading을 구현한다. Zero-config(`init` 한 줄)에서 8개 로케일 커스터마이징까지 계단식으로 확장되는 설계가 실제로 동작한다. `ALLOWED_KEYS` 화이트리스트가 알 수 없는 설정 키를 명시적으로 거부하는 것도 좋은 습관(조용한 오타 허용 방지). + +**솔루션**: 현재 설정 항목이 2개뿐이라 과설계 위험이 더 크다 — 추가 조치 불요, 항목이 늘어날 때(예: CHECK 규칙 severity 임계값) 동일 cascading 패턴을 재사용할 것. + +**테스트 시나리오**: 기존 `test_config.py`가 우선순위 4단계를 이미 개별 테스트 중 — 신규 설정 키 추가 시 동일 매트릭스 확장 + +#### 6.2 CLI 인터랙션 & 가독성 — 🟡 Medium + +**진단**: 모든 커맨드가 `--json`이 강제된 기계 친화적 출력이라는 점은 명확한 설계 선택이다(사람이 아니라 에이전트가 1차 소비자). `create --interactive`가 에이전트 없이도 터미널에서 직접 인터뷰를 진행하는 폴백을 제공하는 것도 실용적이다. 반면 **사람**이 직접 CLI를 두드릴 때를 위한 배려는 약하다: 색상 강조, 진행 표시줄, `--quiet`(현재는 JSON이 유일 출력이라 quiet 자체가 무의미), TTY 감지 분기가 전무하다. + +**솔루션**: TTY 감지 시에만 최소한의 사람용 요약 라인을 stderr에 추가(stdout 계약은 불변 유지): + +```python +if sys.stderr.isatty() and not os.environ.get("ADR_TOOLKIT_NO_COLOR"): + print(f"\033[2m→ {args.operation} {'ok' if result.get('ok') else 'FAILED'}\033[0m", file=sys.stderr) +``` + +**테스트 시나리오**: stdout이 파이프로 리다이렉트된 상태(`isatty()==False`)에서 stderr에 추가 출력이 전혀 없는지(기존 소비자 회귀 방지) + +#### 6.3 스캐폴딩 및 플러그인 SDK — 🟠 High + +**진단**: 새 AI 하네스를 위한 어댑터를 만들려는 사람에게 제공되는 것은 `adapters/generic/README.md` 한 장뿐이다 — 공식 SDK, 매니페스트 스캐폴딩 CLI, Mock Context 테스트 킷이 없다. 4개 기존 어댑터(Claude/Codex/Gemini/Antigravity)가 사실상의 참고 구현 역할을 하고는 있지만, 신규 기여자는 4개 매니페스트 형식을 손으로 비교하며 5번째를 작성해야 한다. `tests/unit/test_*_adapter.py` 4종이 각 매니페스트의 구조를 검증하는 건 좋지만, 이 자체가 "SDK 부재"의 방증이다(테스트가 매번 새로 작성되지, 공유 검증기가 없음). + +**솔루션**: 공통 어댑터 매니페스트 검증기를 추출해 신규 어댑터 작성자가 재사용하도록 공개: + +```python +# scripts/adapter_sdk.py +def validate_adapter_manifest(manifest: dict, *, required_fields=("name","version","description")) -> list: + return [f"missing {f}" for f in required_fields if f not in manifest] + +# adapters/README.md 에 "5번째 하네스 추가하기" 튜토리얼 + 이 검증기 사용법 추가 +``` + +**테스트 시나리오**: 4개 기존 매니페스트가 신규 `validate_adapter_manifest`를 통과하는지(기존 4개 테스트를 이 공유 함수 호출로 리팩터링) + +--- + +### 2.7 관측 가능성 & 원격 측정 — 25/100 + +#### 7.1 구조화된 로깅 — 🔴 Critical + +**진단**: Top-3 #3과 동일. `logging` 모듈 import 자체가 코드베이스 전체에서 0건(`grep -rn "import logging"` 결과 없음). 모든 진단 정보는 최종 JSON 응답의 `warnings`/`errors` 배열뿐이며, 이는 "결과 보고"이지 "실행 과정 로그"가 아니다. 상관관계 ID, 로그 레벨 세분화, JSON 구조화 로그 모두 부재. + +**솔루션**: Top-3 #3의 `telemetry.py` 참조. + +**테스트 시나리오**: `ADR_TOOLKIT_LOG_LEVEL=debug` 설정 시 stderr에 JSON Lines 로그가 나오고 stdout은 순수 결과 JSON만 유지되는지 + +#### 7.2 프로파일링 & 진단 모드 — 🟠 High + +**진단**: 실행 시간 측정, 병목 리포트, 메모리 프로파일링 커맨드 모두 없음. §3의 확장성 주장(느리다/안 느리다)조차 현재는 측정 도구가 없어 **검증 불가능한 주장**이다 — 이 항목의 부재가 §3 진단의 신뢰도 자체를 낮춘다. + +**솔루션**: `adr.py::main`에 선택적 타이밍 계측을 추가하고 `--diagnostic` 플래그로 노출: + +```python +start = time.perf_counter() +result = HANDLERS[args.operation](args) +if getattr(args, "diagnostic", False): + result["_diagnostics"] = {"elapsed_ms": round((time.perf_counter()-start)*1000, 1)} +``` + +**테스트 시나리오**: ADR 100/1,000/2,000개 픽스처에서 `--diagnostic` 출력의 `elapsed_ms`를 CI 아티팩트로 기록해 회귀 추세를 추적 + +--- + +### 2.8 테스트 완전성 & 릴리스 엔지니어링 — 82/100 + +#### 8.1 테스트 피라미드 구성 — 🟡 Medium + +**진단**: ✓ 강점: 유닛 42개 + 통합 7개 파일, 약 5,200줄의 테스트 코드가 `tmp_path` 기반 **실제 파일시스템**에서 동작하는 진짜 E2E에 가깝다(모킹 남용 없음). `harness-parity` CI 잡은 실제 Codex/Gemini CLI를 설치해 어댑터를 검증하는, 이 규모 오픈소스에서는 보기 드문 수준의 통합 테스트다. **결함**: `pytest-cov`/coverage 측정 자체가 CI 어디에도 없다 — "Branch 90%+"라는 목표를 애초에 **측정할 수 없다**. 커버리지 수치가 없으므로 실제 커버리지가 90%든 60%든 현재는 아무도 모른다. + +**솔루션**: + +```yaml +# .github/workflows/test.yml 에 추가 +- run: pip install pytest pytest-cov +- run: python -m pytest tests/unit tests/integration --cov=scripts --cov-branch --cov-fail-under=85 --cov-report=xml +``` + +**테스트 시나리오**: coverage 도입 직후 baseline 측정 → 85% 미만 모듈 식별 → 우선순위화된 보강 목록 작성 + +#### 8.2 카오스 & 엣지 케이스 복원력 — 🟠 High + +**진단**: ✓ 강점: 손상된 프론트매터, 깨진 링크, 순환 참조, 만료된 예외 등 "문서가 잘못됐을 때"의 우아한 성능 저하(graceful degradation)는 `check.py`·`index.py`·`locale.py` 전반에 걸쳐 의도적으로 잘 설계되어 있다(예: 로케일 파일 파싱 실패 시 크래시 대신 조용히 fallback). **결함**: "플러그인이 무한 루프에 빠지거나 크래시"라는 원 질문은 실행되는 플러그인이 없으므로 성립하지 않지만, 실제 대응 개념인 "프로세스가 쓰기 도중 죽었을 때"의 복원력은 Top-3 #1에서 지적한 대로 전무하다 — 이것이 진짜 카오스 시나리오다. + +**솔루션**: Top-3 #1 적용 후, kill -9 카오스 테스트를 테스트 스위트에 추가. + +**테스트 시나리오**: `os.fork()` 후 자식 프로세스가 `atomic_write_text` 중간에 `os.kill(pid, SIGKILL)`로 강제 종료됐을 때, 부모가 확인하는 ADR 파일이 항상 파싱 가능한(이전 또는 이후) 유효 상태인지 + +#### 8.3 Cross-Platform & 런타임 호환성 — 🟢 Low + +**진단**: ✓ 강점: CI 매트릭스가 `ubuntu-latest / macos-latest / windows-latest` × `Python 3.9 / 3.12`를 실제로 돌린다(`test.yml` 확인) — 이는 요청서가 요구한 항목을 이미 충족하는 몇 안 되는 사례다. `git_paths.py`가 `core.quotePath=false`로 비-ASCII 경로를, `-z`로 개행 포함 파일명을 안전 처리한다. Node/Deno/Bun 다중 런타임 지원은 해당 사항 없음(Python 전용 도구). 유일한 갭은 Windows에서 `fcntl` 기반 락(Top-3 #1 해결책)이 그대로 동작하지 않는다는 점 — 이미 `msvcrt` 분기로 코드 예시에 반영함. + +**솔루션**: 추가 조치 불요. Top-3 #1의 잠금 구현이 Windows/POSIX 양쪽을 이미 분기 처리했는지 CI 매트릭스로 재확인. + +**테스트 시나리오**: Windows 러너에서 신규 `adr_directory_lock`으로 동시 `create` 20회 실행 후 ID 유일성 검증(기존 3-OS 매트릭스에 자동 편입됨) + +#### 8.4 오픈소스 기여 거버넌스 — 🟢 Low + +**진단**: ✓ 강점: `CONTRIBUTING.md`·`CODE_OF_CONDUCT.md`·`SECURITY.md` 3종 모두 구비되어 있고, `SECURITY.md`의 "Scope"가 "path handling", "release workflow", "plugin manifest" 등 실제 코드 위협 표면과 정확히 일치하게 작성되어 있어 형식적 문서가 아니다. Git Flow(`develop`/`master`/`release/*`)와 태그 기반 릴리스가 `AGENTS.md`에 명문화되고 `release.yml`이 태그-VERSION 일치를 강제한다. 다만 Semantic Commit 강제(commitlint 등)나 Changesets/semantic-release 같은 **자동 버전 산정**은 없다 — 버전은 인간이 수동으로 올린다(문서에 "No auto version bump"로 명시된 의도적 선택). + +**솔루션**: 수동 버전 관리가 이 규모에서는 합리적 선택이므로 강제 도입 비권장. 다만 PR 제목에 Conventional Commits 형식을 요구하는 경량 CI 체크(예: `amannn/action-semantic-pull-request`) 정도는 비용 대비 효과가 높음. + +**테스트 시나리오**: PR 제목이 `feat:`/`fix:`/`docs:` 접두어 없이 열렸을 때 체크가 실패하는지 + +--- + +## 3. 우선순위 로드맵 + +24개 항목을 리스크 레벨과 착수 난이도로 재정렬한 실행 순서. + +| 순서 | 항목 | 리스크 | 왜 이 순서인가 | +|---|---|---|---| +| 1주차 | 3.3 / Top-3 #1 — 원자적 쓰기 + ID 락 | 🔴 Critical | 데이터 무결성은 다른 모든 기능의 전제조건. 신규 모듈 1개(`atomic_io.py`)로 5개 커맨드에 즉시 적용 가능. | +| 1주차 | 2.3 / Top-3 #2 — Markdown 이스케이프 + ReDoS 가드 | 🔴 Critical | 기존 `_mermaid_label` 패턴을 재사용만 하면 되는 낮은 난이도 대비 높은 임팩트. | +| 2주차 | 7.1 / Top-3 #3 — 구조화 로깅 | 🔴 Critical | 이후 모든 항목의 디버깅 가능성을 좌우 — 조기 도입할수록 나머지 작업의 비용이 줄어든다. | +| 2주차 | 8.1 — 커버리지 측정 도입 | 🟠 High | CI 설정 한 줄. 이후 리팩터링(§1.3, §4.1)의 안전망이 된다. | +| 3주차 | 1.3 — 출력 계약 스키마 고정 | 🟠 High | 4.2의 JSON Schema 단일화와 작업을 공유할 수 있어 묶어서 진행. | +| 3주차 | 4.2 — JSON Schema를 단일 진실 소스로 역전 | 🟡 Medium | 1.3과 동일 파일을 다루므로 순차 진행이 자연스러움. | +| 4주차 | 2.2 — 릴리스 아티팩트 체크섬/서명 | 🟠 High | 릴리스 워크플로 변경은 배포본이 늘어나기 전(현재 v0.2.0)에 도입하는 편이 마이그레이션 비용이 낮음. | +| 백로그 | 6.3 — 어댑터 SDK 추출 | 🟠 High | 5번째 하네스 요청이 실제로 들어오는 시점까지 지연 가능(YAGNI). | +| 백로그 | 4.1 — mypy 전면 도입 | 🟠 High | 점진적 도입 가능하나 전체 적용은 공수가 커 커버리지 안전망(8.1) 확보 후 진행. | + +--- + +## 방법론 한계 + +본 감사는 정적 코드 검토와 CI 설정 분석에 기반하며, 런타임 부하 테스트나 실제 침투 테스트는 수행하지 않았다. 점수는 감사자의 판단이 반영된 정성 평가이며, 정량 지표(커버리지 %, 응답 시간 ms)는 §7.2/§8.1에서 지적한 대로 현재 도구 자체에 계측이 없어 다수 항목에서 "측정 불가"를 "가정된 안전"으로 대체하지 않고 명시적으로 리스크로 처리했다. diff --git a/handoff.md b/handoff.md index 9ff1ccf..dd8a591 100644 --- a/handoff.md +++ b/handoff.md @@ -1,93 +1,97 @@ # handoff.md -## Current task (2026-08-31) - -**v0.2.0 is released.** The full Git Flow release gate that -`improvements.md` had listed as P0 is done: - -1. PR #3 (`feature/project-roadmap-implements` -> `develop`) merged at - `36af56c`. -2. `release/v0.2.0` cut from `develop`, PR #4 (`release/v0.2.0` -> `master`) - merged at `4b5dded`. -3. `v0.2.0` tag pushed on `master`; `.github/workflows/release.yml` ran - clean (tests, sync-check, tag/VERSION match) and published - https://github.com/SHcommit/ADR-toolkit/releases/tag/v0.2.0. -4. `feature/project-roadmap-implements` and `release/v0.2.0` deleted - locally and on origin (both fully merged first, verified with - `git merge-base --is-ancestor`). - -This doc-only cleanup (`changelog.md`, `improvements.md`, this file) is on -branch `docs/post-release-wrapup`, based on current `origin/develop` -(`36af56c`) -- not yet pushed or PR'd. - -Session summary (everything shipped in v0.2.0 beyond what was already on -`develop` before this session): - -- ADR relationship graph: `adr.py graph --format mermaid|svg|both`, - Mermaid embed in `adr.py index`. Recorded as ADR-0011. -- Public repository hygiene: `CONTRIBUTING.md`, `SECURITY.md`, - `.github/PULL_REQUEST_TEMPLATE.md`, `CODE_OF_CONDUCT.md`, - `.github/ISSUE_TEMPLATE/{bug_report,feature_request}.md`. -- Harness parity re-verified end-to-end against the real CLIs (Codex - 0.151.0, Gemini 0.46.0, Antigravity's `agy` 1.1.13 -- previously - undocumented as available). New `harness-parity` CI job automates the - Codex/Gemini checks; confirmed green on GitHub's real `ubuntu-latest` - runner (not just locally) before it was trusted. Antigravity stays - manual-only (`agy` has no package-registry distribution). -- `project-roadmap.md` audited against the codebase and narrowed to what's - actually still open, including a recorded decision *not* to build a - SessionStart-style hook right now even though Codex/Gemini both gained - hook extension points this session -- no usage evidence supports it, and - it cuts against the project's own minimal-interruption principle. +## Current task (2026-09-01) + +Executing `docs/superpowers/plans/2026-09-01-critical-hardening.md` -- +implementation plan for the 4 Critical-risk findings from +`docs/adr-toolkit-audit-report.md` (also tracked in `improvements.md`'s +`## Open` -> `### Critical` section): + +1. Atomic file writes + ID allocation lock (`core/atomic_io.py`, wired into + `create.py`, `exception.py`, `supersede.py`). +2. ReDoS timeout guard for author-supplied regex (`rules/conflict.py`). +3. Markdown link-injection escape for ADR titles in the generated + `docs/decisions/README.md` (`core/rendering.py`, `commands/index.py`). +4. Structured stderr logging with correlation IDs (`core/telemetry.py`, + wired into `adr.py`'s global exception handler). + +The plan is split into 8 tasks (Task 1-7 = one deliverable each, Task 8 = +close out the backlog docs). Each task ends with its own commit, so +**`git log --oneline` against the plan's task list is the source of truth +for what's already done** if this session is interrupted -- check which of +these commit messages exist before resuming (and cross-check the plan +file's own `- [ ]`/`- [x]` checkboxes, which are updated as steps land): + +- `feat: add atomic write and directory lock primitives` (Task 1) +- `fix: make ADR creation race-free under concurrent invocation` (Task 2) +- `fix: make exception creation race-free under concurrent invocation` (Task 3) +- `fix: make SUPERSEDE writes atomic and lock-protected` (Task 4) +- `fix: add ReDoS timeout guard to CHECK's author-supplied regex patterns` (Task 5) +- `fix: escape ADR titles in generated README to prevent link injection` (Task 6) +- `feat: add structured stderr logging with correlation IDs` (Task 7) +- `docs: close out Critical hardening backlog items` (Task 8) + +This session chose **inline execution** (`superpowers:executing-plans`), +not subagent-driven -- a fresh session resuming should do the same unless +the owner says otherwise. + +## Scope for this worktree + +Excluded here, being handled elsewhere -- do not touch: + +- Domains 1 (core/plugin architecture) and 5 (governance/FSM) from the + audit report -- already scored 72/80, mostly "no action needed" per the + audit itself. +- Anything Antigravity (`agy`) adapter-related -- owner is working on this + in another branch. +- Automatic version sync -- owner is working on this in another worktree; + as a direct consequence, **do not touch `.github/workflows/release.yml` + for any reason**. Two backlog items (High: supply-chain checksums/ + signing; a note under 8.4 about auto-version-bump direction) were + deliberately deferred to that other worktree for exactly this reason -- + see the "(다른 워크트리 확인)" flags in `improvements.md`. +- README prose (root README.md, `adapters/*/README.md` content) -- another + worktree. Task 6's fix to `commands/index.py` is a security fix in the + *generator code* for `docs/decisions/README.md`, not README prose, and + correctly stays in scope here -- don't confuse the two if asked to skip + "README work". ## Next step -1. Push `docs/post-release-wrapup`, open a PR into `develop`, merge once - CI is green (small doc-only diff: `changelog.md` unreleased-section - entries for this session's work, `improvements.md`'s Open section - cleared). -2. This worktree currently sits on `docs/post-release-wrapup`; `develop` - itself is checked out in another worktree - (`/Users/yangseunghyeon/Development/ADR-toolkit`), so `git checkout - develop` here will fail -- branch from `origin/develop` instead, the - way this branch was created. -3. `project-roadmap.md` has no unblocked items left: every remaining - section (Conflict detection depth, ADR navigation and scale, - Internationalization, Public and enterprise governance, Ecosystem - integration, Lifecycle research) is gated on usage evidence that - doesn't exist yet, or -- for Public and enterprise governance - specifically -- on the repository actually going public, which the - owner has deferred deliberately ("조만간 할거야", not now). Don't start - any of them without a fresh signal (real user report, real scale, real - non-English contributor) or explicit owner direction. - -Do not perform merge, release branch, push, or tag operations without -explicit owner approval -- that approval was given and executed this -session for v0.2.0 specifically; it does not carry forward to future -releases. - -## Latest local verification - -At `v0.2.0` tag / `master` HEAD (`4b5dded`): - -- `python3 -m pytest -q` -> `395 passed` -- `python3 scripts/sync_version.py --check` -> exit 0 -- GitHub Actions `release` workflow (run 33367157711): tests, sync-check, - tag/VERSION match, and `Create GitHub Release` all green, ~22s total. -- Both `harness-parity` CI runs on PR #3 and PR #4 passed on GitHub's real - `ubuntu-latest` runner, matching local dry-run output exactly. +If resuming: open `docs/superpowers/plans/2026-09-01-critical-hardening.md`, +find the first unchecked step, confirm against `git log` that its task's +commit doesn't already exist, and continue from there with +`superpowers:executing-plans`. + +After Task 8 lands, the remaining backlog (`improvements.md`'s High/Medium +sections) is unscheduled -- ask the owner before starting any of it. The +Critical-only scope for this pass, and the domain/worktree exclusions +above, were the owner's explicit calls in conversation, not something +derivable from the audit report alone. + +## Verification + +Full suite: `python3 -m pytest tests/unit tests/integration -v`. +Baseline before this session's changes: 395 passed. Expect it to grow by +roughly 15-18 tests across Tasks 1-7 (see the plan file's per-task test +files for the exact count). ## Open risks -- `adr.py check --uncommitted` reports a `VIOLATED` finding for superseded - ADR-0003 when changes touch `index.py`; this is expected (ADR-0006 - supersedes ADR-0003) and was already noted in PR review, not a code - defect. -- CHECK deliberately cannot prove prose, business rationale, or - organizational claims; those remain human-review evidence. -- Search/relationship matching is deterministic substring/exact/prefix - matching, untested at real scale (11 ADRs today). Revisit roadmap scale - items only if an adopting repository actually reaches hundreds of ADRs. -- GitHub branch/tag protection is intentionally unavailable on the current - private plan; configure and API-verify after the repository goes public - (owner has deferred this deliberately, not blocked on it). +- The ReDoS guard (Task 5) is POSIX-only (`signal.SIGALRM`); Windows CI is + unaffected but unguarded against catastrophic-backtracking patterns -- + a known, documented gap in the audit report, not a regression introduced + by this work. +- `supersede.py`'s two-file update (Task 4) guarantees each individual + file is never torn by a mid-write crash, but does not guarantee the + *pair* stays consistent if the process is killed between the two atomic + writes -- true two-phase commit across files was explicitly scoped out + (see the plan's Task 4 code comments). +- (carried over from the audit, still true) CHECK deliberately cannot + prove prose, business rationale, or organizational claims; those remain + human-review evidence. +- (carried over, still true) GitHub branch/tag protection is unavailable + on the current private plan; revisit once the repository goes public -- + owner's stated plan is to do that after most audit findings are done and + the version bumps to 1.0.0 (see project memory + `project_v1_public_release_plan`). diff --git a/improvements.md b/improvements.md index ce7bdca..fe25aef 100644 --- a/improvements.md +++ b/improvements.md @@ -5,7 +5,81 @@ Concrete implementation backlog. Unscheduled product bets belong in ## Open -Nothing open right now. +Backlog derived from `docs/adr-toolkit-audit-report.md`. Scope for this +worktree excludes domains 1 (core/plugin architecture) and 5 (governance/ +FSM) — already scored 72/80 and mostly "no action needed" in the audit — +plus anything Antigravity-adapter-related, automatic version sync, and +README prose, which are being handled in other worktrees/branches. The two +items below flagged "(다른 워크트리 확인)" touch files those efforts may +also touch. + +### Critical + +**구현 계획**: `docs/superpowers/plans/2026-09-01-critical-hardening.md` +(Task 1-8, TDD 단계별). 세션이 끊겨도 이 계획 파일의 체크박스 + +`git log --oneline`에 남는 태스크별 커밋 메시지로 어디까지 됐는지 바로 +알 수 있다 — 상세 재개 절차는 `handoff.md` 참고. + +- [ ] **원자적 파일 쓰기 + ID 채번 락** — `identifiers.py`, `create.py`, + `exception.py`, `supersede.py`; 신규 `core/atomic_io.py`. 락 없는 + glob→max+1 채번과 비원자적 `write_text`를 `adr_directory_lock` + + `atomic_write_text`로 교체. (감사 보고서 §1 Top-3 #1, §2.3 3.3) +- [ ] **ADR 정규식 ReDoS 가드** — `rules/conflict.py`. `constraints:`의 + `pattern` 필드 실행에 하드 타임아웃(POSIX: SIGALRM, Windows: 정적 + 중첩 정량자 린트). (감사 보고서 §1 Top-3 #2, §2.2 2.3) +- [ ] **생성된 README 링크 이스케이프** — `core/rendering.py`, + `commands/index.py`. `render_mermaid`의 `_mermaid_label`처럼 title을 + 이스케이프해 `docs/decisions/README.md` 생성 시 링크 하이재킹 방지. + 자동 생성 코드 수정이며 사람이 쓰는 README 문서 작업과는 무관. + (감사 보고서 §1 Top-3 #2, §2.2 2.3) +- [ ] **구조화 로깅** — 신규 `core/telemetry.py`, `adr.py`. stderr에 + JSON 로그(operation, correlation_id, exception type) 추가, stdout의 + 순수 JSON 결과 계약은 불변 유지. (감사 보고서 §1 Top-3 #3, §2.7 7.1) + +### High + +- [ ] **저장소 경로 탈출 방지** — `core/repository_paths.py`. + `--dir`/`--root`가 저장소 루트 밖을 가리키지 못하도록 경계 검사. + (감사 보고서 §2.2 2.3) +- [ ] **테스트 커버리지 측정 도입** — `.github/workflows/test.yml`에 + `pytest-cov --cov-branch --cov-fail-under=85` 추가. `release.yml`은 + 건드리지 않음. (감사 보고서 §2.8 8.1) +- [ ] **mypy + TypedDict 계약 타이핑** — 신규 `core/contracts.py`, CI에 + `mypy --strict` 게이트. (감사 보고서 §2.4 4.1) +- [ ] **진단/타이밍 모드** — `adr.py`에 `--diagnostic` 플래그로 실행 + 시간 계측 노출. (감사 보고서 §2.7 7.2) +- [ ] **카오스(SIGKILL) 복원력 테스트** — 원자적 쓰기 완료 후, 쓰기 + 도중 강제 종료 시 ADR 파일이 항상 유효 상태인지 검증하는 테스트 추가. + (감사 보고서 §2.8 8.2) +- [ ] **어댑터 매니페스트 검증기 추출 (코드만)** — 신규 + `scripts/adapter_sdk.py`의 `validate_adapter_manifest`. 튜토리얼 + 문서화는 README 작업 쪽에서 처리. (감사 보고서 §2.6 6.3) +- [ ] *(다른 워크트리 확인)* **공급망 보안(체크섬/서명)** — + `.github/workflows/release.yml`에 SHA-256/Sigstore 서명 단계. 자동 + 버전 동기화 작업이 같은 파일을 건드릴 수 있어 그쪽에 붙이는 것을 권장. + (감사 보고서 §2.2 2.2) +- [ ] *(다른 워크트리 확인)* **8.4 자동 버전 산정 방향 재검토** — 감사 + 보고서 원 권고는 "semantic-release류 자동 버전 산정 강제 도입 + 비권장"이었음. 진행 중인 자동 버전 동기화 작업 방향과 배치되지 않는지 + 확인. (감사 보고서 §2.8 8.4) + +### Medium + +- [ ] **런타임 스키마 단일 진실 소스화** — `core/schema.py`를 + `schemas/adr.schema.json`/`exception.schema.json` 기반 `jsonschema` + 검증으로 재작성해 스키마 드리프트 제거. (감사 보고서 §2.4 4.2) +- [ ] **공통 에러 베이스 클래스** — `AdrToolkitError`로 기존 5개 예외 + 클래스 통합, `error_code`를 클래스 속성화. 구조화 로깅 작업과 함께 + 진행하면 자연스러움. (감사 보고서 §2.4 4.3) +- [ ] **출력 계약 스키마 고정(골든 파일)** — 16개 커맨드 출력에 대한 + JSON Schema 스냅샷 테스트, 4.2 작업과 파일 공유 가능. (감사 보고서 + §2.1 1.3 — 도메인 1 제외 대상이지만 4.2와 묶어 진행 시 예외적으로 포함) +- [ ] **파싱 결과 캐시** — `functools.lru_cache` 기반 프로세스 내 + 재파싱 제거. (감사 보고서 §2.3 3.2) +- [ ] **대량 ADR 벤치마크** — 2,000개 픽스처로 `search`/`index` 실행 + 시간 측정, CI 회귀 임계값 설정. (감사 보고서 §2.3 3.1) +- [ ] **CLI TTY 인지 출력** — stderr에 사람이 읽을 요약 라인(비-TTY + 시 무출력). (감사 보고서 §2.6 6.2) ## Done From fc46830afb6c5021d37feea7fa94b3018d64c659 Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 09:48:36 +0900 Subject: [PATCH 10/58] feat: add atomic write and directory lock primitives New core/atomic_io.py: atomic_write_text() (temp-file + os.replace, so a mid-write crash never leaves a torn file) and adr_directory_lock() (a cross-process advisory lock -- fcntl on POSIX, msvcrt on Windows) for serializing ID allocation + writes. Not yet wired into any command. --- skills/adr-toolkit/scripts/core/atomic_io.py | 66 +++++++++++++++++++ tests/unit/test_atomic_io.py | 67 ++++++++++++++++++++ 2 files changed, 133 insertions(+) create mode 100644 skills/adr-toolkit/scripts/core/atomic_io.py create mode 100644 tests/unit/test_atomic_io.py diff --git a/skills/adr-toolkit/scripts/core/atomic_io.py b/skills/adr-toolkit/scripts/core/atomic_io.py new file mode 100644 index 0000000..94fd6f0 --- /dev/null +++ b/skills/adr-toolkit/scripts/core/atomic_io.py @@ -0,0 +1,66 @@ +"""Atomic, lock-protected file writes for ADR Toolkit's mutating commands. + +Every command that writes ADR/exception files must hold `adr_directory_lock` +across its read-compute-write sequence (ID allocation, existence checks) and +write file contents through `atomic_write_text` -- never `Path.write_text` +directly. This closes the TOCTOU window between "compute next ID" and +"create file" under concurrent invocation, and guarantees a process killed +mid-write leaves the previous valid file in place rather than a truncated +one. +""" +import os +import sys +import tempfile +from contextlib import contextmanager +from pathlib import Path + +if sys.platform == "win32": + import msvcrt + + def _lock(fd: int) -> None: + msvcrt.locking(fd, msvcrt.LK_LOCK, 1) + + def _unlock(fd: int) -> None: + try: + msvcrt.locking(fd, msvcrt.LK_UNLCK, 1) + except OSError: + pass +else: + import fcntl + + def _lock(fd: int) -> None: + fcntl.flock(fd, fcntl.LOCK_EX) + + def _unlock(fd: int) -> None: + fcntl.flock(fd, fcntl.LOCK_UN) + + +def atomic_write_text(path: Path, content: str, *, encoding: str = "utf-8") -> None: + path.parent.mkdir(parents=True, exist_ok=True) + fd, tmp_name = tempfile.mkstemp(dir=path.parent, prefix=f".{path.name}.", suffix=".tmp") + tmp_path = Path(tmp_name) + try: + with os.fdopen(fd, "w", encoding=encoding) as f: + f.write(content) + f.flush() + os.fsync(f.fileno()) + os.replace(tmp_path, path) # atomic rename on both POSIX and Windows + except BaseException: + tmp_path.unlink(missing_ok=True) + raise + + +@contextmanager +def adr_directory_lock(directory: Path): + """Serialize ID allocation + writes for one ADR/exceptions directory + across processes. The lock file lives inside `directory` itself so a + fresh clone or a brand-new `docs/decisions/` needs no extra setup.""" + directory.mkdir(parents=True, exist_ok=True) + lock_path = directory / ".adr-toolkit.lock" + fd = os.open(lock_path, os.O_CREAT | os.O_RDWR) + try: + _lock(fd) + yield + finally: + _unlock(fd) + os.close(fd) diff --git a/tests/unit/test_atomic_io.py b/tests/unit/test_atomic_io.py new file mode 100644 index 0000000..c952c74 --- /dev/null +++ b/tests/unit/test_atomic_io.py @@ -0,0 +1,67 @@ +"""Tests for atomic, lock-protected file writes.""" +import multiprocessing +import time +from pathlib import Path + +from scripts.core import atomic_io + + +def test_atomic_write_text_creates_file_with_content(tmp_path): + target = tmp_path / "note.txt" + atomic_io.atomic_write_text(target, "hello world") + assert target.read_text(encoding="utf-8") == "hello world" + + +def test_atomic_write_text_leaves_no_tmp_file_behind(tmp_path): + target = tmp_path / "note.txt" + atomic_io.atomic_write_text(target, "hello world") + leftovers = list(tmp_path.glob(".note.txt.*.tmp")) + assert leftovers == [] + + +def test_atomic_write_text_replaces_existing_content(tmp_path): + target = tmp_path / "note.txt" + atomic_io.atomic_write_text(target, "first") + atomic_io.atomic_write_text(target, "second") + assert target.read_text(encoding="utf-8") == "second" + + +def test_atomic_write_text_creates_missing_parent_directories(tmp_path): + target = tmp_path / "nested" / "dir" / "note.txt" + atomic_io.atomic_write_text(target, "hello") + assert target.read_text(encoding="utf-8") == "hello" + + +def _append_under_lock(payload): + directory_str, log_path_str, worker_id = payload + directory = Path(directory_str) + log_path = Path(log_path_str) + with atomic_io.adr_directory_lock(directory): + with open(log_path, "a", encoding="utf-8") as f: + f.write(f"start {worker_id}\n") + time.sleep(0.05) + with open(log_path, "a", encoding="utf-8") as f: + f.write(f"end {worker_id}\n") + return worker_id + + +def test_adr_directory_lock_serializes_concurrent_workers(tmp_path): + directory = tmp_path / "docs" / "decisions" + directory.mkdir(parents=True) + log_path = tmp_path / "order.log" + log_path.write_text("", encoding="utf-8") + + with multiprocessing.Pool(processes=4) as pool: + pool.map( + _append_under_lock, + [(str(directory), str(log_path), i) for i in range(4)], + ) + + lines = log_path.read_text(encoding="utf-8").splitlines() + # Every "start N" must be immediately followed by "end N" -- the lock + # forbids another worker's "start" from interleaving in between. + assert len(lines) == 8 + for i in range(0, len(lines), 2): + worker = lines[i].split()[1] + assert lines[i] == f"start {worker}" + assert lines[i + 1] == f"end {worker}" From 49ede49800211f95bdda0c5007a5bfb7b7052947 Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 09:50:49 +0900 Subject: [PATCH 11/58] fix: make ADR creation race-free under concurrent invocation Wraps ID allocation + existence check + write in atomic_io.adr_directory_lock and replaces the direct write_text with atomic_io.atomic_write_text. The dry-run path stays outside the lock entirely (it must not create adr_dir or a lock file, per existing dry-run tests) and does an unprotected preview instead -- a race there is harmless since nothing is persisted. Reproduced the race first: 20 concurrent `create` calls produced only 13 unique ADR IDs before this fix. --- skills/adr-toolkit/scripts/commands/create.py | 107 +++++++++++------- tests/integration/test_create_concurrency.py | 40 +++++++ 2 files changed, 107 insertions(+), 40 deletions(-) create mode 100644 tests/integration/test_create_concurrency.py diff --git a/skills/adr-toolkit/scripts/commands/create.py b/skills/adr-toolkit/scripts/commands/create.py index 1da3143..b4df40e 100644 --- a/skills/adr-toolkit/scripts/commands/create.py +++ b/skills/adr-toolkit/scripts/commands/create.py @@ -4,6 +4,7 @@ from datetime import date from pathlib import Path +from scripts.core import atomic_io from scripts.core import frontmatter as fm from scripts.core import identifiers from scripts.core.config import ConfigError, resolve_locale @@ -15,6 +16,21 @@ REQUIRED_DRAFT_FIELDS = {"title", "status", "body"} +def _build_frontmatter(draft: dict, next_num: int, locale: str) -> dict: + return { + "id": f"ADR-{next_num:04d}", + "title": draft["title"], + "status": draft["status"], + "date": draft.get("date") or date.today().isoformat(), + "locale": locale, + "decision_makers": draft.get("decision_makers", []), + "related": draft.get("related", []), + "affected_paths": draft.get("affected_paths", []), + "tags": draft.get("tags", []), + "retrospective": draft.get("retrospective", False), + } + + def _prompt(input_fn, question: str) -> str: print(question, file=sys.stderr) print("> ", end="", file=sys.stderr) @@ -133,50 +149,61 @@ def run(args) -> dict: "errors": [{"code": "INVALID_SLUG", "detail": str(exc)}], } - next_num = identifiers.next_id(adr_dir) - filename = identifiers.format_filename(next_num, slug) - target = adr_dir / filename + if dry_run: + # A dry run must not create anything on disk -- not even adr_dir or + # a lock file -- so ID allocation here is an unprotected preview. A + # concurrent real `create` could take this exact ID before one + # actually runs; that's fine since nothing here is persisted. + next_num = identifiers.next_id(adr_dir) + filename = identifiers.format_filename(next_num, slug) + target = adr_dir / filename + + if target.exists(): + return { + "ok": False, + "operation": "create", + "errors": [{"code": "FILE_ALREADY_EXISTS", "path": str(target)}], + } - if target.exists(): - return { - "ok": False, - "operation": "create", - "errors": [{"code": "FILE_ALREADY_EXISTS", "path": str(target)}], - } + frontmatter_data = _build_frontmatter(draft, next_num, locale) + schema_errors = validate_frontmatter(frontmatter_data) + if schema_errors: + return { + "ok": False, + "operation": "create", + "errors": [{"code": "SCHEMA_ERROR", "detail": e} for e in schema_errors], + } - frontmatter_data = { - "id": f"ADR-{next_num:04d}", - "title": draft["title"], - "status": draft["status"], - "date": draft.get("date") or date.today().isoformat(), - "locale": locale, - "decision_makers": draft.get("decision_makers", []), - "related": draft.get("related", []), - "affected_paths": draft.get("affected_paths", []), - "tags": draft.get("tags", []), - "retrospective": draft.get("retrospective", False), - } + return {"ok": True, "operation": "create", "dry_run": True, "would_create": str(target), "id": frontmatter_data["id"]} - schema_errors = validate_frontmatter(frontmatter_data) - if schema_errors: - return { - "ok": False, - "operation": "create", - "errors": [{"code": "SCHEMA_ERROR", "detail": e} for e in schema_errors], - } + with atomic_io.adr_directory_lock(adr_dir): + next_num = identifiers.next_id(adr_dir) + filename = identifiers.format_filename(next_num, slug) + target = adr_dir / filename - content = fm.serialize(frontmatter_data, draft["body"].strip() + "\n") + if target.exists(): + return { + "ok": False, + "operation": "create", + "errors": [{"code": "FILE_ALREADY_EXISTS", "path": str(target)}], + } - if dry_run: - return {"ok": True, "operation": "create", "dry_run": True, "would_create": str(target), "id": frontmatter_data["id"]} + frontmatter_data = _build_frontmatter(draft, next_num, locale) + schema_errors = validate_frontmatter(frontmatter_data) + if schema_errors: + return { + "ok": False, + "operation": "create", + "errors": [{"code": "SCHEMA_ERROR", "detail": e} for e in schema_errors], + } - adr_dir.mkdir(parents=True, exist_ok=True) - target.write_text(content, encoding="utf-8") + content = fm.serialize(frontmatter_data, draft["body"].strip() + "\n") + atomic_io.atomic_write_text(target, content) - return { - "ok": True, - "operation": "create", - "dry_run": False, - "created": str(target), - "id": frontmatter_data["id"], - } + return { + "ok": True, + "operation": "create", + "dry_run": False, + "created": str(target), + "id": frontmatter_data["id"], + } diff --git a/tests/integration/test_create_concurrency.py b/tests/integration/test_create_concurrency.py new file mode 100644 index 0000000..4460c46 --- /dev/null +++ b/tests/integration/test_create_concurrency.py @@ -0,0 +1,40 @@ +"""Proves ADR creation is race-free under concurrent invocation (the +Top-3 #1 finding in docs/adr-toolkit-audit-report.md).""" +import json +import multiprocessing +from pathlib import Path +from types import SimpleNamespace + +from scripts.commands import create + + +def _create_one(payload): + adr_dir_str, index = payload + adr_dir = Path(adr_dir_str) + draft_path = adr_dir.parent / f"draft-{index}.json" + draft_path.write_text( + json.dumps({"title": f"Decision {index}", "status": "proposed", "body": "Body text."}), + encoding="utf-8", + ) + args = SimpleNamespace( + input=str(draft_path), + interactive=False, + dir=adr_dir_str, + root=".", + locale=None, + slug=f"decision-{index}", + dry_run=False, + ) + return create.run(args) + + +def test_concurrent_create_never_duplicates_ids(tmp_path): + adr_dir = tmp_path / "docs" / "decisions" + adr_dir.mkdir(parents=True) + + with multiprocessing.Pool(processes=8) as pool: + results = pool.map(_create_one, [(str(adr_dir), i) for i in range(20)]) + + assert all(result["ok"] for result in results), results + ids = [result["id"] for result in results] + assert len(ids) == len(set(ids)), f"duplicate IDs allocated: {ids}" From 68bbd9883a20af840349c3dee72eaedeb3dc53d4 Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 09:52:53 +0900 Subject: [PATCH 12/58] fix: make exception creation race-free under concurrent invocation Same shape as the create.py fix, plus one refinement: schema validation runs once against a preview ID before ever touching disk (not even to create exceptions_dir or a lock file), since validity never depends on which sequential number gets assigned. This satisfies the existing "SCHEMA_ERROR must not create exceptions_dir" test as well as dry-run's "must not create anything" test. The lock then wraps only the final ID allocation + atomic write. Reproduced the race first: 20 concurrent `exception` calls produced only 18 unique EXC IDs before this fix. --- .../adr-toolkit/scripts/commands/exception.py | 64 +++++++++++-------- .../integration/test_exception_concurrency.py | 42 ++++++++++++ 2 files changed, 80 insertions(+), 26 deletions(-) create mode 100644 tests/integration/test_exception_concurrency.py diff --git a/skills/adr-toolkit/scripts/commands/exception.py b/skills/adr-toolkit/scripts/commands/exception.py index a9f2e0c..3f933fb 100644 --- a/skills/adr-toolkit/scripts/commands/exception.py +++ b/skills/adr-toolkit/scripts/commands/exception.py @@ -3,12 +3,26 @@ from datetime import date from pathlib import Path +from scripts.core import atomic_io from scripts.core.exceptions import validate_exception from scripts.core.repository_paths import resolve_from_root REQUIRED_DRAFT_FIELDS = {"adr_id", "rule_id", "owner", "reason", "scope", "expiry"} +def _build_exception(draft: dict, exception_id: str) -> dict: + return { + "id": exception_id, + "adr_id": draft["adr_id"], + "rule_id": draft["rule_id"], + "owner": draft["owner"], + "reason": draft["reason"], + "scope": draft["scope"], + "expiry": draft["expiry"], + "created": draft.get("created") or date.today().isoformat(), + } + + def _next_id(exceptions_dir: Path) -> int: existing = [] if exceptions_dir.is_dir(): @@ -53,21 +67,16 @@ def run(args) -> dict: adr_dir = resolve_from_root(root, args.dir) exceptions_dir = adr_dir / "exceptions" - next_num = _next_id(exceptions_dir) - exception_id = f"EXC-{next_num:04d}" - - data = { - "id": exception_id, - "adr_id": draft["adr_id"], - "rule_id": draft["rule_id"], - "owner": draft["owner"], - "reason": draft["reason"], - "scope": draft["scope"], - "expiry": draft["expiry"], - "created": draft.get("created") or date.today().isoformat(), - } - schema_errors = validate_exception(data) + # Validate against a preview ID first. This must not touch disk -- not + # even to create exceptions_dir or a lock file -- so a SCHEMA_ERROR and + # a dry run both leave the filesystem untouched. Validity never depends + # on which sequential number gets assigned (the ID always matches + # EXC-NNNN by construction), so this result stays correct even if a + # concurrent writer changes the real next number before the write below. + preview_num = _next_id(exceptions_dir) + preview_id = f"EXC-{preview_num:04d}" + schema_errors = validate_exception(_build_exception(draft, preview_id)) if schema_errors: return { "ok": False, @@ -75,24 +84,27 @@ def run(args) -> dict: "errors": [{"code": "SCHEMA_ERROR", "detail": e} for e in schema_errors], } - target = exceptions_dir / f"{next_num:04d}.json" - if dry_run: + target = exceptions_dir / f"{preview_num:04d}.json" return { "ok": True, "operation": "exception", "dry_run": True, "would_create": str(target), - "id": exception_id, + "id": preview_id, } - exceptions_dir.mkdir(parents=True, exist_ok=True) - target.write_text(json.dumps(data, indent=2, ensure_ascii=False) + "\n", encoding="utf-8") + with atomic_io.adr_directory_lock(exceptions_dir): + next_num = _next_id(exceptions_dir) + exception_id = f"EXC-{next_num:04d}" + data = _build_exception(draft, exception_id) + target = exceptions_dir / f"{next_num:04d}.json" + atomic_io.atomic_write_text(target, json.dumps(data, indent=2, ensure_ascii=False) + "\n") - return { - "ok": True, - "operation": "exception", - "dry_run": False, - "created": str(target), - "id": exception_id, - } + return { + "ok": True, + "operation": "exception", + "dry_run": False, + "created": str(target), + "id": exception_id, + } diff --git a/tests/integration/test_exception_concurrency.py b/tests/integration/test_exception_concurrency.py new file mode 100644 index 0000000..e49d397 --- /dev/null +++ b/tests/integration/test_exception_concurrency.py @@ -0,0 +1,42 @@ +"""Proves exception creation is race-free under concurrent invocation, the +same class of bug as create.py (docs/adr-toolkit-audit-report.md §2.3 3.3).""" +import json +import multiprocessing +from pathlib import Path + +from types import SimpleNamespace + +from scripts.commands import exception as exception_cmd + + +def _create_exception(payload): + root_str, index = payload + root = Path(root_str) + draft_path = root / f"draft-{index}.json" + draft_path.write_text( + json.dumps({ + "adr_id": "ADR-0001", + "rule_id": "forbidden_import", + "owner": f"owner-{index}", + "reason": "test exception", + "scope": ["src/**"], + "expiry": "2099-01-01", + }), + encoding="utf-8", + ) + args = SimpleNamespace( + input=str(draft_path), + dir="docs/decisions", + root=root_str, + dry_run=False, + ) + return exception_cmd.run(args) + + +def test_concurrent_exception_creation_never_duplicates_ids(tmp_path): + with multiprocessing.Pool(processes=8) as pool: + results = pool.map(_create_exception, [(str(tmp_path), i) for i in range(20)]) + + assert all(result["ok"] for result in results), results + ids = [result["id"] for result in results] + assert len(ids) == len(set(ids)), f"duplicate exception IDs allocated: {ids}" From cec7215d7f06c0296de05dd0d0b20ffbfacba1a8 Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 09:54:49 +0900 Subject: [PATCH 13/58] fix: make SUPERSEDE writes atomic and lock-protected Wraps the whole two-file update in atomic_io.adr_directory_lock and replaces both write_text calls (and the rollback write) with atomic_write_text, so a crash never leaves either file torn. Two existing tests (test_supersede_rolls_back_old_file_when_new_file_write_fails, test_supersede_double_write_failure_reports_inconsistent_state_not_silent) monkeypatched Path.write_text directly to simulate a write failure; that seam no longer exists once writes go through atomic_io, so both were retargeted to monkeypatch supersede.atomic_io.atomic_write_text instead, preserving their original intent and assertions unchanged. --- .../adr-toolkit/scripts/commands/supersede.py | 232 +++++++++--------- tests/unit/test_supersede.py | 16 +- tests/unit/test_supersede_atomicity.py | 80 ++++++ 3 files changed, 208 insertions(+), 120 deletions(-) create mode 100644 tests/unit/test_supersede_atomicity.py diff --git a/skills/adr-toolkit/scripts/commands/supersede.py b/skills/adr-toolkit/scripts/commands/supersede.py index 471bea5..4e8fbdb 100644 --- a/skills/adr-toolkit/scripts/commands/supersede.py +++ b/skills/adr-toolkit/scripts/commands/supersede.py @@ -1,6 +1,7 @@ """Mark one ADR as superseded by another and update both link directions.""" from pathlib import Path +from scripts.core import atomic_io from scripts.core import frontmatter as fm from scripts.core import identifiers from scripts.core.lifecycle import InvalidTransitionError, validate_transition @@ -8,127 +9,134 @@ def run(args) -> dict: adr_dir = Path(args.dir) - old_file = identifiers.find_by_number(adr_dir, args.adr_number) - new_file = identifiers.find_by_number(adr_dir, args.by) - if old_file is None: - return { - "ok": False, - "operation": "supersede", - "errors": [{"code": "ADR_NOT_FOUND", "id": args.adr_number}], - } - if new_file is None: - return { - "ok": False, - "operation": "supersede", - "errors": [{"code": "ADR_NOT_FOUND", "id": args.by}], - } - if args.adr_number == args.by or old_file.resolve() == new_file.resolve(): - return { - "ok": False, - "operation": "supersede", - "errors": [{ - "code": "SELF_SUPERSEDE", - "detail": "An ADR cannot supersede itself.", - "id": args.adr_number, - }], - } + with atomic_io.adr_directory_lock(adr_dir): + old_file = identifiers.find_by_number(adr_dir, args.adr_number) + new_file = identifiers.find_by_number(adr_dir, args.by) - old_text = old_file.read_text(encoding="utf-8") - new_text = new_file.read_text(encoding="utf-8") - try: - old_data, old_body = fm.parse(old_text) - except fm.FrontmatterError as exc: - return { - "ok": False, - "operation": "supersede", - "errors": [{"code": "BAD_FRONTMATTER", "file": old_file.name, "detail": str(exc)}], - } - try: - new_data, new_body = fm.parse(new_text) - except fm.FrontmatterError as exc: - return { - "ok": False, - "operation": "supersede", - "errors": [{"code": "BAD_FRONTMATTER", "file": new_file.name, "detail": str(exc)}], - } + if old_file is None: + return { + "ok": False, + "operation": "supersede", + "errors": [{"code": "ADR_NOT_FOUND", "id": args.adr_number}], + } + if new_file is None: + return { + "ok": False, + "operation": "supersede", + "errors": [{"code": "ADR_NOT_FOUND", "id": args.by}], + } + if args.adr_number == args.by or old_file.resolve() == new_file.resolve(): + return { + "ok": False, + "operation": "supersede", + "errors": [{ + "code": "SELF_SUPERSEDE", + "detail": "An ADR cannot supersede itself.", + "id": args.adr_number, + }], + } - missing_ids = [ - number for number, data in ((args.adr_number, old_data), (args.by, new_data)) - if not data.get("id") - ] - if missing_ids: - return { - "ok": False, - "operation": "supersede", - "errors": [{ - "code": "BAD_FRONTMATTER", - "detail": "ADR file is missing a required 'id' field", - "ids": missing_ids, - }], - } + old_text = old_file.read_text(encoding="utf-8") + new_text = new_file.read_text(encoding="utf-8") + try: + old_data, old_body = fm.parse(old_text) + except fm.FrontmatterError as exc: + return { + "ok": False, + "operation": "supersede", + "errors": [{"code": "BAD_FRONTMATTER", "file": old_file.name, "detail": str(exc)}], + } + try: + new_data, new_body = fm.parse(new_text) + except fm.FrontmatterError as exc: + return { + "ok": False, + "operation": "supersede", + "errors": [{"code": "BAD_FRONTMATTER", "file": new_file.name, "detail": str(exc)}], + } - try: - validate_transition(old_data.get("status"), "superseded") - except InvalidTransitionError as exc: - return { - "ok": False, - "operation": "supersede", - "errors": [{"code": "INVALID_TRANSITION", "detail": str(exc)}], - } + missing_ids = [ + number for number, data in ((args.adr_number, old_data), (args.by, new_data)) + if not data.get("id") + ] + if missing_ids: + return { + "ok": False, + "operation": "supersede", + "errors": [{ + "code": "BAD_FRONTMATTER", + "detail": "ADR file is missing a required 'id' field", + "ids": missing_ids, + }], + } - if new_data.get("status") != "accepted": - return { - "ok": False, - "operation": "supersede", - "errors": [{ - "code": "INVALID_SUPERSEDING_STATUS", - "detail": ( - f"Superseding ADR {new_data['id']} must have status 'accepted', " - f"found {new_data.get('status')!r}." - ), - "id": args.by, - }], - } + try: + validate_transition(old_data.get("status"), "superseded") + except InvalidTransitionError as exc: + return { + "ok": False, + "operation": "supersede", + "errors": [{"code": "INVALID_TRANSITION", "detail": str(exc)}], + } - if getattr(args, "dry_run", False): - return { - "ok": True, - "operation": "supersede", - "dry_run": True, - "would_update": [str(old_file), str(new_file)], - } + if new_data.get("status") != "accepted": + return { + "ok": False, + "operation": "supersede", + "errors": [{ + "code": "INVALID_SUPERSEDING_STATUS", + "detail": ( + f"Superseding ADR {new_data['id']} must have status 'accepted', " + f"found {new_data.get('status')!r}." + ), + "id": args.by, + }], + } + + if getattr(args, "dry_run", False): + return { + "ok": True, + "operation": "supersede", + "dry_run": True, + "would_update": [str(old_file), str(new_file)], + } - old_data["status"] = "superseded" - old_data["superseded_by"] = new_data["id"] + old_data["status"] = "superseded" + old_data["superseded_by"] = new_data["id"] - supersedes = [] - for adr_id in new_data.get("supersedes", []) + [old_data["id"]]: - if adr_id not in supersedes: - supersedes.append(adr_id) - new_data["supersedes"] = supersedes + supersedes = [] + for adr_id in new_data.get("supersedes", []) + [old_data["id"]]: + if adr_id not in supersedes: + supersedes.append(adr_id) + new_data["supersedes"] = supersedes - old_output = fm.serialize(old_data, old_body, body_is_parsed=True) - new_output = fm.serialize(new_data, new_body, body_is_parsed=True) + old_output = fm.serialize(old_data, old_body, body_is_parsed=True) + new_output = fm.serialize(new_data, new_body, body_is_parsed=True) - old_file.write_text(old_output, encoding="utf-8") - try: - new_file.write_text(new_output, encoding="utf-8") - except Exception as write_exc: - # Preserve the original failure while making the completed first write recoverable. + atomic_io.atomic_write_text(old_file, old_output) try: - old_file.write_text(old_text, encoding="utf-8") - except Exception as rollback_exc: - raise RuntimeError( - f"Failed to write {new_file} ({write_exc!r}); rollback of {old_file} also " - f"failed ({rollback_exc!r}); {old_file} may be left in an inconsistent state" - ) from write_exc - raise + atomic_io.atomic_write_text(new_file, new_output) + except Exception as write_exc: + # atomic_write_text already guarantees old_file is never torn by + # a crash; this additionally restores its *content* to the + # pre-supersede value so the two files stay a matched pair when + # the second write fails outright (as opposed to the process + # being killed, which atomic_write_text protects against on its + # own without needing this rollback). + try: + atomic_io.atomic_write_text(old_file, old_text) + except Exception as rollback_exc: + raise RuntimeError( + f"Failed to write {new_file} ({write_exc!r}); rollback of {old_file} also " + f"failed ({rollback_exc!r}); {old_file} may be left in an inconsistent state" + ) from write_exc + raise - return { - "ok": True, - "operation": "supersede", - "dry_run": False, - "old": old_data["id"], - "new": new_data["id"], - } + return { + "ok": True, + "operation": "supersede", + "dry_run": False, + "old": old_data["id"], + "new": new_data["id"], + } diff --git a/tests/unit/test_supersede.py b/tests/unit/test_supersede.py index df2cc6d..79881cf 100644 --- a/tests/unit/test_supersede.py +++ b/tests/unit/test_supersede.py @@ -143,14 +143,14 @@ def test_supersede_rolls_back_old_file_when_new_file_write_fails(tmp_path, monke new_path.write_text(NEW_ADR, encoding="utf-8") old_before = old_path.read_text(encoding="utf-8") new_before = new_path.read_text(encoding="utf-8") - original_write_text = type(old_path).write_text + original_atomic_write_text = supersede.atomic_io.atomic_write_text - def fail_new_file_write(path, text, *args, **kwargs): + def fail_new_file_write(path, content, **kwargs): if path == new_path: raise OSError("simulated new ADR write failure") - return original_write_text(path, text, *args, **kwargs) + return original_atomic_write_text(path, content, **kwargs) - monkeypatch.setattr(type(old_path), "write_text", fail_new_file_write) + monkeypatch.setattr(supersede.atomic_io, "atomic_write_text", fail_new_file_write) with pytest.raises(OSError, match="simulated new ADR write failure"): supersede.run(_args(tmp_path)) @@ -202,18 +202,18 @@ def test_supersede_double_write_failure_reports_inconsistent_state_not_silent(tm new_path.write_text(NEW_ADR, encoding="utf-8") call_count = {"old_writes": 0} - original_write_text = type(old_path).write_text + original_atomic_write_text = supersede.atomic_io.atomic_write_text - def fail_new_then_rollback(path, text, *args, **kwargs): + def fail_new_then_rollback(path, content, **kwargs): if path == new_path: raise OSError("simulated new ADR write failure") if path == old_path: call_count["old_writes"] += 1 if call_count["old_writes"] == 2: raise OSError("simulated rollback failure") - return original_write_text(path, text, *args, **kwargs) + return original_atomic_write_text(path, content, **kwargs) - monkeypatch.setattr(type(old_path), "write_text", fail_new_then_rollback) + monkeypatch.setattr(supersede.atomic_io, "atomic_write_text", fail_new_then_rollback) with pytest.raises(RuntimeError, match="rollback of .* also failed"): supersede.run(_args(tmp_path)) diff --git a/tests/unit/test_supersede_atomicity.py b/tests/unit/test_supersede_atomicity.py new file mode 100644 index 0000000..66cf651 --- /dev/null +++ b/tests/unit/test_supersede_atomicity.py @@ -0,0 +1,80 @@ +"""Proves SUPERSEDE's two-file update never leaves a torn file, and that +its existing rollback-on-failure behavior still works once writes go +through atomic_io (docs/adr-toolkit-audit-report.md, Top-3 #1).""" +from types import SimpleNamespace + +import pytest + +from scripts.commands import supersede + + +_OLD_ADR = ( + "---\n" + "id: ADR-0001\n" + "title: Old decision\n" + "status: accepted\n" + "date: 2026-01-01\n" + "decision_makers: []\n" + "related: []\n" + "affected_paths: []\n" + "tags: []\n" + "retrospective: false\n" + "---\n\n" + "Body of the old decision.\n" +) + +_NEW_ADR = ( + "---\n" + "id: ADR-0002\n" + "title: New decision\n" + "status: accepted\n" + "date: 2026-01-02\n" + "decision_makers: []\n" + "related: []\n" + "affected_paths: []\n" + "tags: []\n" + "retrospective: false\n" + "---\n\n" + "Body of the new decision.\n" +) + + +def _write_fixture_adrs(adr_dir): + adr_dir.mkdir(parents=True) + old_file = adr_dir / "0001-old-decision.md" + new_file = adr_dir / "0002-new-decision.md" + old_file.write_text(_OLD_ADR, encoding="utf-8") + new_file.write_text(_NEW_ADR, encoding="utf-8") + return old_file, new_file + + +def test_old_file_is_rolled_back_when_second_write_fails(tmp_path, monkeypatch): + adr_dir = tmp_path / "docs" / "decisions" + old_file, new_file = _write_fixture_adrs(adr_dir) + + real_write = supersede.atomic_io.atomic_write_text + calls = {"count": 0} + + def flaky_write(path, content, **kwargs): + calls["count"] += 1 + if calls["count"] == 2: + raise OSError("simulated disk failure") + return real_write(path, content, **kwargs) + + monkeypatch.setattr(supersede.atomic_io, "atomic_write_text", flaky_write) + + with pytest.raises(OSError): + supersede.run(SimpleNamespace(adr_number=1, by=2, dir=str(adr_dir), dry_run=False)) + + assert old_file.read_text(encoding="utf-8") == _OLD_ADR + assert calls["count"] == 3 # old write, failed new write, rollback write + + +def test_no_tmp_files_survive_a_successful_supersede(tmp_path): + adr_dir = tmp_path / "docs" / "decisions" + old_file, new_file = _write_fixture_adrs(adr_dir) + + result = supersede.run(SimpleNamespace(adr_number=1, by=2, dir=str(adr_dir), dry_run=False)) + + assert result["ok"] is True + assert list(adr_dir.glob("*.tmp")) == [] From c0ff907ee62ac63c550d07eb8924800d904fd867 Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 09:55:44 +0900 Subject: [PATCH 14/58] fix: add ReDoS timeout guard to CHECK's author-supplied regex patterns conflict._guarded_search wraps regex.search with a 0.25s SIGALRM-based timeout on POSIX (Windows has no SIGALRM and runs unguarded there -- tracked as a known gap, not a regression). RegexTimeout subclasses re.error so commands/check.py's existing `except re.error` handling downgrades a timeout to a BAD_CONSTRAINTS warning without any change needed there. Verified against a classic catastrophic-backtracking pattern ((a+)+$) that would otherwise hang; the guard interrupts it in well under 1s. --- skills/adr-toolkit/scripts/rules/conflict.py | 38 +++++++++++++++++++- tests/unit/test_conflict_redos.py | 32 +++++++++++++++++ 2 files changed, 69 insertions(+), 1 deletion(-) create mode 100644 tests/unit/test_conflict_redos.py diff --git a/skills/adr-toolkit/scripts/rules/conflict.py b/skills/adr-toolkit/scripts/rules/conflict.py index d6640f2..99a4978 100644 --- a/skills/adr-toolkit/scripts/rules/conflict.py +++ b/skills/adr-toolkit/scripts/rules/conflict.py @@ -7,6 +7,8 @@ spec, not configurable. """ import re +import signal +import sys from scripts.core import globs @@ -15,6 +17,40 @@ FORBIDDEN_PATH_KINDS = {"forbidden_path"} EXISTENCE_KINDS = {"file_must_exist", "test_must_exist"} +_REGEX_TIMEOUT_SECONDS = 0.25 + + +class RegexTimeout(re.error): + """An author-supplied `constraints:` pattern exceeded its evaluation + budget (e.g. catastrophic backtracking). Subclasses re.error so + commands/check.py's existing `except re.error` handling downgrades it + to a BAD_CONSTRAINTS warning without any change there.""" + + def __init__(self, pattern: str): + super().__init__( + f"pattern exceeded {_REGEX_TIMEOUT_SECONDS}s evaluation budget: {pattern!r}" + ) + + +def _guarded_search(regex, line: str): + """Run regex.search with a wall-clock budget on POSIX. Must run on the + main thread -- signal handlers are process-wide and only installable + there. signal.SIGALRM does not exist on Windows, so this runs unguarded + there; the CI matrix's linux/macOS legs still cover the timeout path.""" + if sys.platform == "win32": + return regex.search(line) + + def _raise_timeout(signum, frame): + raise RegexTimeout(regex.pattern) + + previous_handler = signal.signal(signal.SIGALRM, _raise_timeout) + signal.setitimer(signal.ITIMER_REAL, _REGEX_TIMEOUT_SECONDS) + try: + return regex.search(line) + finally: + signal.setitimer(signal.ITIMER_REAL, 0) + signal.signal(signal.SIGALRM, previous_handler) + def evaluate_rule(rule: dict, diff_files: list, existing_paths: set): kind = rule.get("kind") @@ -61,7 +97,7 @@ def _content_pattern(rule: dict, diff_files: list): for file_entry in _files_matching(diff_files, rule.get("paths", [])): for line in file_entry.get("added_lines", []): for regex in regexes: - if regex.search(line): + if _guarded_search(regex, line): return _violation(rule, file=file_entry["path"], evidence={"line": line, "pattern": regex.pattern}) return None diff --git a/tests/unit/test_conflict_redos.py b/tests/unit/test_conflict_redos.py new file mode 100644 index 0000000..f869053 --- /dev/null +++ b/tests/unit/test_conflict_redos.py @@ -0,0 +1,32 @@ +"""Tests for the ReDoS timeout guard on author-supplied CHECK patterns +(docs/adr-toolkit-audit-report.md, Top-3 #2).""" +import re +import sys +import time + +import pytest + +from scripts.rules import conflict + + +@pytest.mark.skipif(sys.platform == "win32", reason="SIGALRM guard is POSIX-only") +def test_guarded_search_raises_regex_timeout_on_pathological_pattern(): + regex = re.compile(r"(a+)+$") + pathological_input = "a" * 35 + "!" + + started = time.monotonic() + with pytest.raises(conflict.RegexTimeout): + conflict._guarded_search(regex, pathological_input) + elapsed = time.monotonic() - started + + assert elapsed < 1.0 + + +def test_regex_timeout_is_a_re_error_so_check_py_catches_it(): + assert issubclass(conflict.RegexTimeout, re.error) + + +def test_guarded_search_still_matches_normal_patterns(): + regex = re.compile(r"forbidden_call\(") + assert conflict._guarded_search(regex, "x = forbidden_call(1)") is not None + assert conflict._guarded_search(regex, "x = 1") is None From 7afdcd5672d939115a00853b946eab19d5660074 Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 09:57:14 +0900 Subject: [PATCH 15/58] fix: escape ADR titles in generated README to prevent link injection MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit core/rendering.py gains safe_md_link_text(), which escapes \, [, ], ( and ) and collapses embedded newlines. commands/index.py's four [id — title](filename) link sites now route the title through it, so an ADR title like "foo](http://evil.example)[bar" can no longer split the generated docs/decisions/README.md into two links, one pointing off-repo. This is a fix to the README *generator code*, unrelated to hand-authored README prose being worked on elsewhere. --- skills/adr-toolkit/scripts/commands/index.py | 9 +++-- skills/adr-toolkit/scripts/core/rendering.py | 13 +++++++ .../test_index_markdown_injection.py | 38 +++++++++++++++++++ tests/unit/test_rendering_safe_links.py | 18 +++++++++ 4 files changed, 74 insertions(+), 4 deletions(-) create mode 100644 tests/integration/test_index_markdown_injection.py create mode 100644 tests/unit/test_rendering_safe_links.py diff --git a/skills/adr-toolkit/scripts/commands/index.py b/skills/adr-toolkit/scripts/commands/index.py index b9148df..56112b3 100644 --- a/skills/adr-toolkit/scripts/commands/index.py +++ b/skills/adr-toolkit/scripts/commands/index.py @@ -6,6 +6,7 @@ from scripts.core.config import ConfigError, resolve_locale from scripts.core.locale import load_locale from scripts.core.relationships import render_mermaid, resolve +from scripts.core.rendering import safe_md_link_text from scripts.core.repository_paths import resolve_from_root # Last-resort English headers, used when even the English locale file is @@ -94,7 +95,7 @@ def _render(entries: list, strings: dict) -> str: label = strings.get(f"status.{status}", status.capitalize()) lines.append(f"### {label}") for entry in sorted(by_status[status], key=lambda e: e["filename"]): - lines.append(f"- [{entry['id']} — {entry['title']}]({entry['filename']})") + lines.append(f"- [{entry['id']} — {safe_md_link_text(entry['title'])}]({entry['filename']})") lines.append("") lines.append(f"## {_s(strings, 'by_tag')}") @@ -106,7 +107,7 @@ def _render(entries: list, strings: dict) -> str: for tag in sorted(by_tag): lines.append(f"### {tag}") for entry in sorted(by_tag[tag], key=lambda e: e["filename"]): - lines.append(f"- [{entry['id']} — {entry['title']}]({entry['filename']})") + lines.append(f"- [{entry['id']} — {safe_md_link_text(entry['title'])}]({entry['filename']})") lines.append("") lines.append(f"## {_s(strings, 'by_affected_path')}") @@ -118,13 +119,13 @@ def _render(entries: list, strings: dict) -> str: for path in sorted(by_path): lines.append(f"### `{path}`") for entry in sorted(by_path[path], key=lambda e: e["filename"]): - lines.append(f"- [{entry['id']} — {entry['title']}]({entry['filename']})") + lines.append(f"- [{entry['id']} — {safe_md_link_text(entry['title'])}]({entry['filename']})") lines.append("") lines.append(f"## {_s(strings, 'chronological')}") lines.append("") for entry in sorted(entries, key=lambda e: e["date"], reverse=True): - lines.append(f"- {entry['date']} — [{entry['id']} — {entry['title']}]({entry['filename']})") + lines.append(f"- {entry['date']} — [{entry['id']} — {safe_md_link_text(entry['title'])}]({entry['filename']})") by_id = {entry["id"]: entry for entry in entries} edges = resolve(entries) diff --git a/skills/adr-toolkit/scripts/core/rendering.py b/skills/adr-toolkit/scripts/core/rendering.py index 7673dfa..ecbd538 100644 --- a/skills/adr-toolkit/scripts/core/rendering.py +++ b/skills/adr-toolkit/scripts/core/rendering.py @@ -1,6 +1,19 @@ """Render deterministic ADR Markdown from localized structural strings.""" +import re + from scripts.core.locale import load_locale +_MD_LINK_UNSAFE_RE = re.compile(r"[\\\[\]()]") + + +def safe_md_link_text(text: str) -> str: + """Escape characters that would let ADR-authored text break out of a + Markdown `[text](target)` span -- e.g. close the link early and open a + second one pointing somewhere else.""" + collapsed = " ".join(str(text).split()) + return _MD_LINK_UNSAFE_RE.sub(lambda match: "\\" + match.group(0), collapsed) + + PROMPT_KEYS = ( "prompt.title", "prompt.problem", "prompt.options", "prompt.decision", "prompt.rationale", "prompt.good", "prompt.bad", "prompt.confirmation", diff --git a/tests/integration/test_index_markdown_injection.py b/tests/integration/test_index_markdown_injection.py new file mode 100644 index 0000000..f182bc8 --- /dev/null +++ b/tests/integration/test_index_markdown_injection.py @@ -0,0 +1,38 @@ +"""Proves INDEX's generated README cannot be split into a second, +attacker-controlled link via an ADR title (docs/adr-toolkit-audit-report.md, +Top-3 #2).""" +from pathlib import Path +from types import SimpleNamespace + +from scripts.commands import index + + +_MALICIOUS_ADR = ( + "---\n" + "id: ADR-0001\n" + "title: foo](http://evil.example)[bar\n" + "status: accepted\n" + "date: 2026-01-01\n" + "decision_makers: []\n" + "related: []\n" + "affected_paths: []\n" + "tags: []\n" + "retrospective: false\n" + "---\n\n" + "Body.\n" +) + + +def test_index_readme_cannot_be_split_into_a_second_link(tmp_path): + adr_dir = tmp_path / "docs" / "decisions" + adr_dir.mkdir(parents=True) + (adr_dir / "0001-test.md").write_text(_MALICIOUS_ADR, encoding="utf-8") + + result = index.run(SimpleNamespace(dir=str(adr_dir), root=str(tmp_path), locale=None)) + assert result["ok"] is True + + readme = (adr_dir / "README.md").read_text(encoding="utf-8") + # An unescaped "](...)[ " sequence is what a Markdown renderer reads as + # "close this link, open a second one" -- it must not appear. + assert "](http://evil.example)[" not in readme + assert "foo\\]\\(http://evil.example\\)\\[bar" in readme diff --git a/tests/unit/test_rendering_safe_links.py b/tests/unit/test_rendering_safe_links.py new file mode 100644 index 0000000..c587251 --- /dev/null +++ b/tests/unit/test_rendering_safe_links.py @@ -0,0 +1,18 @@ +"""Tests for Markdown link-text escaping (docs/adr-toolkit-audit-report.md, +Top-3 #2).""" +from scripts.core.rendering import safe_md_link_text + + +def test_escapes_characters_that_break_out_of_link_text(): + result = safe_md_link_text("foo](http://evil.example)[bar") + assert result == "foo\\]\\(http://evil.example\\)\\[bar" + + +def test_collapses_embedded_newlines_to_spaces(): + result = safe_md_link_text("line one\nline two") + assert result == "line one line two" + + +def test_leaves_ordinary_titles_unchanged(): + result = safe_md_link_text("Adopt PostgreSQL for primary storage") + assert result == "Adopt PostgreSQL for primary storage" From 11c8f4bf6aa5e0c5f6e8c5dc6782905e806d43b4 Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 09:58:38 +0900 Subject: [PATCH 16/58] feat: add structured stderr logging with correlation IDs New core/telemetry.get_logger(operation) returns a LoggerAdapter that writes JSON Lines to stderr (level, operation, correlation_id, message, and exception_type on exceptions). adr.py's global exception handler logs via logger.exception() and includes the same correlation_id in the stdout JSON error response, so a failure reported by an agent/CI can be matched back to its stderr log line. Default level is WARNING (quiet on success); ADR_TOOLKIT_LOG_LEVEL overrides it. stdout's pure-JSON contract is unchanged except for the additive correlation_id field. --- skills/adr-toolkit/scripts/adr.py | 9 +++- skills/adr-toolkit/scripts/core/telemetry.py | 50 ++++++++++++++++++++ tests/unit/test_main_error_logging.py | 27 +++++++++++ tests/unit/test_telemetry.py | 37 +++++++++++++++ 4 files changed, 122 insertions(+), 1 deletion(-) create mode 100644 skills/adr-toolkit/scripts/core/telemetry.py create mode 100644 tests/unit/test_main_error_logging.py create mode 100644 tests/unit/test_telemetry.py diff --git a/skills/adr-toolkit/scripts/adr.py b/skills/adr-toolkit/scripts/adr.py index 9d139dc..68249c4 100755 --- a/skills/adr-toolkit/scripts/adr.py +++ b/skills/adr-toolkit/scripts/adr.py @@ -26,6 +26,7 @@ ) from scripts.core.lifecycle import STATUSES from scripts.core.locale import SUPPORTED_LOCALES +from scripts.core.telemetry import get_logger def _add_json_flag(parser: argparse.ArgumentParser) -> None: @@ -186,10 +187,16 @@ def main(argv=None) -> int: try: result = HANDLERS[args.operation](args) except Exception as exc: # noqa: BLE001 - last-resort safety net for the JSON-only-stdout contract + logger = get_logger(args.operation) + logger.exception("operation failed") result = { "ok": False, "operation": args.operation, - "errors": [{"code": "INTERNAL_ERROR", "detail": str(exc)}], + "errors": [{ + "code": "INTERNAL_ERROR", + "detail": str(exc), + "correlation_id": logger.extra["correlation_id"], + }], } print(json.dumps(result, indent=2, ensure_ascii=False)) return 0 if result.get("ok") else 1 diff --git a/skills/adr-toolkit/scripts/core/telemetry.py b/skills/adr-toolkit/scripts/core/telemetry.py new file mode 100644 index 0000000..da59bf5 --- /dev/null +++ b/skills/adr-toolkit/scripts/core/telemetry.py @@ -0,0 +1,50 @@ +"""Structured stderr logging for ADR Toolkit commands. + +stdout is a machine-readable JSON contract consumed by agents and CI -- +never write anything else there. Diagnostic and crash logs go to stderr as +JSON Lines instead, tagged with a correlation ID so a failure reported in +the stdout JSON can be matched back to its log line. +""" +import json +import logging +import os +import sys +import time +import uuid +from typing import Optional + +_LOG_LEVEL_ENV = "ADR_TOOLKIT_LOG_LEVEL" + + +class _JsonLogFormatter(logging.Formatter): + def format(self, record: logging.LogRecord) -> str: + payload = { + "ts": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime(record.created)), + "level": record.levelname.lower(), + "operation": getattr(record, "operation", None), + "correlation_id": getattr(record, "correlation_id", None), + "message": record.getMessage(), + } + if record.exc_info: + payload["exception_type"] = record.exc_info[0].__name__ + return json.dumps(payload, ensure_ascii=False) + + +def get_logger(operation: str, *, correlation_id: Optional[str] = None) -> logging.LoggerAdapter: + """Return a per-call logger bound to `operation`. The handler is + rebuilt on every call (rather than cached on the module-level logger) + so it always binds to the *current* sys.stderr -- this is what makes + the logger correctly testable under pytest's capsys, and costs nothing + in real usage since each CLI invocation is a fresh process that calls + this exactly once.""" + logger = logging.getLogger("adr_toolkit") + logger.handlers.clear() + handler = logging.StreamHandler(sys.stderr) + handler.setFormatter(_JsonLogFormatter()) + logger.addHandler(handler) + logger.propagate = False + logger.setLevel(os.environ.get(_LOG_LEVEL_ENV, "WARNING").upper()) + return logging.LoggerAdapter( + logger, + {"operation": operation, "correlation_id": correlation_id or uuid.uuid4().hex[:12]}, + ) diff --git a/tests/unit/test_main_error_logging.py b/tests/unit/test_main_error_logging.py new file mode 100644 index 0000000..5e17bdc --- /dev/null +++ b/tests/unit/test_main_error_logging.py @@ -0,0 +1,27 @@ +"""Verifies adr.py's global exception handler surfaces a correlation ID in +both the stdout JSON error and the stderr structured log (Top-3 #3).""" +import json + +from scripts import adr + + +def test_internal_error_includes_correlation_id_and_stderr_log(monkeypatch, capsys): + def _boom(args): + raise RuntimeError("boom") + + monkeypatch.setitem(adr.HANDLERS, "preflight", _boom) + + exit_code = adr.main(["preflight"]) + + assert exit_code == 1 + captured = capsys.readouterr() + result = json.loads(captured.out) + assert result["ok"] is False + assert result["errors"][0]["code"] == "INTERNAL_ERROR" + correlation_id = result["errors"][0]["correlation_id"] + assert correlation_id + + log_payload = json.loads(captured.err.strip().splitlines()[-1]) + assert log_payload["level"] == "error" + assert log_payload["operation"] == "preflight" + assert log_payload["correlation_id"] == correlation_id diff --git a/tests/unit/test_telemetry.py b/tests/unit/test_telemetry.py new file mode 100644 index 0000000..84725d7 --- /dev/null +++ b/tests/unit/test_telemetry.py @@ -0,0 +1,37 @@ +"""Tests for structured stderr logging (docs/adr-toolkit-audit-report.md, +Top-3 #3).""" +import json + +from scripts.core import telemetry + + +def test_get_logger_writes_json_lines_to_stderr(capsys): + logger = telemetry.get_logger("check", correlation_id="abc123") + logger.warning("something looked odd") + + captured = capsys.readouterr() + assert captured.out == "" + payload = json.loads(captured.err.strip()) + assert payload["level"] == "warning" + assert payload["operation"] == "check" + assert payload["correlation_id"] == "abc123" + assert payload["message"] == "something looked odd" + + +def test_get_logger_generates_a_correlation_id_when_none_given(): + logger = telemetry.get_logger("index") + assert logger.extra["correlation_id"] + assert logger.extra["operation"] == "index" + + +def test_get_logger_records_exception_type_on_exception_logging(capsys): + logger = telemetry.get_logger("create") + try: + raise ValueError("boom") + except ValueError: + logger.exception("operation failed") + + captured = capsys.readouterr() + payload = json.loads(captured.err.strip()) + assert payload["level"] == "error" + assert payload["exception_type"] == "ValueError" From f0cbc8615cd1ceff9c010b265b5971f1e7f88e2a Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 10:00:10 +0900 Subject: [PATCH 17/58] docs: close out Critical hardening backlog items improvements.md's Critical section is cleared (resolved work now lives in changelog.md's Unreleased section + git history, per this file's own convention). handoff.md records the 7 commits that made up this pass, the 2 real regressions TDD caught and how they were fixed, and the remaining unscheduled High/Medium backlog for a future session to pick up only with explicit owner direction. --- changelog.md | 12 +++++ handoff.md | 119 +++++++++++++++++++++++++----------------------- improvements.md | 23 ---------- 3 files changed, 73 insertions(+), 81 deletions(-) diff --git a/changelog.md b/changelog.md index 6a90ad0..e558fbb 100644 --- a/changelog.md +++ b/changelog.md @@ -4,6 +4,18 @@ Lightweight human-readable summary of meaningful repository changes. ## Unreleased +- ADR and exception creation, and SUPERSEDE's two-file update, are now + atomic and race-free under concurrent invocation (file locking + write to + a temp file followed by an atomic rename). +- CHECK's author-supplied `constraints:` regex patterns now have a 0.25s + evaluation timeout on Linux/macOS, closing a ReDoS risk (Windows CI is + unaffected but not yet guarded — tracked in `improvements.md`). +- INDEX's generated decision-log README now escapes ADR titles, closing a + Markdown link-injection risk. +- Uncaught errors now log a structured, correlation-ID-tagged JSON line to + stderr (`ADR_TOOLKIT_LOG_LEVEL` controls the threshold) instead of + disappearing silently; the same correlation ID appears in the stdout + JSON error response. - Added a `harness-parity` CI job that installs the real Codex CLI and Gemini CLI and drives their own plugin/extension commands (marketplace add, install, list) against this repo, then runs `preflight`/`init`/ diff --git a/handoff.md b/handoff.md index dd8a591..9fd0b0f 100644 --- a/handoff.md +++ b/handoff.md @@ -2,38 +2,38 @@ ## Current task (2026-09-01) -Executing `docs/superpowers/plans/2026-09-01-critical-hardening.md` -- -implementation plan for the 4 Critical-risk findings from -`docs/adr-toolkit-audit-report.md` (also tracked in `improvements.md`'s -`## Open` -> `### Critical` section): - -1. Atomic file writes + ID allocation lock (`core/atomic_io.py`, wired into - `create.py`, `exception.py`, `supersede.py`). -2. ReDoS timeout guard for author-supplied regex (`rules/conflict.py`). -3. Markdown link-injection escape for ADR titles in the generated - `docs/decisions/README.md` (`core/rendering.py`, `commands/index.py`). -4. Structured stderr logging with correlation IDs (`core/telemetry.py`, - wired into `adr.py`'s global exception handler). - -The plan is split into 8 tasks (Task 1-7 = one deliverable each, Task 8 = -close out the backlog docs). Each task ends with its own commit, so -**`git log --oneline` against the plan's task list is the source of truth -for what's already done** if this session is interrupted -- check which of -these commit messages exist before resuming (and cross-check the plan -file's own `- [ ]`/`- [x]` checkboxes, which are updated as steps land): - -- `feat: add atomic write and directory lock primitives` (Task 1) -- `fix: make ADR creation race-free under concurrent invocation` (Task 2) -- `fix: make exception creation race-free under concurrent invocation` (Task 3) -- `fix: make SUPERSEDE writes atomic and lock-protected` (Task 4) -- `fix: add ReDoS timeout guard to CHECK's author-supplied regex patterns` (Task 5) -- `fix: escape ADR titles in generated README to prevent link injection` (Task 6) -- `feat: add structured stderr logging with correlation IDs` (Task 7) -- `docs: close out Critical hardening backlog items` (Task 8) - -This session chose **inline execution** (`superpowers:executing-plans`), -not subagent-driven -- a fresh session resuming should do the same unless -the owner says otherwise. +**The Critical hardening pass is done.** All 4 Critical-risk findings from +`docs/adr-toolkit-audit-report.md` are implemented, tested, and committed +on `feature/analyzing-adr-toolkit`, following +`docs/superpowers/plans/2026-09-01-critical-hardening.md` (gitignored by +convention, still on disk in this worktree) task-by-task with TDD: + +1. `fc46830` feat: add atomic write and directory lock primitives +2. `49ede49` fix: make ADR creation race-free under concurrent invocation +3. `68bbd98` fix: make exception creation race-free under concurrent invocation +4. `cec7215` fix: make SUPERSEDE writes atomic and lock-protected +5. `c0ff907` fix: add ReDoS timeout guard to CHECK's author-supplied regex patterns +6. `7afdcd5` fix: escape ADR titles in generated README to prevent link injection +7. `11c8f4b` feat: add structured stderr logging with correlation IDs + +`improvements.md`'s Critical section is now empty (items removed per this +file's own convention: resolved work lives in `changelog.md`'s Unreleased +section + git history, not duplicated into `## Done`). Full suite: 415 +passed (up from 395 at session start). + +Two real regressions were caught by TDD mid-session and fixed before +committing, worth knowing if touching this code again: +- `create.py`/`exception.py`: naively wrapping everything in + `adr_directory_lock` made *dry runs* (and, for `exception.py`, + *schema-validation failures*) create the directory + lock file as a side + effect, breaking existing tests that assert nothing is created. Fixed by + keeping preview/validation paths outside the lock and only locking the + actual allocate-and-write step. +- `supersede.py`: two existing tests monkeypatched `Path.write_text` + directly to simulate a write failure; that seam disappeared once writes + route through `atomic_io.atomic_write_text`, so both were retargeted to + patch `supersede.atomic_io.atomic_write_text` instead (same intent, same + assertions). ## Scope for this worktree @@ -51,42 +51,45 @@ Excluded here, being handled elsewhere -- do not touch: deliberately deferred to that other worktree for exactly this reason -- see the "(다른 워크트리 확인)" flags in `improvements.md`. - README prose (root README.md, `adapters/*/README.md` content) -- another - worktree. Task 6's fix to `commands/index.py` is a security fix in the - *generator code* for `docs/decisions/README.md`, not README prose, and - correctly stays in scope here -- don't confuse the two if asked to skip - "README work". + worktree. The Task 6 fix above touched `commands/index.py` -- that's a + security fix in the *generator code* for `docs/decisions/README.md`, not + README prose, and correctly stayed in scope here. ## Next step -If resuming: open `docs/superpowers/plans/2026-09-01-critical-hardening.md`, -find the first unchecked step, confirm against `git log` that its task's -commit doesn't already exist, and continue from there with -`superpowers:executing-plans`. - -After Task 8 lands, the remaining backlog (`improvements.md`'s High/Medium -sections) is unscheduled -- ask the owner before starting any of it. The -Critical-only scope for this pass, and the domain/worktree exclusions -above, were the owner's explicit calls in conversation, not something -derivable from the audit report alone. +Nothing is currently in flight. `improvements.md`'s remaining backlog +(High: repository path escape guard, test coverage measurement, mypy/ +TypedDict, diagnostic/timing mode, chaos SIGKILL test, adapter SDK +extraction, plus the two other-worktree-flagged items; Medium: JSON Schema +single-source-of-truth, common error base class, output contract schema +freeze, parsing cache, bulk-ADR benchmark, TTY-aware CLI output) is +unscheduled -- **ask the owner before starting any of it**. The +Critical-only scope for the pass just completed, and the domain/worktree +exclusions above, were the owner's explicit calls in conversation, not +something derivable from the audit report alone. ## Verification -Full suite: `python3 -m pytest tests/unit tests/integration -v`. -Baseline before this session's changes: 395 passed. Expect it to grow by -roughly 15-18 tests across Tasks 1-7 (see the plan file's per-task test -files for the exact count). +Full suite: `python3 -m pytest tests/unit tests/integration -v` -> 415 +passed as of commit `11c8f4b`. ## Open risks -- The ReDoS guard (Task 5) is POSIX-only (`signal.SIGALRM`); Windows CI is +- The ReDoS guard is POSIX-only (`signal.SIGALRM`); Windows CI is unaffected but unguarded against catastrophic-backtracking patterns -- - a known, documented gap in the audit report, not a regression introduced - by this work. -- `supersede.py`'s two-file update (Task 4) guarantees each individual - file is never torn by a mid-write crash, but does not guarantee the - *pair* stays consistent if the process is killed between the two atomic - writes -- true two-phase commit across files was explicitly scoped out - (see the plan's Task 4 code comments). + a known, documented gap, not a regression introduced by this work. +- `supersede.py`'s two-file update guarantees each individual file is + never torn by a mid-write crash, but does not guarantee the *pair* + stays consistent if the process is killed between the two atomic writes + -- true two-phase commit across files was explicitly scoped out (see + the plan's Task 4 code comments). The backlog's "카오스(SIGKILL) + 복원력 테스트" High item follows up on this. +- Every successful `create`/`exception`/`supersede` call now leaves a + `.adr-toolkit.lock` (0-byte, dotfile) inside `docs/decisions/` and + `docs/decisions/exceptions/` permanently -- this is intentional (it's + the cross-process mutex), doesn't match `*.md`/`*.json` globs so nothing + else picks it up, but is a new, permanent artifact worth knowing about + if someone notices it in a repo diff. - (carried over from the audit, still true) CHECK deliberately cannot prove prose, business rationale, or organizational claims; those remain human-review evidence. diff --git a/improvements.md b/improvements.md index fe25aef..43b5be1 100644 --- a/improvements.md +++ b/improvements.md @@ -13,29 +13,6 @@ README prose, which are being handled in other worktrees/branches. The two items below flagged "(다른 워크트리 확인)" touch files those efforts may also touch. -### Critical - -**구현 계획**: `docs/superpowers/plans/2026-09-01-critical-hardening.md` -(Task 1-8, TDD 단계별). 세션이 끊겨도 이 계획 파일의 체크박스 + -`git log --oneline`에 남는 태스크별 커밋 메시지로 어디까지 됐는지 바로 -알 수 있다 — 상세 재개 절차는 `handoff.md` 참고. - -- [ ] **원자적 파일 쓰기 + ID 채번 락** — `identifiers.py`, `create.py`, - `exception.py`, `supersede.py`; 신규 `core/atomic_io.py`. 락 없는 - glob→max+1 채번과 비원자적 `write_text`를 `adr_directory_lock` + - `atomic_write_text`로 교체. (감사 보고서 §1 Top-3 #1, §2.3 3.3) -- [ ] **ADR 정규식 ReDoS 가드** — `rules/conflict.py`. `constraints:`의 - `pattern` 필드 실행에 하드 타임아웃(POSIX: SIGALRM, Windows: 정적 - 중첩 정량자 린트). (감사 보고서 §1 Top-3 #2, §2.2 2.3) -- [ ] **생성된 README 링크 이스케이프** — `core/rendering.py`, - `commands/index.py`. `render_mermaid`의 `_mermaid_label`처럼 title을 - 이스케이프해 `docs/decisions/README.md` 생성 시 링크 하이재킹 방지. - 자동 생성 코드 수정이며 사람이 쓰는 README 문서 작업과는 무관. - (감사 보고서 §1 Top-3 #2, §2.2 2.3) -- [ ] **구조화 로깅** — 신규 `core/telemetry.py`, `adr.py`. stderr에 - JSON 로그(operation, correlation_id, exception type) 추가, stdout의 - 순수 JSON 결과 계약은 불변 유지. (감사 보고서 §1 Top-3 #3, §2.7 7.1) - ### High - [ ] **저장소 경로 탈출 방지** — `core/repository_paths.py`. From 114bcaa1031a4e728efc47dc60102d3d247eb7b0 Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 10:08:02 +0900 Subject: [PATCH 18/58] fix(ci): resolve Windows CP1252 UnicodeEncodeError in verify_examples script --- changelog.md | 1 + scripts/verify_examples.py | 9 +++++++-- tests/integration/test_examples.py | 6 ++++++ 3 files changed, 14 insertions(+), 2 deletions(-) diff --git a/changelog.md b/changelog.md index aa6729d..b8cf6ae 100644 --- a/changelog.md +++ b/changelog.md @@ -4,6 +4,7 @@ Lightweight human-readable summary of meaningful repository changes. ## Unreleased +- Fixed Windows CP1252 console encoding failure (`UnicodeEncodeError: 'charmap' codec can't encode character '\u2713'`) in `scripts/verify_examples.py` and `tests/integration/test_examples.py` by replacing non-ASCII symbols with ASCII tags (`[ok]`, `[error]`), reconfiguring stdout/stderr UTF-8 streams, and setting `PYTHONIOENCODING=utf-8` in subprocess calls. - Added untracked manifest discovery (`discover_untracked_manifests`) in `scripts/sync_version.py` to automatically prevent untracked plugin/extension manifests from being added in PRs without version/description tracking. - Added `.pre-commit-config.yaml` for local contributor pre-commit checks and updated `CONTRIBUTING.md` with manifest governance guidelines. - Enhanced Antigravity CLI (`agy`) plugin manifest (`adapters/antigravity/plugin.json`) with `version` tracking integrated into `scripts/sync_version.py`, expanded unit test assertions in `test_antigravity_adapter.py` (including symlink layout simulation) and `test_readme.py`, and updated `README.md` documentation. diff --git a/scripts/verify_examples.py b/scripts/verify_examples.py index 267a8b9..acc33e1 100644 --- a/scripts/verify_examples.py +++ b/scripts/verify_examples.py @@ -252,13 +252,18 @@ def main(argv=None) -> int: parser.add_argument("--update", action="store_true", help="Auto-update examples if needed") args = parser.parse_args(argv) + if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="backslashreplace") + if hasattr(sys.stderr, "reconfigure"): + sys.stderr.reconfigure(encoding="utf-8", errors="backslashreplace") + print("Verifying examples execution against adr.py...") try: verify_all_flows() - print("✓ All example workflows executed successfully and verified clean.") + print("[ok] All example workflows executed successfully and verified clean.") return 0 except AssertionError as err: - print(f"❌ Verification failed: {err}", file=sys.stderr) + print(f"[error] Verification failed: {err}", file=sys.stderr) return 1 diff --git a/tests/integration/test_examples.py b/tests/integration/test_examples.py index cebecf4..0822642 100644 --- a/tests/integration/test_examples.py +++ b/tests/integration/test_examples.py @@ -1,5 +1,6 @@ """Integration test verifying that examples/*.md workflows remain executable and up-to-date with adr.py logic. """ +import os import subprocess import sys from pathlib import Path @@ -10,10 +11,15 @@ def test_examples_execution_and_schema_parity(): """Verify that all example workflows execute cleanly without error.""" + env = dict(os.environ, PYTHONIOENCODING="utf-8") res = subprocess.run( [sys.executable, str(VERIFY_SCRIPT), "--check"], cwd=REPO_ROOT, capture_output=True, text=True, + encoding="utf-8", + errors="replace", + env=env, ) assert res.returncode == 0, f"Example verification script failed:\nstdout: {res.stdout}\nstderr: {res.stderr}" + From f92c8f59d6f81dc6dc1a9acecd18d14f1d13ddd0 Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 10:13:29 +0900 Subject: [PATCH 19/58] fix: reject a --dir/--root that escapes the repository root resolve_from_root now raises PathEscapesRootError when a relative path (e.g. --dir ../../etc/cron.d) would resolve outside the given root. An absolute path is left unchanged, as before -- it's the caller's own explicit choice, with no relative containment to check. --- .../scripts/core/repository_paths.py | 14 +++++++- tests/unit/test_repository_paths.py | 33 +++++++++++++++++++ 2 files changed, 46 insertions(+), 1 deletion(-) create mode 100644 tests/unit/test_repository_paths.py diff --git a/skills/adr-toolkit/scripts/core/repository_paths.py b/skills/adr-toolkit/scripts/core/repository_paths.py index 8e887f0..43f9e0c 100644 --- a/skills/adr-toolkit/scripts/core/repository_paths.py +++ b/skills/adr-toolkit/scripts/core/repository_paths.py @@ -2,8 +2,20 @@ from pathlib import Path +class PathEscapesRootError(ValueError): + """A relative path resolved outside the root it was given against.""" + + def resolve_from_root(root, path) -> Path: candidate = Path(path) if candidate.is_absolute(): + # An explicit absolute path is the caller's own responsibility -- + # there's no relative containment to check. Only a *relative* + # `path` can accidentally (or maliciously) walk out of `root`. return candidate - return Path(root) / candidate + + joined = Path(root) / candidate + root_resolved = Path(root).resolve() + if not joined.resolve().is_relative_to(root_resolved): + raise PathEscapesRootError(f"{str(path)!r} escapes root {str(root)!r}") + return joined diff --git a/tests/unit/test_repository_paths.py b/tests/unit/test_repository_paths.py new file mode 100644 index 0000000..c0f2997 --- /dev/null +++ b/tests/unit/test_repository_paths.py @@ -0,0 +1,33 @@ +"""Tests for resolve_from_root's boundary enforcement +(docs/adr-toolkit-audit-report.md §2.2 2.3).""" +from pathlib import Path + +import pytest + +from scripts.core.repository_paths import PathEscapesRootError, resolve_from_root + + +def test_relative_path_under_root_resolves_normally(tmp_path): + result = resolve_from_root(tmp_path, "docs/decisions") + assert result == tmp_path / "docs/decisions" + + +def test_absolute_path_is_returned_unchanged_even_outside_root(tmp_path): + outside = tmp_path.parent / "somewhere-else" + result = resolve_from_root(tmp_path, outside) + assert result == outside + + +def test_relative_path_escaping_root_is_rejected(tmp_path): + with pytest.raises(PathEscapesRootError): + resolve_from_root(tmp_path, "../../etc/cron.d") + + +def test_relative_path_escaping_root_via_nested_traversal_is_rejected(tmp_path): + with pytest.raises(PathEscapesRootError): + resolve_from_root(tmp_path, "docs/../../outside") + + +def test_dot_path_resolves_to_root_itself(tmp_path): + result = resolve_from_root(tmp_path, ".") + assert result.resolve() == tmp_path.resolve() From 52bd76140c7936ff4940ea5f1b5d4b822a745f31 Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 10:14:02 +0900 Subject: [PATCH 20/58] ci: measure and gate branch coverage at 85% Measured baseline before adding the gate: 93.32% branch+statement coverage across skills/adr-toolkit/scripts. --cov-fail-under=85 leaves real headroom rather than being a guessed threshold. release.yml is untouched. --- .github/workflows/test.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 5fb11b4..1f4f2d4 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -29,9 +29,9 @@ jobs: with: python-version: ${{ matrix.python-version }} - name: Install dependencies - run: pip install pytest + run: pip install pytest pytest-cov - name: Run tests - run: python -m pytest tests/unit tests/integration -v + run: python -m pytest tests/unit tests/integration -v --cov=skills/adr-toolkit/scripts --cov-branch --cov-report=term-missing --cov-fail-under=85 version-drift: runs-on: ubuntu-latest From 305c8360e76d86f2017a711d0f1e60e06cfa0e8e Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 10:15:34 +0900 Subject: [PATCH 21/58] feat: add typed result contracts and a mypy --strict CI gate New core/contracts.py defines TypedDicts (CommandError, BaseResult, ErrorResult, CreateResult) for command JSON output shapes. A new type-check CI job runs `mypy --strict` over the fully-typed core modules (atomic_io, telemetry, contracts) -- fixed 3 real errors mypy found in atomic_io.py/telemetry.py to get there (missing return type on the contextmanager generator, an unnarrowed Optional in exc_info[0], and an unparameterized generic LoggerAdapter). Command *arguments* stay untyped (argparse.Namespace resists TypedDict without a larger refactor) -- extending strict typing into the 16 command modules is deliberately out of scope for this pass. --- .github/workflows/test.yml | 17 ++++++++++ skills/adr-toolkit/scripts/core/atomic_io.py | 3 +- skills/adr-toolkit/scripts/core/contracts.py | 33 ++++++++++++++++++++ skills/adr-toolkit/scripts/core/telemetry.py | 4 +-- tests/unit/test_contracts.py | 25 +++++++++++++++ 5 files changed, 79 insertions(+), 3 deletions(-) create mode 100644 skills/adr-toolkit/scripts/core/contracts.py create mode 100644 tests/unit/test_contracts.py diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 1f4f2d4..8b45805 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -33,6 +33,23 @@ jobs: - name: Run tests run: python -m pytest tests/unit tests/integration -v --cov=skills/adr-toolkit/scripts --cov-branch --cov-report=term-missing --cov-fail-under=85 + type-check: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 + with: + python-version: "3.12" + - name: Install mypy + run: pip install mypy + - name: Type-check the fully-typed core modules + run: >- + mypy + skills/adr-toolkit/scripts/core/atomic_io.py + skills/adr-toolkit/scripts/core/telemetry.py + skills/adr-toolkit/scripts/core/contracts.py + --strict + version-drift: runs-on: ubuntu-latest steps: diff --git a/skills/adr-toolkit/scripts/core/atomic_io.py b/skills/adr-toolkit/scripts/core/atomic_io.py index 94fd6f0..95324d3 100644 --- a/skills/adr-toolkit/scripts/core/atomic_io.py +++ b/skills/adr-toolkit/scripts/core/atomic_io.py @@ -13,6 +13,7 @@ import tempfile from contextlib import contextmanager from pathlib import Path +from typing import Iterator if sys.platform == "win32": import msvcrt @@ -51,7 +52,7 @@ def atomic_write_text(path: Path, content: str, *, encoding: str = "utf-8") -> N @contextmanager -def adr_directory_lock(directory: Path): +def adr_directory_lock(directory: Path) -> Iterator[None]: """Serialize ID allocation + writes for one ADR/exceptions directory across processes. The lock file lives inside `directory` itself so a fresh clone or a brand-new `docs/decisions/` needs no extra setup.""" diff --git a/skills/adr-toolkit/scripts/core/contracts.py b/skills/adr-toolkit/scripts/core/contracts.py new file mode 100644 index 0000000..ed25219 --- /dev/null +++ b/skills/adr-toolkit/scripts/core/contracts.py @@ -0,0 +1,33 @@ +"""Structural type contracts for ADR Toolkit's JSON command results. + +Every command returns a plain dict matching one of these shapes -- typing +them here lets `mypy --strict` (see the `type-check` CI job) catch a test +or caller that reads a field that was renamed or removed. Command +*arguments* are argparse.Namespace objects (dynamic attribute access), +which TypedDict can't model without a larger refactor; that is tracked +separately and not attempted here. +""" +from typing import List, TypedDict + + +class CommandError(TypedDict, total=False): + code: str + detail: str + correlation_id: str + + +class BaseResult(TypedDict): + ok: bool + operation: str + + +class ErrorResult(BaseResult): + errors: List[CommandError] + + +class CreateResult(BaseResult, total=False): + dry_run: bool + created: str + would_create: str + id: str + errors: List[CommandError] diff --git a/skills/adr-toolkit/scripts/core/telemetry.py b/skills/adr-toolkit/scripts/core/telemetry.py index da59bf5..0f4bd2a 100644 --- a/skills/adr-toolkit/scripts/core/telemetry.py +++ b/skills/adr-toolkit/scripts/core/telemetry.py @@ -25,12 +25,12 @@ def format(self, record: logging.LogRecord) -> str: "correlation_id": getattr(record, "correlation_id", None), "message": record.getMessage(), } - if record.exc_info: + if record.exc_info and record.exc_info[0] is not None: payload["exception_type"] = record.exc_info[0].__name__ return json.dumps(payload, ensure_ascii=False) -def get_logger(operation: str, *, correlation_id: Optional[str] = None) -> logging.LoggerAdapter: +def get_logger(operation: str, *, correlation_id: Optional[str] = None) -> "logging.LoggerAdapter[logging.Logger]": """Return a per-call logger bound to `operation`. The handler is rebuilt on every call (rather than cached on the module-level logger) so it always binds to the *current* sys.stderr -- this is what makes diff --git a/tests/unit/test_contracts.py b/tests/unit/test_contracts.py new file mode 100644 index 0000000..94de492 --- /dev/null +++ b/tests/unit/test_contracts.py @@ -0,0 +1,25 @@ +"""Tests that core/contracts.py's TypedDicts describe real command output +shapes (docs/adr-toolkit-audit-report.md §2.4 4.1).""" +from types import SimpleNamespace + +from scripts.commands import create +from scripts.core import contracts + + +def test_create_dry_run_result_matches_contract_keys(tmp_path): + draft_path = tmp_path / "draft.json" + draft_path.write_text( + '{"title": "Use Kafka", "status": "proposed", "body": "Body."}', + encoding="utf-8", + ) + args = SimpleNamespace( + input=str(draft_path), interactive=False, dir="docs/decisions", + root=str(tmp_path), locale=None, slug=None, dry_run=True, + ) + result = create.run(args) + + contract_keys = set(contracts.CreateResult.__annotations__) + assert set(result.keys()) <= contract_keys, ( + f"create.run() returned keys not in contracts.CreateResult: " + f"{set(result.keys()) - contract_keys}" + ) From 46dd863d5592b4f3307dd11ef5dbc3a61d460430 Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 10:16:29 +0900 Subject: [PATCH 22/58] feat: add --diagnostic flag for per-invocation timing adr.py --diagnostic adds an elapsed_ms field to the JSON result via time.perf_counter(). Must precede the operation name (argparse subparsers can't see a flag registered only on the parent parser). Omitted by default -- stdout's JSON shape is unchanged unless requested. --- skills/adr-toolkit/scripts/adr.py | 9 +++++++++ tests/unit/test_diagnostic_mode.py | 23 +++++++++++++++++++++++ 2 files changed, 32 insertions(+) create mode 100644 tests/unit/test_diagnostic_mode.py diff --git a/skills/adr-toolkit/scripts/adr.py b/skills/adr-toolkit/scripts/adr.py index 68249c4..30edc9b 100755 --- a/skills/adr-toolkit/scripts/adr.py +++ b/skills/adr-toolkit/scripts/adr.py @@ -3,6 +3,7 @@ import argparse import json import sys +import time from pathlib import Path sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) @@ -50,6 +51,11 @@ def _add_diff_mode_arguments(parser: argparse.ArgumentParser) -> None: def build_parser() -> argparse.ArgumentParser: parser = argparse.ArgumentParser(prog="adr.py") + parser.add_argument( + "--diagnostic", action="store_true", + help="Add an elapsed_ms timing field to the JSON result. Must " + "precede the operation name, e.g. `adr.py --diagnostic check`.", + ) sub = parser.add_subparsers(dest="operation", required=True) p_preflight = sub.add_parser("preflight") @@ -184,6 +190,7 @@ def main(argv=None) -> int: parser = build_parser() args = parser.parse_args(argv) + started_at = time.perf_counter() try: result = HANDLERS[args.operation](args) except Exception as exc: # noqa: BLE001 - last-resort safety net for the JSON-only-stdout contract @@ -198,6 +205,8 @@ def main(argv=None) -> int: "correlation_id": logger.extra["correlation_id"], }], } + if getattr(args, "diagnostic", False): + result["_diagnostics"] = {"elapsed_ms": round((time.perf_counter() - started_at) * 1000, 1)} print(json.dumps(result, indent=2, ensure_ascii=False)) return 0 if result.get("ok") else 1 diff --git a/tests/unit/test_diagnostic_mode.py b/tests/unit/test_diagnostic_mode.py new file mode 100644 index 0000000..cb522c0 --- /dev/null +++ b/tests/unit/test_diagnostic_mode.py @@ -0,0 +1,23 @@ +"""Tests for the --diagnostic timing flag (docs/adr-toolkit-audit-report.md +§2.7 7.2).""" +import json + +from scripts import adr + + +def test_diagnostic_flag_adds_elapsed_ms(tmp_path, capsys): + exit_code = adr.main(["--diagnostic", "preflight", "--root", str(tmp_path)]) + + assert exit_code == 0 + result = json.loads(capsys.readouterr().out) + assert "_diagnostics" in result + assert isinstance(result["_diagnostics"]["elapsed_ms"], (int, float)) + assert result["_diagnostics"]["elapsed_ms"] >= 0 + + +def test_diagnostic_flag_omitted_by_default(tmp_path, capsys): + exit_code = adr.main(["preflight", "--root", str(tmp_path)]) + + assert exit_code == 0 + result = json.loads(capsys.readouterr().out) + assert "_diagnostics" not in result From 9708bb2103bba7d77dcbabd72abb886c0e85e726 Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 10:17:05 +0900 Subject: [PATCH 23/58] test: prove atomic_write_text survives a mid-write SIGKILL Forks a child that pauses right before os.replace() (after the temp file is written, before the rename), SIGKILLs it there, and asserts the target file still holds its original content -- an OS-level proof of the guarantee atomic_write_text already provides, not a simulated exception. Skipped on Windows (os.fork is POSIX-only). --- tests/unit/test_atomic_io_chaos.py | 54 ++++++++++++++++++++++++++++++ 1 file changed, 54 insertions(+) create mode 100644 tests/unit/test_atomic_io_chaos.py diff --git a/tests/unit/test_atomic_io_chaos.py b/tests/unit/test_atomic_io_chaos.py new file mode 100644 index 0000000..087cdd0 --- /dev/null +++ b/tests/unit/test_atomic_io_chaos.py @@ -0,0 +1,54 @@ +"""Chaos test: a process killed mid-write must never leave a torn ADR file +(docs/adr-toolkit-audit-report.md §2.8 8.2 -- atomic_io's core guarantee, +proven here at the OS level rather than by simulating a raised exception).""" +import os +import signal +import sys +import time +from pathlib import Path + +import pytest + +from scripts.core import atomic_io + + +def _write_slowly_then_get_killed(path_str: str, ready_flag_str: str) -> None: + path = Path(path_str) + ready_flag = Path(ready_flag_str) + original_replace = os.replace + + def paused_replace(src, dst): + # Signal the parent that the temp file exists and we're about to + # rename it over the real target -- the single most dangerous + # instant for a non-atomic write scheme -- then stall long enough + # that the parent's SIGKILL always arrives first. + ready_flag.write_text("ready", encoding="utf-8") + time.sleep(10) + return original_replace(src, dst) + + os.replace = paused_replace # child-process-only; fork gives us a private copy + atomic_io.atomic_write_text(path, "new content that must never land") + + +@pytest.mark.skipif(sys.platform == "win32", reason="os.fork is POSIX-only") +def test_process_killed_mid_write_never_leaves_a_torn_file(tmp_path): + target = tmp_path / "0001-decision.md" + target.write_text("original valid content\n", encoding="utf-8") + ready_flag = tmp_path / "ready.flag" + + pid = os.fork() + if pid == 0: + try: + _write_slowly_then_get_killed(str(target), str(ready_flag)) + finally: + os._exit(1) # should never actually reach here + + deadline = time.monotonic() + 5 + while not ready_flag.exists() and time.monotonic() < deadline: + time.sleep(0.01) + assert ready_flag.exists(), "child never reached its pre-rename pause" + + os.kill(pid, signal.SIGKILL) + os.waitpid(pid, 0) + + assert target.read_text(encoding="utf-8") == "original valid content\n" From cff3d5e543e00d9ac98acad89f10c29fd00688bd Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 10:18:34 +0900 Subject: [PATCH 24/58] feat: extract a shared adapter-manifest validator New scripts/adapter_sdk.py (repo-root tooling, same category as sync_version.py) validates the two fields every manifest-based adapter shares: name and description, both required non-empty strings. All 4 manifest-based adapter test files (Claude, Codex, Gemini CLI, Antigravity) now assert their real manifest passes it, loaded via importlib.util the same way test_sync_version.py already works around the scripts/ vs skills/adr-toolkit/scripts/ naming collision. adapters/generic/ has no manifest and is unaffected. --- scripts/adapter_sdk.py | 26 +++++++++++++++++++ tests/unit/test_adapter_sdk.py | 36 ++++++++++++++++++++++++++ tests/unit/test_antigravity_adapter.py | 17 +++++++++--- tests/unit/test_claude_adapter.py | 11 ++++++++ tests/unit/test_codex_adapter.py | 17 +++++++++--- tests/unit/test_gemini_cli_adapter.py | 17 +++++++++--- 6 files changed, 112 insertions(+), 12 deletions(-) create mode 100644 scripts/adapter_sdk.py create mode 100644 tests/unit/test_adapter_sdk.py diff --git a/scripts/adapter_sdk.py b/scripts/adapter_sdk.py new file mode 100644 index 0000000..3f5c0c9 --- /dev/null +++ b/scripts/adapter_sdk.py @@ -0,0 +1,26 @@ +"""Shared structural validation for AI-harness adapter manifests. + +Repo tooling -- not part of the distributable skills/adr-toolkit/ package +(same category as scripts/sync_version.py). Each harness (Claude Code, +Codex, Gemini CLI, Antigravity) defines its own manifest shape with extra +harness-specific keys ($schema, version, ...); this only checks the two +fields every one of them shares. The generic fallback adapter +(adapters/generic/) has no manifest at all and is not covered here. +""" +REQUIRED_FIELDS = {"name": str, "description": str} + + +def validate_adapter_manifest(manifest: dict) -> list: + errors = [] + for field, expected_type in REQUIRED_FIELDS.items(): + if field not in manifest: + errors.append(f"missing required field: {field}") + continue + value = manifest[field] + if not isinstance(value, expected_type): + errors.append( + f"field {field!r} must be {expected_type.__name__}, got {type(value).__name__}" + ) + elif not value.strip(): + errors.append(f"field {field!r} must not be empty") + return errors diff --git a/tests/unit/test_adapter_sdk.py b/tests/unit/test_adapter_sdk.py new file mode 100644 index 0000000..ca8d782 --- /dev/null +++ b/tests/unit/test_adapter_sdk.py @@ -0,0 +1,36 @@ +"""Tests for the shared adapter-manifest validator +(docs/adr-toolkit-audit-report.md §2.6 6.3).""" +import importlib.util +from pathlib import Path + +_ADAPTER_SDK_PATH = Path(__file__).resolve().parents[2] / "scripts" / "adapter_sdk.py" +_spec = importlib.util.spec_from_file_location("_repo_root_adapter_sdk", _ADAPTER_SDK_PATH) +adapter_sdk = importlib.util.module_from_spec(_spec) +_spec.loader.exec_module(adapter_sdk) + + +def test_valid_manifest_has_no_errors(): + errors = adapter_sdk.validate_adapter_manifest( + {"name": "adr-toolkit", "description": "Does the thing."} + ) + assert errors == [] + + +def test_missing_name_is_reported(): + errors = adapter_sdk.validate_adapter_manifest({"description": "Does the thing."}) + assert any("name" in e for e in errors) + + +def test_missing_description_is_reported(): + errors = adapter_sdk.validate_adapter_manifest({"name": "adr-toolkit"}) + assert any("description" in e for e in errors) + + +def test_empty_string_fields_are_rejected(): + errors = adapter_sdk.validate_adapter_manifest({"name": "", "description": " "}) + assert len(errors) == 2 + + +def test_non_string_fields_are_rejected(): + errors = adapter_sdk.validate_adapter_manifest({"name": 123, "description": None}) + assert len(errors) == 2 diff --git a/tests/unit/test_antigravity_adapter.py b/tests/unit/test_antigravity_adapter.py index 2e8c18f..010962f 100644 --- a/tests/unit/test_antigravity_adapter.py +++ b/tests/unit/test_antigravity_adapter.py @@ -1,14 +1,18 @@ +import importlib.util import json import re from pathlib import Path -MANIFEST = ( - Path(__file__).resolve().parent.parent.parent - / "adapters" / "antigravity" / "plugin.json" -) +REPO_ROOT = Path(__file__).resolve().parent.parent.parent +MANIFEST = REPO_ROOT / "adapters" / "antigravity" / "plugin.json" NAME_RE = re.compile(r"^[A-Za-z0-9_-]+$") +_ADAPTER_SDK_PATH = REPO_ROOT / "scripts" / "adapter_sdk.py" +_spec = importlib.util.spec_from_file_location("_repo_root_adapter_sdk", _ADAPTER_SDK_PATH) +adapter_sdk = importlib.util.module_from_spec(_spec) +_spec.loader.exec_module(adapter_sdk) + def test_manifest_is_valid_json_with_required_fields(): data = json.loads(MANIFEST.read_text(encoding="utf-8")) @@ -23,3 +27,8 @@ def test_manifest_name_matches_antigravity_naming_rule(): def test_manifest_schema_field_points_at_antigravity_schema(): data = json.loads(MANIFEST.read_text(encoding="utf-8")) assert data["$schema"] == "https://antigravity.google/schemas/v1/plugin.json" + + +def test_manifest_passes_the_shared_adapter_validator(): + data = json.loads(MANIFEST.read_text(encoding="utf-8")) + assert adapter_sdk.validate_adapter_manifest(data) == [] diff --git a/tests/unit/test_claude_adapter.py b/tests/unit/test_claude_adapter.py index 3c68d73..d79fb6b 100644 --- a/tests/unit/test_claude_adapter.py +++ b/tests/unit/test_claude_adapter.py @@ -1,9 +1,15 @@ +import importlib.util import json from pathlib import Path REPO_ROOT = Path(__file__).resolve().parent.parent.parent PLUGIN_DIR = REPO_ROOT / ".claude-plugin" +_ADAPTER_SDK_PATH = REPO_ROOT / "scripts" / "adapter_sdk.py" +_spec = importlib.util.spec_from_file_location("_repo_root_adapter_sdk", _ADAPTER_SDK_PATH) +adapter_sdk = importlib.util.module_from_spec(_spec) +_spec.loader.exec_module(adapter_sdk) + def test_plugin_manifest_has_required_keys_and_no_skills_key(): manifest = json.loads((PLUGIN_DIR / "plugin.json").read_text(encoding="utf-8")) @@ -25,3 +31,8 @@ def test_marketplace_manifest_lists_the_plugin(): marketplace = json.loads((PLUGIN_DIR / "marketplace.json").read_text(encoding="utf-8")) names = [p["name"] for p in marketplace["plugins"]] assert "adr-toolkit" in names + + +def test_plugin_manifest_passes_the_shared_adapter_validator(): + manifest = json.loads((PLUGIN_DIR / "plugin.json").read_text(encoding="utf-8")) + assert adapter_sdk.validate_adapter_manifest(manifest) == [] diff --git a/tests/unit/test_codex_adapter.py b/tests/unit/test_codex_adapter.py index fab6f9b..2ac1b11 100644 --- a/tests/unit/test_codex_adapter.py +++ b/tests/unit/test_codex_adapter.py @@ -1,10 +1,14 @@ +import importlib.util import json from pathlib import Path -MANIFEST = ( - Path(__file__).resolve().parent.parent.parent - / "adapters" / "codex" / ".codex-plugin" / "plugin.json" -) +REPO_ROOT = Path(__file__).resolve().parent.parent.parent +MANIFEST = REPO_ROOT / "adapters" / "codex" / ".codex-plugin" / "plugin.json" + +_ADAPTER_SDK_PATH = REPO_ROOT / "scripts" / "adapter_sdk.py" +_spec = importlib.util.spec_from_file_location("_repo_root_adapter_sdk", _ADAPTER_SDK_PATH) +adapter_sdk = importlib.util.module_from_spec(_spec) +_spec.loader.exec_module(adapter_sdk) def test_manifest_is_valid_json_with_required_fields(): @@ -15,3 +19,8 @@ def test_manifest_is_valid_json_with_required_fields(): def test_manifest_has_no_extra_undocumented_top_level_keys(): data = json.loads(MANIFEST.read_text(encoding="utf-8")) assert set(data.keys()) <= {"$schema", "name", "description"} + + +def test_manifest_passes_the_shared_adapter_validator(): + data = json.loads(MANIFEST.read_text(encoding="utf-8")) + assert adapter_sdk.validate_adapter_manifest(data) == [] diff --git a/tests/unit/test_gemini_cli_adapter.py b/tests/unit/test_gemini_cli_adapter.py index c0f344b..fc65427 100644 --- a/tests/unit/test_gemini_cli_adapter.py +++ b/tests/unit/test_gemini_cli_adapter.py @@ -1,10 +1,14 @@ +import importlib.util import json from pathlib import Path -MANIFEST = ( - Path(__file__).resolve().parent.parent.parent - / "adapters" / "gemini-cli" / "gemini-extension.json" -) +REPO_ROOT = Path(__file__).resolve().parent.parent.parent +MANIFEST = REPO_ROOT / "adapters" / "gemini-cli" / "gemini-extension.json" + +_ADAPTER_SDK_PATH = REPO_ROOT / "scripts" / "adapter_sdk.py" +_spec = importlib.util.spec_from_file_location("_repo_root_adapter_sdk", _ADAPTER_SDK_PATH) +adapter_sdk = importlib.util.module_from_spec(_spec) +_spec.loader.exec_module(adapter_sdk) def test_manifest_is_valid_json_with_required_fields(): @@ -16,3 +20,8 @@ def test_manifest_name_uses_dashes_not_underscores_or_spaces(): data = json.loads(MANIFEST.read_text(encoding="utf-8")) assert " " not in data["name"] assert "_" not in data["name"] + + +def test_manifest_passes_the_shared_adapter_validator(): + data = json.loads(MANIFEST.read_text(encoding="utf-8")) + assert adapter_sdk.validate_adapter_manifest(data) == [] From 9a974b7e772de13231411a899ca97a784041970e Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 10:19:50 +0900 Subject: [PATCH 25/58] docs: close out High-priority hardening backlog items improvements.md's Critical section is gone and High now holds only the 2 items deferred to another worktree. handoff.md records all 13 commits across both the Critical and High-priority passes, the naming-collision and dry-run-side-effect gotchas discovered mid-session, and the remaining unscheduled Medium backlog for a future session to pick up only with explicit owner direction. --- changelog.md | 12 ++++++ handoff.md | 106 ++++++++++++++++++++++++++++++++---------------- improvements.md | 16 -------- 3 files changed, 82 insertions(+), 52 deletions(-) diff --git a/changelog.md b/changelog.md index e558fbb..7645567 100644 --- a/changelog.md +++ b/changelog.md @@ -4,6 +4,18 @@ Lightweight human-readable summary of meaningful repository changes. ## Unreleased +- `--dir`/`--root` now reject a relative path that resolves outside the + given root, closing a path-escape gap. +- CI now measures branch coverage (currently 93%) and fails below 85%. +- Added a `mypy --strict` CI gate over the fully-typed core modules + (`atomic_io`, `telemetry`, the new `contracts` module of TypedDict result + shapes). +- Added `adr.py --diagnostic` (must precede the operation name) to include + an `elapsed_ms` timing field in the JSON result. +- Verified at the OS level (fork + SIGKILL) that a process killed mid-write + never leaves a torn ADR file. +- Extracted a shared adapter-manifest validator (`scripts/adapter_sdk.py`) + used by all 4 manifest-based harness adapters' tests. - ADR and exception creation, and SUPERSEDE's two-file update, are now atomic and race-free under concurrent invocation (file locking + write to a temp file followed by an atomic rename). diff --git a/handoff.md b/handoff.md index 9fd0b0f..c98758a 100644 --- a/handoff.md +++ b/handoff.md @@ -2,11 +2,15 @@ ## Current task (2026-09-01) -**The Critical hardening pass is done.** All 4 Critical-risk findings from +**Both the Critical and High-priority hardening passes are done.** All 4 +Critical-risk findings and 6 of 8 High-priority findings from `docs/adr-toolkit-audit-report.md` are implemented, tested, and committed -on `feature/analyzing-adr-toolkit`, following -`docs/superpowers/plans/2026-09-01-critical-hardening.md` (gitignored by -convention, still on disk in this worktree) task-by-task with TDD: +on `feature/analyzing-adr-toolkit`. Full suite: 433 passed (up from 395 at +session start). Branch stays as-is per owner's explicit choice (not +merged/PR'd yet). + +**Critical pass** (`docs/superpowers/plans/2026-09-01-critical-hardening.md`, +gitignored by convention, still on disk in this worktree): 1. `fc46830` feat: add atomic write and directory lock primitives 2. `49ede49` fix: make ADR creation race-free under concurrent invocation @@ -15,14 +19,27 @@ convention, still on disk in this worktree) task-by-task with TDD: 5. `c0ff907` fix: add ReDoS timeout guard to CHECK's author-supplied regex patterns 6. `7afdcd5` fix: escape ADR titles in generated README to prevent link injection 7. `11c8f4b` feat: add structured stderr logging with correlation IDs +8. `f0cbc86` docs: close out Critical hardening backlog items + +**High-priority pass** (`docs/superpowers/plans/2026-09-01-high-priority-hardening.md`, +same gitignore convention): + +1. `f92c8f5` fix: reject a --dir/--root that escapes the repository root +2. `52bd761` ci: measure and gate branch coverage at 85% +3. `305c836` feat: add typed result contracts and a mypy --strict CI gate +4. `46dd863` feat: add --diagnostic flag for per-invocation timing +5. `9708bb2` test: prove atomic_write_text survives a mid-write SIGKILL +6. `cff3d5e` feat: extract a shared adapter-manifest validator +7. (this commit) docs: close out High-priority hardening backlog items -`improvements.md`'s Critical section is now empty (items removed per this -file's own convention: resolved work lives in `changelog.md`'s Unreleased -section + git history, not duplicated into `## Done`). Full suite: 415 -passed (up from 395 at session start). +`improvements.md` now has an empty `### Critical` section (removed +entirely) and a `### High` section containing only the 2 items explicitly +deferred to another worktree. The `### Medium` section is untouched and +unscheduled. + +Notable things discovered mid-session, worth knowing if touching this code +again: -Two real regressions were caught by TDD mid-session and fixed before -committing, worth knowing if touching this code again: - `create.py`/`exception.py`: naively wrapping everything in `adr_directory_lock` made *dry runs* (and, for `exception.py`, *schema-validation failures*) create the directory + lock file as a side @@ -34,6 +51,16 @@ committing, worth knowing if touching this code again: route through `atomic_io.atomic_write_text`, so both were retargeted to patch `supersede.atomic_io.atomic_write_text` instead (same intent, same assertions). +- Repo-root `scripts/` and `skills/adr-toolkit/scripts/` share the name + `scripts` for Python's import system, and the latter (which has an + `__init__.py`) wins whichever imports first in a pytest session. Any new + file under repo-root `scripts/` must be loaded via + `importlib.util.spec_from_file_location` in its tests, exactly like + `scripts/sync_version.py` already does -- `scripts/adapter_sdk.py` + follows the same pattern. +- Measured, not assumed: branch+statement coverage was 93.32% before + adding the 85% CI gate; `mypy --strict` on `atomic_io.py`/`telemetry.py` + had exactly 3 real errors, fixed as part of adding the `type-check` job. ## Scope for this worktree @@ -46,32 +73,35 @@ Excluded here, being handled elsewhere -- do not touch: in another branch. - Automatic version sync -- owner is working on this in another worktree; as a direct consequence, **do not touch `.github/workflows/release.yml` - for any reason**. Two backlog items (High: supply-chain checksums/ - signing; a note under 8.4 about auto-version-bump direction) were - deliberately deferred to that other worktree for exactly this reason -- - see the "(다른 워크트리 확인)" flags in `improvements.md`. + for any reason**. The remaining "(다른 워크트리 확인)" items in + `improvements.md`'s `### High` section (supply-chain checksums/signing; + the 8.4 auto-version-bump direction note) are deliberately left there + for that other worktree. - README prose (root README.md, `adapters/*/README.md` content) -- another - worktree. The Task 6 fix above touched `commands/index.py` -- that's a - security fix in the *generator code* for `docs/decisions/README.md`, not - README prose, and correctly stayed in scope here. + worktree. Every fix in this session that touched adapter or index code + was a code/generator fix, not README prose, and correctly stayed in + scope here. ## Next step -Nothing is currently in flight. `improvements.md`'s remaining backlog -(High: repository path escape guard, test coverage measurement, mypy/ -TypedDict, diagnostic/timing mode, chaos SIGKILL test, adapter SDK -extraction, plus the two other-worktree-flagged items; Medium: JSON Schema -single-source-of-truth, common error base class, output contract schema -freeze, parsing cache, bulk-ADR benchmark, TTY-aware CLI output) is -unscheduled -- **ask the owner before starting any of it**. The -Critical-only scope for the pass just completed, and the domain/worktree -exclusions above, were the owner's explicit calls in conversation, not -something derivable from the audit report alone. +Nothing is currently in flight. `improvements.md`'s remaining backlog -- +the 2 other-worktree-flagged High items, plus the entire `### Medium` +section (JSON Schema single-source-of-truth, common error base class, +output contract schema freeze, parsing cache, bulk-ADR benchmark, +TTY-aware CLI output) -- is unscheduled. **Ask the owner before starting +any of it.** Every scope decision in this session (Critical-then-High +ordering, domain 1/5 exclusion, the other-worktree exclusions) was the +owner's explicit call in conversation, not something derivable from the +audit report alone. ## Verification -Full suite: `python3 -m pytest tests/unit tests/integration -v` -> 415 -passed as of commit `11c8f4b`. +Full suite: `python3 -m pytest tests/unit tests/integration -v` -> 433 +passed as of commit `cff3d5e` (432 on Windows, where the SIGKILL chaos +test in `test_atomic_io_chaos.py` is skipped). + +CI now also runs a `type-check` job (`mypy --strict` on 3 modules) and +gates the `pytest` job's coverage at 85% -- both new since this session. ## Open risks @@ -81,15 +111,19 @@ passed as of commit `11c8f4b`. - `supersede.py`'s two-file update guarantees each individual file is never torn by a mid-write crash, but does not guarantee the *pair* stays consistent if the process is killed between the two atomic writes - -- true two-phase commit across files was explicitly scoped out (see - the plan's Task 4 code comments). The backlog's "카오스(SIGKILL) - 복원력 테스트" High item follows up on this. + -- true two-phase commit across files was explicitly scoped out. - Every successful `create`/`exception`/`supersede` call now leaves a `.adr-toolkit.lock` (0-byte, dotfile) inside `docs/decisions/` and - `docs/decisions/exceptions/` permanently -- this is intentional (it's - the cross-process mutex), doesn't match `*.md`/`*.json` globs so nothing - else picks it up, but is a new, permanent artifact worth knowing about - if someone notices it in a repo diff. + `docs/decisions/exceptions/` permanently -- intentional (the + cross-process mutex), doesn't match `*.md`/`*.json` globs so nothing + else picks it up, but worth knowing about if someone notices it in a + repo diff. +- `core/contracts.py`'s TypedDicts currently model only a subset of one + command's result shape (`CreateResult`) plus the shared error/base + shapes -- extending coverage to the other 15 commands, and extending + `mypy --strict` beyond the 3 fully-typed core modules into the command + modules themselves (blocked on typing `argparse.Namespace` args), is + future work, not started. - (carried over from the audit, still true) CHECK deliberately cannot prove prose, business rationale, or organizational claims; those remain human-review evidence. diff --git a/improvements.md b/improvements.md index 43b5be1..7c2b78c 100644 --- a/improvements.md +++ b/improvements.md @@ -15,22 +15,6 @@ also touch. ### High -- [ ] **저장소 경로 탈출 방지** — `core/repository_paths.py`. - `--dir`/`--root`가 저장소 루트 밖을 가리키지 못하도록 경계 검사. - (감사 보고서 §2.2 2.3) -- [ ] **테스트 커버리지 측정 도입** — `.github/workflows/test.yml`에 - `pytest-cov --cov-branch --cov-fail-under=85` 추가. `release.yml`은 - 건드리지 않음. (감사 보고서 §2.8 8.1) -- [ ] **mypy + TypedDict 계약 타이핑** — 신규 `core/contracts.py`, CI에 - `mypy --strict` 게이트. (감사 보고서 §2.4 4.1) -- [ ] **진단/타이밍 모드** — `adr.py`에 `--diagnostic` 플래그로 실행 - 시간 계측 노출. (감사 보고서 §2.7 7.2) -- [ ] **카오스(SIGKILL) 복원력 테스트** — 원자적 쓰기 완료 후, 쓰기 - 도중 강제 종료 시 ADR 파일이 항상 유효 상태인지 검증하는 테스트 추가. - (감사 보고서 §2.8 8.2) -- [ ] **어댑터 매니페스트 검증기 추출 (코드만)** — 신규 - `scripts/adapter_sdk.py`의 `validate_adapter_manifest`. 튜토리얼 - 문서화는 README 작업 쪽에서 처리. (감사 보고서 §2.6 6.3) - [ ] *(다른 워크트리 확인)* **공급망 보안(체크섬/서명)** — `.github/workflows/release.yml`에 SHA-256/Sigstore 서명 단계. 자동 버전 동기화 작업이 같은 파일을 건드릴 수 있어 그쪽에 붙이는 것을 권장. From 41998a29992a38e4d8b681b678e864cfe5a9aba4 Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 10:31:11 +0900 Subject: [PATCH 26/58] feat: add a common domain-error base class and fix unhandled path-escape errors New core/errors.py's AdrToolkitError(Exception) is now the base for ConfigError, FrontmatterError, ConstraintsError, InvalidTransitionError, GitPathsError, and PathEscapesRootError, each carrying a stable error_code matching the string already used at its call sites. No call site anywhere catches these by their old ValueError/RuntimeError base, only by name, so this is a safe change (verified via grep before starting). Bundled fix: PathEscapesRootError (added in the prior High-priority pass) was never actually caught anywhere -- a rejected path escape fell through to adr.py's generic INTERNAL_ERROR instead of a specific code, unlike every other domain exception in this codebase. New resolve_from_root_or_error() wraps resolve_from_root() and is now used at all 7 call sites (create, exception, index, validate, check, init, and graph's two sites), returning a proper PATH_ESCAPES_ROOT error. --- skills/adr-toolkit/scripts/commands/check.py | 6 +- skills/adr-toolkit/scripts/commands/create.py | 6 +- .../adr-toolkit/scripts/commands/exception.py | 6 +- skills/adr-toolkit/scripts/commands/graph.py | 29 ++++--- skills/adr-toolkit/scripts/commands/index.py | 6 +- skills/adr-toolkit/scripts/commands/init.py | 6 +- .../adr-toolkit/scripts/commands/validate.py | 6 +- skills/adr-toolkit/scripts/core/config.py | 4 +- .../adr-toolkit/scripts/core/constraints.py | 6 +- skills/adr-toolkit/scripts/core/errors.py | 10 +++ .../adr-toolkit/scripts/core/frontmatter.py | 6 +- skills/adr-toolkit/scripts/core/git_paths.py | 5 +- skills/adr-toolkit/scripts/core/lifecycle.py | 5 +- .../scripts/core/repository_paths.py | 21 ++++- .../test_path_escape_error_wiring.py | 78 +++++++++++++++++++ tests/unit/test_errors.py | 28 +++++++ 16 files changed, 195 insertions(+), 33 deletions(-) create mode 100644 skills/adr-toolkit/scripts/core/errors.py create mode 100644 tests/integration/test_path_escape_error_wiring.py create mode 100644 tests/unit/test_errors.py diff --git a/skills/adr-toolkit/scripts/commands/check.py b/skills/adr-toolkit/scripts/commands/check.py index 3544426..34f9e6f 100644 --- a/skills/adr-toolkit/scripts/commands/check.py +++ b/skills/adr-toolkit/scripts/commands/check.py @@ -11,7 +11,7 @@ from scripts.core.constraints import ConstraintsError, extract_constraints from scripts.core.exceptions import applies_to, is_expired, validate_exception from scripts.core.git_paths import GitPathsError, list_existing_paths -from scripts.core.repository_paths import resolve_from_root +from scripts.core.repository_paths import resolve_from_root_or_error from scripts.core.schema import validate_frontmatter from scripts.rules import conflict @@ -55,7 +55,9 @@ def run(args) -> dict: # `--dir` is relative to `--root`, so `check --root /repo --dir docs/decisions` # means the same directory no matter what the process CWD happens to be. - adr_dir = resolve_from_root(root, args.dir) + adr_dir, error = resolve_from_root_or_error(root, args.dir, operation="check") + if error: + return error if not adr_dir.is_dir(): # Silently proceeding here would emit a confident "no conflicts" for # what is really a configuration error. diff --git a/skills/adr-toolkit/scripts/commands/create.py b/skills/adr-toolkit/scripts/commands/create.py index b4df40e..e2363d2 100644 --- a/skills/adr-toolkit/scripts/commands/create.py +++ b/skills/adr-toolkit/scripts/commands/create.py @@ -10,7 +10,7 @@ from scripts.core.config import ConfigError, resolve_locale from scripts.core.locale import DEFAULT_LOCALE from scripts.core.rendering import interactive_prompts, render_minimal -from scripts.core.repository_paths import resolve_from_root +from scripts.core.repository_paths import resolve_from_root_or_error from scripts.core.schema import validate_frontmatter REQUIRED_DRAFT_FIELDS = {"title", "status", "body"} @@ -123,7 +123,9 @@ def run(args) -> dict: "errors": [{"code": "CONFIG_ERROR", "detail": str(exc)}], } - adr_dir = resolve_from_root(root, args.dir) + adr_dir, error = resolve_from_root_or_error(root, args.dir, operation="create") + if error: + return error missing = REQUIRED_DRAFT_FIELDS - draft.keys() if missing: return { diff --git a/skills/adr-toolkit/scripts/commands/exception.py b/skills/adr-toolkit/scripts/commands/exception.py index 3f933fb..d8ae726 100644 --- a/skills/adr-toolkit/scripts/commands/exception.py +++ b/skills/adr-toolkit/scripts/commands/exception.py @@ -5,7 +5,7 @@ from scripts.core import atomic_io from scripts.core.exceptions import validate_exception -from scripts.core.repository_paths import resolve_from_root +from scripts.core.repository_paths import resolve_from_root_or_error REQUIRED_DRAFT_FIELDS = {"adr_id", "rule_id", "owner", "reason", "scope", "expiry"} @@ -65,7 +65,9 @@ def run(args) -> dict: "errors": [{"code": "MISSING_DRAFT_FIELD", "fields": sorted(missing)}], } - adr_dir = resolve_from_root(root, args.dir) + adr_dir, error = resolve_from_root_or_error(root, args.dir, operation="exception") + if error: + return error exceptions_dir = adr_dir / "exceptions" # Validate against a preview ID first. This must not touch disk -- not diff --git a/skills/adr-toolkit/scripts/commands/graph.py b/skills/adr-toolkit/scripts/commands/graph.py index 19c9ffb..0c1d95d 100644 --- a/skills/adr-toolkit/scripts/commands/graph.py +++ b/skills/adr-toolkit/scripts/commands/graph.py @@ -4,12 +4,14 @@ from scripts.core import frontmatter as fm from scripts.core.adr_directory import iter_adr_files from scripts.core.relationships import render_mermaid, render_svg, resolve -from scripts.core.repository_paths import resolve_from_root +from scripts.core.repository_paths import PathEscapesRootError, resolve_from_root, resolve_from_root_or_error def run(args) -> dict: root = Path(getattr(args, "root", ".")) - adr_dir = resolve_from_root(root, args.dir) + adr_dir, error = resolve_from_root_or_error(root, args.dir, operation="graph") + if error: + return error entries = [] warnings = [] @@ -33,16 +35,19 @@ def run(args) -> dict: output = getattr(args, "output", None) format_ = getattr(args, "format", "both") outputs = [] - if format_ in {"mermaid", "both"}: - path = _output_path(root, adr_dir, output, format_, "mermaid") - path.parent.mkdir(parents=True, exist_ok=True) - path.write_text(render_mermaid(entries), encoding="utf-8") - outputs.append(str(path)) - if format_ in {"svg", "both"}: - path = _output_path(root, adr_dir, output, format_, "svg") - path.parent.mkdir(parents=True, exist_ok=True) - path.write_text(render_svg(entries), encoding="utf-8") - outputs.append(str(path)) + try: + if format_ in {"mermaid", "both"}: + path = _output_path(root, adr_dir, output, format_, "mermaid") + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(render_mermaid(entries), encoding="utf-8") + outputs.append(str(path)) + if format_ in {"svg", "both"}: + path = _output_path(root, adr_dir, output, format_, "svg") + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(render_svg(entries), encoding="utf-8") + outputs.append(str(path)) + except PathEscapesRootError as exc: + return {"ok": False, "operation": "graph", "errors": [{"code": exc.error_code, "detail": str(exc)}]} rendered_edges = [edge for edge in resolve(entries) if edge.type in {"related", "supersedes"}] return { diff --git a/skills/adr-toolkit/scripts/commands/index.py b/skills/adr-toolkit/scripts/commands/index.py index 56112b3..6a8e871 100644 --- a/skills/adr-toolkit/scripts/commands/index.py +++ b/skills/adr-toolkit/scripts/commands/index.py @@ -7,7 +7,7 @@ from scripts.core.locale import load_locale from scripts.core.relationships import render_mermaid, resolve from scripts.core.rendering import safe_md_link_text -from scripts.core.repository_paths import resolve_from_root +from scripts.core.repository_paths import resolve_from_root_or_error # Last-resort English headers, used when even the English locale file is # unavailable — e.g. a copy-based install (permitted by @@ -29,7 +29,9 @@ def run(args) -> dict: root = Path(getattr(args, "root", ".")) - adr_dir = resolve_from_root(root, args.dir) + adr_dir, error = resolve_from_root_or_error(root, args.dir, operation="index") + if error: + return error try: locale = resolve_locale( cli_locale=getattr(args, "locale", None), diff --git a/skills/adr-toolkit/scripts/commands/init.py b/skills/adr-toolkit/scripts/commands/init.py index 644aa59..348c33a 100644 --- a/skills/adr-toolkit/scripts/commands/init.py +++ b/skills/adr-toolkit/scripts/commands/init.py @@ -10,12 +10,14 @@ resolve_locale, ) from scripts.core.rendering import render_initial_adr, render_template -from scripts.core.repository_paths import resolve_from_root +from scripts.core.repository_paths import resolve_from_root_or_error def run(args) -> dict: root = Path(getattr(args, "root", ".")) - adr_dir = resolve_from_root(root, args.dir) + adr_dir, error = resolve_from_root_or_error(root, args.dir, operation="init") + if error: + return error dry_run = getattr(args, "dry_run", False) if adr_dir.exists() and any(adr_dir.iterdir()): diff --git a/skills/adr-toolkit/scripts/commands/validate.py b/skills/adr-toolkit/scripts/commands/validate.py index afb614c..5c70406 100644 --- a/skills/adr-toolkit/scripts/commands/validate.py +++ b/skills/adr-toolkit/scripts/commands/validate.py @@ -6,12 +6,14 @@ from scripts.core.config import ConfigError, load_repository_config from scripts.core.relationships import find_cycles, missing_targets, resolve, supersession_mismatches from scripts.core.schema import validate_frontmatter -from scripts.core.repository_paths import resolve_from_root +from scripts.core.repository_paths import resolve_from_root_or_error def run(args) -> dict: root = Path(getattr(args, "root", ".")) - adr_dir = resolve_from_root(root, args.dir) + adr_dir, error = resolve_from_root_or_error(root, args.dir, operation="validate") + if error: + return error errors = [] try: diff --git a/skills/adr-toolkit/scripts/core/config.py b/skills/adr-toolkit/scripts/core/config.py index d2c77e2..f422562 100644 --- a/skills/adr-toolkit/scripts/core/config.py +++ b/skills/adr-toolkit/scripts/core/config.py @@ -3,6 +3,7 @@ from pathlib import Path from typing import Optional +from scripts.core.errors import AdrToolkitError from scripts.core.locale import DEFAULT_LOCALE, SUPPORTED_LOCALES CONFIG_FILENAME = ".adr-toolkit.json" @@ -10,8 +11,9 @@ ALLOWED_KEYS = {"schema_version", "locale"} -class ConfigError(ValueError): +class ConfigError(AdrToolkitError): """The repository configuration is malformed or unsupported.""" + error_code = "CONFIG_ERROR" def load_repository_config(root: Path) -> dict: diff --git a/skills/adr-toolkit/scripts/core/constraints.py b/skills/adr-toolkit/scripts/core/constraints.py index 790a5d0..f84c91d 100644 --- a/skills/adr-toolkit/scripts/core/constraints.py +++ b/skills/adr-toolkit/scripts/core/constraints.py @@ -9,6 +9,8 @@ import json import re +from scripts.core.errors import AdrToolkitError + FENCE_RE = re.compile(r"```ya?ml\n(.*?)\n```", re.DOTALL) KNOWN_FIELDS = {"id", "kind", "paths", "pattern", "severity", "message"} @@ -27,8 +29,8 @@ } -class ConstraintsError(ValueError): - pass +class ConstraintsError(AdrToolkitError): + error_code = "BAD_CONSTRAINTS" def extract_constraints(body: str) -> list: diff --git a/skills/adr-toolkit/scripts/core/errors.py b/skills/adr-toolkit/scripts/core/errors.py new file mode 100644 index 0000000..d242b91 --- /dev/null +++ b/skills/adr-toolkit/scripts/core/errors.py @@ -0,0 +1,10 @@ +"""Common base for ADR Toolkit's domain-specific exceptions. + +Lets a caller catch "any ADR Toolkit domain error" in one except clause, +and gives every such error a stable, class-level error_code instead of +each call site retyping the same string. +""" + + +class AdrToolkitError(Exception): + error_code: str = "UNKNOWN_ERROR" diff --git a/skills/adr-toolkit/scripts/core/frontmatter.py b/skills/adr-toolkit/scripts/core/frontmatter.py index 5bf087e..d9dc2af 100644 --- a/skills/adr-toolkit/scripts/core/frontmatter.py +++ b/skills/adr-toolkit/scripts/core/frontmatter.py @@ -5,11 +5,13 @@ """ import re +from scripts.core.errors import AdrToolkitError + FRONTMATTER_RE = re.compile(r"\A---\n(.*?)\n---\n(.*)\Z", re.DOTALL) -class FrontmatterError(ValueError): - pass +class FrontmatterError(AdrToolkitError): + error_code = "BAD_FRONTMATTER" def parse(text: str) -> tuple: diff --git a/skills/adr-toolkit/scripts/core/git_paths.py b/skills/adr-toolkit/scripts/core/git_paths.py index 9b8b028..3d632e1 100644 --- a/skills/adr-toolkit/scripts/core/git_paths.py +++ b/skills/adr-toolkit/scripts/core/git_paths.py @@ -2,9 +2,12 @@ import subprocess from pathlib import Path +from scripts.core.errors import AdrToolkitError -class GitPathsError(RuntimeError): + +class GitPathsError(AdrToolkitError): """Git could not provide a trustworthy repository path inventory.""" + error_code = "GIT_LS_FILES_FAILED" def list_existing_paths(root: Path) -> set: diff --git a/skills/adr-toolkit/scripts/core/lifecycle.py b/skills/adr-toolkit/scripts/core/lifecycle.py index 520d96f..b3b932b 100644 --- a/skills/adr-toolkit/scripts/core/lifecycle.py +++ b/skills/adr-toolkit/scripts/core/lifecycle.py @@ -1,4 +1,5 @@ """ADR status lifecycle rules.""" +from scripts.core.errors import AdrToolkitError STATUSES = {"proposed", "accepted", "rejected", "deprecated", "superseded"} @@ -11,8 +12,8 @@ } -class InvalidTransitionError(ValueError): - pass +class InvalidTransitionError(AdrToolkitError): + error_code = "INVALID_TRANSITION" def validate_transition(current: str, target: str) -> None: diff --git a/skills/adr-toolkit/scripts/core/repository_paths.py b/skills/adr-toolkit/scripts/core/repository_paths.py index 43f9e0c..44a2a50 100644 --- a/skills/adr-toolkit/scripts/core/repository_paths.py +++ b/skills/adr-toolkit/scripts/core/repository_paths.py @@ -1,9 +1,12 @@ """Resolve repository-owned paths independently of the caller's CWD.""" from pathlib import Path +from scripts.core.errors import AdrToolkitError -class PathEscapesRootError(ValueError): + +class PathEscapesRootError(AdrToolkitError): """A relative path resolved outside the root it was given against.""" + error_code = "PATH_ESCAPES_ROOT" def resolve_from_root(root, path) -> Path: @@ -19,3 +22,19 @@ def resolve_from_root(root, path) -> Path: if not joined.resolve().is_relative_to(root_resolved): raise PathEscapesRootError(f"{str(path)!r} escapes root {str(root)!r}") return joined + + +def resolve_from_root_or_error(root, path, *, operation: str): + """Same as resolve_from_root, but converts a rejected escape into the + same {"ok": False, "operation": ..., "errors": [...]} shape every + other domain error already gets at its call site, instead of falling + through to adr.py's generic INTERNAL_ERROR. Returns (Path, None) on + success or (None, error_dict) on rejection.""" + try: + return resolve_from_root(root, path), None + except PathEscapesRootError as exc: + return None, { + "ok": False, + "operation": operation, + "errors": [{"code": exc.error_code, "detail": str(exc)}], + } diff --git a/tests/integration/test_path_escape_error_wiring.py b/tests/integration/test_path_escape_error_wiring.py new file mode 100644 index 0000000..d8b3ccd --- /dev/null +++ b/tests/integration/test_path_escape_error_wiring.py @@ -0,0 +1,78 @@ +"""Proves every resolve_from_root call site converts a path-escape attempt +into a structured PATH_ESCAPES_ROOT error instead of an opaque +INTERNAL_ERROR (docs/adr-toolkit-audit-report.md §2.2 2.3, closing a gap +left by the High-priority pass).""" +import json +import subprocess +from types import SimpleNamespace + +from scripts.commands import check, create, exception, graph, index, init, validate + +ESCAPING_DIR = "../../outside" + + +def _assert_rejected(result, operation): + assert result["ok"] is False + assert result["operation"] == operation + assert result["errors"][0]["code"] == "PATH_ESCAPES_ROOT" + + +def test_init_rejects_escaping_dir(tmp_path): + result = init.run(SimpleNamespace(dir=ESCAPING_DIR, root=str(tmp_path), locale=None, dry_run=False)) + _assert_rejected(result, "init") + + +def test_index_rejects_escaping_dir(tmp_path): + result = index.run(SimpleNamespace(dir=ESCAPING_DIR, root=str(tmp_path), locale=None)) + _assert_rejected(result, "index") + + +def test_validate_rejects_escaping_dir(tmp_path): + result = validate.run(SimpleNamespace(dir=ESCAPING_DIR, root=str(tmp_path))) + _assert_rejected(result, "validate") + + +def test_check_rejects_escaping_dir(tmp_path): + subprocess.run(["git", "init", "-q"], cwd=tmp_path, check=True) + result = check.run(SimpleNamespace(dir=ESCAPING_DIR, root=str(tmp_path), staged=False, since=None)) + _assert_rejected(result, "check") + + +def test_graph_rejects_escaping_dir(tmp_path): + result = graph.run(SimpleNamespace(dir=ESCAPING_DIR, root=str(tmp_path), format="both", output=None)) + _assert_rejected(result, "graph") + + +def test_graph_rejects_escaping_output(tmp_path): + adr_dir = tmp_path / "docs" / "decisions" + adr_dir.mkdir(parents=True) + result = graph.run(SimpleNamespace( + dir=str(adr_dir), root=str(tmp_path), format="mermaid", output=ESCAPING_DIR, + )) + _assert_rejected(result, "graph") + + +def test_create_rejects_escaping_dir(tmp_path): + draft_path = tmp_path / "draft.json" + draft_path.write_text( + json.dumps({"title": "Use Kafka", "status": "proposed", "body": "Body."}), + encoding="utf-8", + ) + result = create.run(SimpleNamespace( + input=str(draft_path), interactive=False, dir=ESCAPING_DIR, root=str(tmp_path), + locale=None, slug=None, dry_run=False, + )) + _assert_rejected(result, "create") + + +def test_exception_rejects_escaping_dir(tmp_path): + draft_path = tmp_path / "draft.json" + draft_path.write_text( + json.dumps({ + "adr_id": "ADR-0001", "rule_id": "r", "owner": "o", "reason": "r", + "scope": ["src/**"], "expiry": "2099-01-01", + }), + encoding="utf-8", + ) + result = exception.run(SimpleNamespace(input=str(draft_path), dir=ESCAPING_DIR, root=str(tmp_path), dry_run=False)) + _assert_rejected(result, "exception") diff --git a/tests/unit/test_errors.py b/tests/unit/test_errors.py new file mode 100644 index 0000000..4f6d64d --- /dev/null +++ b/tests/unit/test_errors.py @@ -0,0 +1,28 @@ +"""Tests that every ADR Toolkit domain exception shares a common base and +a stable error_code (docs/adr-toolkit-audit-report.md §2.4 4.3).""" +from scripts.core.config import ConfigError +from scripts.core.constraints import ConstraintsError +from scripts.core.errors import AdrToolkitError +from scripts.core.frontmatter import FrontmatterError +from scripts.core.git_paths import GitPathsError +from scripts.core.lifecycle import InvalidTransitionError +from scripts.core.repository_paths import PathEscapesRootError + +EXPECTED = { + ConfigError: "CONFIG_ERROR", + FrontmatterError: "BAD_FRONTMATTER", + ConstraintsError: "BAD_CONSTRAINTS", + InvalidTransitionError: "INVALID_TRANSITION", + GitPathsError: "GIT_LS_FILES_FAILED", + PathEscapesRootError: "PATH_ESCAPES_ROOT", +} + + +def test_every_domain_exception_is_an_adr_toolkit_error(): + for cls in EXPECTED: + assert issubclass(cls, AdrToolkitError) + + +def test_every_domain_exception_has_its_documented_error_code(): + for cls, expected_code in EXPECTED.items(): + assert cls.error_code == expected_code From e3d592b6898dfa60bb02cde8d325c9c712f31b8e Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 10:31:55 +0900 Subject: [PATCH 27/58] test: detect drift between JSON Schema docs and runtime validators Cross-checks schemas/adr.schema.json and schemas/exception.schema.json's required-field lists and enum values against core/schema.py, core/exceptions.py, core/lifecycle.py, and core/locale.py. Deliberately stdlib-only -- adopting the `jsonschema` library, as the audit report originally sketched, would trade away this project's zero-dependency design for a moderate documentation-drift risk. Verified the test has teeth by temporarily removing a required field from the schema file and confirming it fails (reverted before committing). --- tests/unit/test_schema_contract_sync.py | 41 +++++++++++++++++++++++++ 1 file changed, 41 insertions(+) create mode 100644 tests/unit/test_schema_contract_sync.py diff --git a/tests/unit/test_schema_contract_sync.py b/tests/unit/test_schema_contract_sync.py new file mode 100644 index 0000000..217e87c --- /dev/null +++ b/tests/unit/test_schema_contract_sync.py @@ -0,0 +1,41 @@ +"""Detects drift between the hand-rolled runtime validators and the JSON +Schema files in schemas/ that document the same shape for external tools +(docs/adr-toolkit-audit-report.md §2.4 4.2). + +Deliberately stdlib-only -- this project has zero third-party runtime +dependencies by design, and adopting the `jsonschema` library just to keep +two already-hand-maintained definitions in sync would be a worse trade +than a plain field-name/enum comparison.""" +import json +from pathlib import Path + +from scripts.core.exceptions import REQUIRED_FIELDS as EXCEPTION_REQUIRED_FIELDS +from scripts.core.lifecycle import STATUSES +from scripts.core.locale import SUPPORTED_LOCALES +from scripts.core.schema import REQUIRED_FIELDS as ADR_REQUIRED_FIELDS + +_SCHEMAS_DIR = Path(__file__).resolve().parents[2] / "skills" / "adr-toolkit" / "schemas" + + +def _load_schema(filename: str) -> dict: + return json.loads((_SCHEMAS_DIR / filename).read_text(encoding="utf-8")) + + +def test_adr_schema_required_fields_match_the_runtime_validator(): + schema = _load_schema("adr.schema.json") + assert set(schema["required"]) == set(ADR_REQUIRED_FIELDS) + + +def test_adr_schema_status_enum_matches_the_runtime_lifecycle_statuses(): + schema = _load_schema("adr.schema.json") + assert set(schema["properties"]["status"]["enum"]) == STATUSES + + +def test_adr_schema_locale_enum_matches_the_runtime_supported_locales(): + schema = _load_schema("adr.schema.json") + assert set(schema["properties"]["locale"]["enum"]) == set(SUPPORTED_LOCALES) + + +def test_exception_schema_required_fields_match_the_runtime_validator(): + schema = _load_schema("exception.schema.json") + assert set(schema["required"]) == set(EXCEPTION_REQUIRED_FIELDS) From 2933575c57a4eedb78813d3c0abf651983efdf4c Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 10:33:22 +0900 Subject: [PATCH 28/58] feat: extend output contract coverage to CHECK's result shape Adds CheckFinding and CheckResult TypedDicts, using Dict[str, Any] for the genuinely heterogeneous evidence/exception/warning payloads (bare `dict` fails mypy --strict's type-arg check). Only CreateResult and CheckResult are covered so far -- the other 14 commands remain future work per the module's existing docstring. --- skills/adr-toolkit/scripts/core/contracts.py | 22 +++++++++++++++++++- tests/unit/test_contracts.py | 18 +++++++++++++++- 2 files changed, 38 insertions(+), 2 deletions(-) diff --git a/skills/adr-toolkit/scripts/core/contracts.py b/skills/adr-toolkit/scripts/core/contracts.py index ed25219..229d8d5 100644 --- a/skills/adr-toolkit/scripts/core/contracts.py +++ b/skills/adr-toolkit/scripts/core/contracts.py @@ -7,7 +7,7 @@ which TypedDict can't model without a larger refactor; that is tracked separately and not attempted here. """ -from typing import List, TypedDict +from typing import Any, Dict, List, TypedDict class CommandError(TypedDict, total=False): @@ -31,3 +31,23 @@ class CreateResult(BaseResult, total=False): would_create: str id: str errors: List[CommandError] + + +class CheckFinding(TypedDict, total=False): + adr_id: str + kind: str + confidence: str + rule_id: str + severity: str + message: str + file: str + evidence: Dict[str, Any] + resolutions: List[str] + exception: Dict[str, Any] + + +class CheckResult(BaseResult, total=False): + diff: Dict[str, Any] + findings: List[CheckFinding] + warnings: List[Dict[str, Any]] + errors: List[CommandError] diff --git a/tests/unit/test_contracts.py b/tests/unit/test_contracts.py index 94de492..6e65aad 100644 --- a/tests/unit/test_contracts.py +++ b/tests/unit/test_contracts.py @@ -1,8 +1,9 @@ """Tests that core/contracts.py's TypedDicts describe real command output shapes (docs/adr-toolkit-audit-report.md §2.4 4.1).""" +import subprocess from types import SimpleNamespace -from scripts.commands import create +from scripts.commands import check, create from scripts.core import contracts @@ -23,3 +24,18 @@ def test_create_dry_run_result_matches_contract_keys(tmp_path): f"create.run() returned keys not in contracts.CreateResult: " f"{set(result.keys()) - contract_keys}" ) + + +def test_check_result_with_no_adrs_matches_contract_keys(tmp_path): + subprocess.run(["git", "init", "-q"], cwd=tmp_path, check=True) + adr_dir = tmp_path / "docs" / "decisions" + adr_dir.mkdir(parents=True) + + result = check.run(SimpleNamespace(root=str(tmp_path), dir=str(adr_dir), staged=False, since=None)) + + assert result["ok"] is True + contract_keys = set(contracts.CheckResult.__annotations__) + assert set(result.keys()) <= contract_keys, ( + f"check.run() returned keys not in contracts.CheckResult: " + f"{set(result.keys()) - contract_keys}" + ) From d7368f64a426b43f24839688d1e41e38b4be1c51 Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 10:33:57 +0900 Subject: [PATCH 29/58] test: add a bulk-ADR performance sanity check 200 generated ADRs, search+index complete in ~0.07s (bound: 5s). Proves no catastrophic (e.g. quadratic) blowup rather than building a benchmarking system -- the audit's original "2,000 fixtures + CI regression tracking" needs historical-baseline infrastructure this project doesn't have. --- .../integration/test_bulk_adr_performance.py | 49 +++++++++++++++++++ 1 file changed, 49 insertions(+) create mode 100644 tests/integration/test_bulk_adr_performance.py diff --git a/tests/integration/test_bulk_adr_performance.py b/tests/integration/test_bulk_adr_performance.py new file mode 100644 index 0000000..007b82e --- /dev/null +++ b/tests/integration/test_bulk_adr_performance.py @@ -0,0 +1,49 @@ +"""Proves search/index don't blow up catastrophically as ADR count grows +(docs/adr-toolkit-audit-report.md §2.3 3.1, scoped down from "2,000 +fixtures + CI regression tracking" -- no historical-baseline +infrastructure exists to make trend tracking meaningful; this instead +asserts a generous, one-shot wall-clock bound at a size well past any +observed real-world ADR count).""" +import time +from types import SimpleNamespace + +from scripts.commands import index, search + + +def _write_adr(adr_dir, number, title): + text = ( + "---\n" + f"id: ADR-{number:04d}\n" + f"title: {title}\n" + "status: accepted\n" + "date: 2026-01-01\n" + "decision_makers: []\n" + "related: []\n" + "affected_paths: []\n" + "tags:\n" + " - performance\n" + "retrospective: false\n" + "---\n\n" + f"# {title}\n\nDecision body text for ADR {number}.\n" + ) + (adr_dir / f"{number:04d}-decision-{number}.md").write_text(text, encoding="utf-8") + + +def test_search_and_index_handle_200_adrs_without_catastrophic_slowdown(tmp_path): + adr_dir = tmp_path / "docs" / "decisions" + adr_dir.mkdir(parents=True) + for i in range(1, 201): + _write_adr(adr_dir, i, f"Decision number {i}") + + started = time.monotonic() + search_result = search.run(SimpleNamespace( + dir=str(adr_dir), id=None, keyword="decision", tags=None, status=None, path=None, limit=None, + )) + index_result = index.run(SimpleNamespace(dir=str(adr_dir), root=str(tmp_path), locale=None)) + elapsed = time.monotonic() - started + + assert search_result["ok"] is True + assert search_result["total"] == 200 + assert index_result["ok"] is True + assert index_result["count"] == 200 + assert elapsed < 5.0, f"search+index over 200 ADRs took {elapsed:.2f}s -- investigate before real repos hit this scale" From c3ed01d26e4e9714687b8e7a634cecaba1c5ee38 Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 10:34:57 +0900 Subject: [PATCH 30/58] feat: add a TTY-only human summary line on stderr adr.py prints one dim "-> ok|FAILED" line to stderr when stderr is a real terminal, suppressible via ADR_TOOLKIT_NO_COLOR=1. stdout's JSON contract is completely unaffected, and the common piped/redirected case (e.g. `adr.py check ... | jq`) sees no extra output since capsys/pipes never report isatty() as true. --- skills/adr-toolkit/scripts/adr.py | 4 ++++ tests/unit/test_tty_summary.py | 35 +++++++++++++++++++++++++++++++ 2 files changed, 39 insertions(+) create mode 100644 tests/unit/test_tty_summary.py diff --git a/skills/adr-toolkit/scripts/adr.py b/skills/adr-toolkit/scripts/adr.py index 30edc9b..1a78551 100755 --- a/skills/adr-toolkit/scripts/adr.py +++ b/skills/adr-toolkit/scripts/adr.py @@ -2,6 +2,7 @@ """Single entrypoint for all ADR Toolkit deterministic operations.""" import argparse import json +import os import sys import time from pathlib import Path @@ -208,6 +209,9 @@ def main(argv=None) -> int: if getattr(args, "diagnostic", False): result["_diagnostics"] = {"elapsed_ms": round((time.perf_counter() - started_at) * 1000, 1)} print(json.dumps(result, indent=2, ensure_ascii=False)) + if sys.stderr.isatty() and not os.environ.get("ADR_TOOLKIT_NO_COLOR"): + status_word = "ok" if result.get("ok") else "FAILED" + print(f"\033[2m→ {args.operation} {status_word}\033[0m", file=sys.stderr) return 0 if result.get("ok") else 1 diff --git a/tests/unit/test_tty_summary.py b/tests/unit/test_tty_summary.py new file mode 100644 index 0000000..154aca7 --- /dev/null +++ b/tests/unit/test_tty_summary.py @@ -0,0 +1,35 @@ +"""Tests for the TTY-only human summary line +(docs/adr-toolkit-audit-report.md §2.6 6.2).""" +import sys + +from scripts import adr + + +def test_summary_line_appears_when_stderr_is_a_tty(tmp_path, monkeypatch, capsys): + monkeypatch.setattr(sys.stderr, "isatty", lambda: True) + monkeypatch.delenv("ADR_TOOLKIT_NO_COLOR", raising=False) + + exit_code = adr.main(["preflight", "--root", str(tmp_path)]) + + assert exit_code == 0 + captured = capsys.readouterr() + assert "preflight" in captured.err + assert "ok" in captured.err + + +def test_no_summary_line_when_stderr_is_not_a_tty(tmp_path, capsys): + # capsys-captured stderr is not a real TTY by default -- this is the + # common redirected/piped case, e.g. `adr.py check ... | jq`. + exit_code = adr.main(["preflight", "--root", str(tmp_path)]) + + assert exit_code == 0 + assert capsys.readouterr().err == "" + + +def test_no_summary_line_when_no_color_env_is_set(tmp_path, monkeypatch, capsys): + monkeypatch.setattr(sys.stderr, "isatty", lambda: True) + monkeypatch.setenv("ADR_TOOLKIT_NO_COLOR", "1") + + adr.main(["preflight", "--root", str(tmp_path)]) + + assert capsys.readouterr().err == "" From 77ce2068655ff09ceba178f0247bba9f5f2c0aaa Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 10:36:22 +0900 Subject: [PATCH 31/58] docs: close out Medium-priority hardening backlog items improvements.md's Medium section now records the parsing-cache decline with its rationale and the output-contract item's partial (2/16) status; everything else in Medium is removed as done. handoff.md summarizes all 3 hardening passes (Critical, High, Medium) completed this session -- 19 implementation commits total -- plus the discovered gaps and their fixes, for a future session or different harness to resume from cold. --- changelog.md | 14 ++++ handoff.md | 210 ++++++++++++++++++++++++------------------------ improvements.md | 26 +++--- 3 files changed, 130 insertions(+), 120 deletions(-) diff --git a/changelog.md b/changelog.md index 7645567..4964d22 100644 --- a/changelog.md +++ b/changelog.md @@ -4,6 +4,20 @@ Lightweight human-readable summary of meaningful repository changes. ## Unreleased +- `PathEscapesRootError` (added in the prior session's path-escape fix) + is now caught at all 7 call sites and reported as a structured + `PATH_ESCAPES_ROOT` error instead of falling through to a generic + internal error. All 6 domain exception classes now share a common + `AdrToolkitError` base with a stable `error_code`. +- Added a test that fails if `schemas/*.json` and the runtime validators + in `core/schema.py`/`core/exceptions.py` ever diverge -- without adding + a `jsonschema` dependency. +- `core/contracts.py` now also covers CHECK's result shape. +- Added a sanity check proving `search`/`index` don't degrade + catastrophically at 200 ADRs. +- `adr.py` now prints a one-line human-readable summary to stderr when + stderr is a real terminal (set `ADR_TOOLKIT_NO_COLOR` to suppress); + stdout's JSON contract and piped/redirected usage are unaffected. - `--dir`/`--root` now reject a relative path that resolves outside the given root, closing a path-escape gap. - CI now measures branch coverage (currently 93%) and fails below 85%. diff --git a/handoff.md b/handoff.md index c98758a..1ce64ff 100644 --- a/handoff.md +++ b/handoff.md @@ -2,133 +2,133 @@ ## Current task (2026-09-01) -**Both the Critical and High-priority hardening passes are done.** All 4 -Critical-risk findings and 6 of 8 High-priority findings from -`docs/adr-toolkit-audit-report.md` are implemented, tested, and committed -on `feature/analyzing-adr-toolkit`. Full suite: 433 passed (up from 395 at -session start). Branch stays as-is per owner's explicit choice (not -merged/PR'd yet). - -**Critical pass** (`docs/superpowers/plans/2026-09-01-critical-hardening.md`, -gitignored by convention, still on disk in this worktree): - -1. `fc46830` feat: add atomic write and directory lock primitives -2. `49ede49` fix: make ADR creation race-free under concurrent invocation -3. `68bbd98` fix: make exception creation race-free under concurrent invocation -4. `cec7215` fix: make SUPERSEDE writes atomic and lock-protected -5. `c0ff907` fix: add ReDoS timeout guard to CHECK's author-supplied regex patterns -6. `7afdcd5` fix: escape ADR titles in generated README to prevent link injection -7. `11c8f4b` feat: add structured stderr logging with correlation IDs -8. `f0cbc86` docs: close out Critical hardening backlog items - -**High-priority pass** (`docs/superpowers/plans/2026-09-01-high-priority-hardening.md`, -same gitignore convention): - -1. `f92c8f5` fix: reject a --dir/--root that escapes the repository root -2. `52bd761` ci: measure and gate branch coverage at 85% -3. `305c836` feat: add typed result contracts and a mypy --strict CI gate -4. `46dd863` feat: add --diagnostic flag for per-invocation timing -5. `9708bb2` test: prove atomic_write_text survives a mid-write SIGKILL -6. `cff3d5e` feat: extract a shared adapter-manifest validator -7. (this commit) docs: close out High-priority hardening backlog items - -`improvements.md` now has an empty `### Critical` section (removed -entirely) and a `### High` section containing only the 2 items explicitly -deferred to another worktree. The `### Medium` section is untouched and -unscheduled. - -Notable things discovered mid-session, worth knowing if touching this code -again: +**The Critical, High-priority, and Medium-priority hardening passes are +all done.** Every findable item from `docs/adr-toolkit-audit-report.md` +that was in scope for this worktree is implemented, tested, and +committed on `feature/analyzing-adr-toolkit`. Full suite: 452 passed (up +from 395 at session start). Branch stays as-is per owner's explicit +choice (not merged/PR'd yet). + +**Critical pass** (`docs/superpowers/plans/2026-09-01-critical-hardening.md`): +`fc46830` atomic write + lock primitives, `49ede49` create.py race fix, +`68bbd98` exception.py race fix, `cec7215` supersede.py atomic writes, +`c0ff907` ReDoS guard, `7afdcd5` README link-injection fix, `11c8f4b` +structured logging, `f0cbc86` docs closeout. + +**High-priority pass** (`docs/superpowers/plans/2026-09-01-high-priority-hardening.md`): +`f92c8f5` path escape guard (initial), `52bd761` coverage CI gate, +`305c836` mypy strict gate + contracts.py, `46dd863` `--diagnostic` flag, +`9708bb2` SIGKILL chaos test, `cff3d5e` adapter manifest validator, +`9a974b7` docs closeout. + +**Medium-priority pass** (`docs/superpowers/plans/2026-09-01-medium-priority-hardening.md`): +`41998a2` common `AdrToolkitError` base + fixed the path-escape gap the +High pass left behind (all 7 `resolve_from_root` call sites now catch +it), `e3d592b` schema-drift detection test (no `jsonschema` dependency +added -- see rationale in that commit), `2933575` extended +`contracts.py` to cover CHECK, `d7368f6` bulk-ADR performance sanity +check, `c3ed01d` TTY-only stderr summary line, (this commit) docs +closeout. + +All 3 plan files are gitignored by convention (`docs/superpowers/plans/`) +but still on disk in this worktree. + +`improvements.md` now has: an empty `### Critical` section, a `### High` +section containing only the 2 items explicitly deferred to another +worktree, and a `### Medium` section with one item marked **declined with +rationale** (parsing-result caching -- see below) and one marked +partially done (output contract schema, 2 of 16 commands covered). + +**One Medium item was declined, not silently skipped:** the audit's +`functools.lru_cache` suggestion for parsing-result caching provides zero +real benefit for this CLI -- it's a fresh process per invocation (no +shared memory across separate `python adr.py X` calls, which was the +actual scenario the audit worried about), and no single command +internally re-parses the same file more than once. A cache that would +actually help (persistent, on-disk, mtime-keyed, shared across process +invocations) is a much bigger, staleness-risk-bearing feature +disproportionate to real ADR counts. Recorded in `improvements.md` with +this rationale. + +Notable things discovered mid-session, worth knowing if touching this +code again: - `create.py`/`exception.py`: naively wrapping everything in `adr_directory_lock` made *dry runs* (and, for `exception.py`, - *schema-validation failures*) create the directory + lock file as a side - effect, breaking existing tests that assert nothing is created. Fixed by - keeping preview/validation paths outside the lock and only locking the - actual allocate-and-write step. + *schema-validation failures*) create the directory + lock file as a + side effect, breaking existing tests. Fixed by keeping preview/ + validation paths outside the lock. - `supersede.py`: two existing tests monkeypatched `Path.write_text` - directly to simulate a write failure; that seam disappeared once writes - route through `atomic_io.atomic_write_text`, so both were retargeted to - patch `supersede.atomic_io.atomic_write_text` instead (same intent, same - assertions). -- Repo-root `scripts/` and `skills/adr-toolkit/scripts/` share the name - `scripts` for Python's import system, and the latter (which has an - `__init__.py`) wins whichever imports first in a pytest session. Any new - file under repo-root `scripts/` must be loaded via - `importlib.util.spec_from_file_location` in its tests, exactly like - `scripts/sync_version.py` already does -- `scripts/adapter_sdk.py` - follows the same pattern. -- Measured, not assumed: branch+statement coverage was 93.32% before - adding the 85% CI gate; `mypy --strict` on `atomic_io.py`/`telemetry.py` - had exactly 3 real errors, fixed as part of adding the `type-check` job. + directly; retargeted to `supersede.atomic_io.atomic_write_text` once + writes moved through it. +- Repo-root `scripts/` and `skills/adr-toolkit/scripts/` share the import + name `scripts` -- anything new under repo-root `scripts/` needs + `importlib.util.spec_from_file_location` in its tests, like + `scripts/sync_version.py` and `scripts/adapter_sdk.py` both do. +- `PathEscapesRootError` was added in the High-priority pass but never + actually caught anywhere until the Medium pass noticed and fixed it -- + every *other* domain exception in this codebase is caught explicitly at + its call site, so an uncaught one was an inconsistency worth closing. +- Measured, not assumed: branch+statement coverage was 93.32% before the + 85% CI gate; `mypy --strict` had exactly 3 real errors on the 2 + pre-existing typed modules; a bare `dict` field in a TypedDict fails + `mypy --strict`'s `type-arg` check -- use `Dict[str, Any]`. ## Scope for this worktree Excluded here, being handled elsewhere -- do not touch: - Domains 1 (core/plugin architecture) and 5 (governance/FSM) from the - audit report -- already scored 72/80, mostly "no action needed" per the - audit itself. -- Anything Antigravity (`agy`) adapter-related -- owner is working on this - in another branch. -- Automatic version sync -- owner is working on this in another worktree; - as a direct consequence, **do not touch `.github/workflows/release.yml` - for any reason**. The remaining "(다른 워크트리 확인)" items in - `improvements.md`'s `### High` section (supply-chain checksums/signing; - the 8.4 auto-version-bump direction note) are deliberately left there - for that other worktree. -- README prose (root README.md, `adapters/*/README.md` content) -- another - worktree. Every fix in this session that touched adapter or index code - was a code/generator fix, not README prose, and correctly stayed in - scope here. + audit report. +- Anything Antigravity (`agy`) adapter-related -- another branch. +- Automatic version sync -- another worktree; **do not touch + `.github/workflows/release.yml` for any reason.** The 2 remaining + "(다른 워크트리 확인)" items in `improvements.md`'s `### High` section + are deliberately left there for that other worktree. +- README prose (root README.md, `adapters/*/README.md` content) -- + another worktree. Every fix across all 3 passes that touched adapter or + generator code was a code fix, not README prose. ## Next step Nothing is currently in flight. `improvements.md`'s remaining backlog -- -the 2 other-worktree-flagged High items, plus the entire `### Medium` -section (JSON Schema single-source-of-truth, common error base class, -output contract schema freeze, parsing cache, bulk-ADR benchmark, -TTY-aware CLI output) -- is unscheduled. **Ask the owner before starting -any of it.** Every scope decision in this session (Critical-then-High -ordering, domain 1/5 exclusion, the other-worktree exclusions) was the -owner's explicit call in conversation, not something derivable from the -audit report alone. +2 other-worktree-flagged High items, plus 1 declined and 1 +partially-done Medium item -- is unscheduled. **Ask the owner before +starting any of it.** Every scope decision across all 3 passes +(Critical-then-High-then-Medium ordering, domain 1/5 exclusion, the +other-worktree exclusions, the parsing-cache decline) was the owner's +explicit call or a judgment call made and explained in-session, not +something derivable from the audit report alone. ## Verification -Full suite: `python3 -m pytest tests/unit tests/integration -v` -> 433 -passed as of commit `cff3d5e` (432 on Windows, where the SIGKILL chaos -test in `test_atomic_io_chaos.py` is skipped). +Full suite: `python3 -m pytest tests/unit tests/integration -v` -> 452 +passed as of commit `c3ed01d` (451 on Windows, where the SIGKILL chaos +test is skipped). -CI now also runs a `type-check` job (`mypy --strict` on 3 modules) and -gates the `pytest` job's coverage at 85% -- both new since this session. +CI now also runs a `type-check` job (`mypy --strict`) and gates the +`pytest` job's coverage at 85%. ## Open risks - The ReDoS guard is POSIX-only (`signal.SIGALRM`); Windows CI is - unaffected but unguarded against catastrophic-backtracking patterns -- - a known, documented gap, not a regression introduced by this work. + unaffected but unguarded -- a known, documented gap. - `supersede.py`'s two-file update guarantees each individual file is - never torn by a mid-write crash, but does not guarantee the *pair* - stays consistent if the process is killed between the two atomic writes - -- true two-phase commit across files was explicitly scoped out. -- Every successful `create`/`exception`/`supersede` call now leaves a - `.adr-toolkit.lock` (0-byte, dotfile) inside `docs/decisions/` and - `docs/decisions/exceptions/` permanently -- intentional (the - cross-process mutex), doesn't match `*.md`/`*.json` globs so nothing - else picks it up, but worth knowing about if someone notices it in a - repo diff. -- `core/contracts.py`'s TypedDicts currently model only a subset of one - command's result shape (`CreateResult`) plus the shared error/base - shapes -- extending coverage to the other 15 commands, and extending - `mypy --strict` beyond the 3 fully-typed core modules into the command - modules themselves (blocked on typing `argparse.Namespace` args), is - future work, not started. + never torn by a mid-write crash, but not that the *pair* stays + consistent if killed between the two writes -- true two-phase commit + was explicitly scoped out. +- Every successful `create`/`exception`/`supersede` call leaves a + `.adr-toolkit.lock` (0-byte dotfile) permanently inside + `docs/decisions/` and `docs/decisions/exceptions/` -- intentional (the + cross-process mutex), doesn't match `*.md`/`*.json` globs. +- `core/contracts.py` covers only `CreateResult` and `CheckResult`; + extending to the other 14 commands, and extending `mypy --strict` + beyond the fully-typed core modules into the command modules + themselves (blocked on typing `argparse.Namespace` args), is future + work. - (carried over from the audit, still true) CHECK deliberately cannot - prove prose, business rationale, or organizational claims; those remain - human-review evidence. + prove prose, business rationale, or organizational claims. - (carried over, still true) GitHub branch/tag protection is unavailable - on the current private plan; revisit once the repository goes public -- - owner's stated plan is to do that after most audit findings are done and - the version bumps to 1.0.0 (see project memory - `project_v1_public_release_plan`). + on the current private plan; revisit once the repository goes public + after most audit findings are done and the version bumps to 1.0.0 (see + project memory `project_v1_public_release_plan`). diff --git a/improvements.md b/improvements.md index 7c2b78c..0c9a5dd 100644 --- a/improvements.md +++ b/improvements.md @@ -26,21 +26,17 @@ also touch. ### Medium -- [ ] **런타임 스키마 단일 진실 소스화** — `core/schema.py`를 - `schemas/adr.schema.json`/`exception.schema.json` 기반 `jsonschema` - 검증으로 재작성해 스키마 드리프트 제거. (감사 보고서 §2.4 4.2) -- [ ] **공통 에러 베이스 클래스** — `AdrToolkitError`로 기존 5개 예외 - 클래스 통합, `error_code`를 클래스 속성화. 구조화 로깅 작업과 함께 - 진행하면 자연스러움. (감사 보고서 §2.4 4.3) -- [ ] **출력 계약 스키마 고정(골든 파일)** — 16개 커맨드 출력에 대한 - JSON Schema 스냅샷 테스트, 4.2 작업과 파일 공유 가능. (감사 보고서 - §2.1 1.3 — 도메인 1 제외 대상이지만 4.2와 묶어 진행 시 예외적으로 포함) -- [ ] **파싱 결과 캐시** — `functools.lru_cache` 기반 프로세스 내 - 재파싱 제거. (감사 보고서 §2.3 3.2) -- [ ] **대량 ADR 벤치마크** — 2,000개 픽스처로 `search`/`index` 실행 - 시간 측정, CI 회귀 임계값 설정. (감사 보고서 §2.3 3.1) -- [ ] **CLI TTY 인지 출력** — stderr에 사람이 읽을 요약 라인(비-TTY - 시 무출력). (감사 보고서 §2.6 6.2) +- [ ] ~~**파싱 결과 캐시**~~ — **결정: 하지 않음.** 이 CLI는 호출마다 + 새 프로세스라 `functools.lru_cache`는 프로세스 간 재파싱을 전혀 줄이지 + 못하고(원 문제였던 `validate → index → check` 연쇄 재파싱은 별도 + 프로세스 3개), 실제로 벌어지는 "단일 커맨드 내 동일 파일 중복 파싱"도 + 없음을 확인함(search/index/validate/check 전부 파일당 1회 읽기). + 진짜 도움이 되려면 mtime 키 영속 캐시가 필요한데, 이는 staleness 리스크 + 대비 ADR 실사용 규모(수백 개 미만, 감사 보고서 자체 진단)에 비해 + 과한 투자. (감사 보고서 §2.3 3.2) +- [ ] **출력 계약 스키마 고정(골든 파일)** — `CreateResult`, `CheckResult` + 두 개만 커버됨. 나머지 14개 커맨드는 여전히 미착수. (감사 보고서 §2.1 + 1.3) ## Done From 1df6066ea30f1e29cc704be7120b94c75fbe85cd Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 11:10:02 +0900 Subject: [PATCH 32/58] feat: extend output contract coverage to all 16 commands MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds TypedDicts for preflight, discover, init, index, related, significance, validate, status, supersede, diff, exception, graph, and search, alongside the existing CreateResult/CheckResult -- closing out improvements.md's "출력 계약 스키마 고정" item (was 2/16, now 16/16). Each shape was determined by reading every command's actual return statements (not guessed), and verified against real run() calls including at least one error-path branch for status and supersede. All errors/ warnings/nested-payload fields use Dict[str, Any] rather than the shared CommandError type where a command's real error dicts carry extra fields (file, id, ids, cycle, ...) that CommandError doesn't declare, to avoid overclaiming structure that isn't true. --- skills/adr-toolkit/scripts/core/contracts.py | 94 +++++++++- tests/unit/test_contracts.py | 170 ++++++++++++++++++- 2 files changed, 257 insertions(+), 7 deletions(-) diff --git a/skills/adr-toolkit/scripts/core/contracts.py b/skills/adr-toolkit/scripts/core/contracts.py index 229d8d5..b5cb279 100644 --- a/skills/adr-toolkit/scripts/core/contracts.py +++ b/skills/adr-toolkit/scripts/core/contracts.py @@ -7,7 +7,7 @@ which TypedDict can't model without a larger refactor; that is tracked separately and not attempted here. """ -from typing import Any, Dict, List, TypedDict +from typing import Any, Dict, List, Optional, TypedDict class CommandError(TypedDict, total=False): @@ -51,3 +51,95 @@ class CheckResult(BaseResult, total=False): findings: List[CheckFinding] warnings: List[Dict[str, Any]] errors: List[CommandError] + + +class PreflightResult(BaseResult): + python_version: str + git_available: bool + existing_adr_directory: Optional[str] + warnings: List[Dict[str, Any]] + errors: List[Dict[str, Any]] + + +class DiscoverResult(BaseResult): + root: str + dependencies: List[Dict[str, Any]] + warnings: List[Dict[str, Any]] + + +class InitResult(BaseResult, total=False): + dry_run: bool + created: List[str] + would_create: List[str] + errors: List[Dict[str, Any]] + + +class IndexResult(BaseResult, total=False): + count: int + path: str + warnings: List[Dict[str, Any]] + errors: List[Dict[str, Any]] + + +class RelatedResult(BaseResult): + count: int + matches: List[Dict[str, Any]] + warnings: List[Dict[str, Any]] + + +class SignificanceResult(BaseResult, total=False): + total: int + classification: str + errors: List[Dict[str, Any]] + + +class ValidateResult(BaseResult): + checked: int + errors: List[Dict[str, Any]] + + +class StatusResult(BaseResult, total=False): + dry_run: bool + would_update: str + updated: str + to: str + errors: List[Dict[str, Any]] + + +class SupersedeResult(BaseResult, total=False): + dry_run: bool + would_update: List[str] + old: str + new: str + errors: List[Dict[str, Any]] + + +class DiffResult(BaseResult, total=False): + mode: str + ref: Optional[str] + files: List[Dict[str, Any]] + errors: List[Dict[str, Any]] + + +class ExceptionResult(BaseResult, total=False): + dry_run: bool + created: str + would_create: str + id: str + errors: List[Dict[str, Any]] + + +class GraphResult(BaseResult, total=False): + count: int + outputs: List[str] + warnings: List[Dict[str, Any]] + errors: List[Dict[str, Any]] + + +class SearchResult(BaseResult): + query: Dict[str, Any] + count: int + total: int + truncated: bool + results: List[Dict[str, Any]] + warnings: List[Dict[str, Any]] diff --git a/tests/unit/test_contracts.py b/tests/unit/test_contracts.py index 6e65aad..1ae6317 100644 --- a/tests/unit/test_contracts.py +++ b/tests/unit/test_contracts.py @@ -1,12 +1,36 @@ """Tests that core/contracts.py's TypedDicts describe real command output shapes (docs/adr-toolkit-audit-report.md §2.4 4.1).""" +import json import subprocess from types import SimpleNamespace -from scripts.commands import check, create +from scripts.commands import ( + check, + create, + diff, + discover, + exception, + graph, + index, + init, + preflight, + related, + search, + significance, + status, + supersede, + validate, +) from scripts.core import contracts +def _assert_keys_subset(result, contract, label): + contract_keys = set(contract.__annotations__) + assert set(result.keys()) <= contract_keys, ( + f"{label} returned keys not in {contract.__name__}: {set(result.keys()) - contract_keys}" + ) + + def test_create_dry_run_result_matches_contract_keys(tmp_path): draft_path = tmp_path / "draft.json" draft_path.write_text( @@ -34,8 +58,142 @@ def test_check_result_with_no_adrs_matches_contract_keys(tmp_path): result = check.run(SimpleNamespace(root=str(tmp_path), dir=str(adr_dir), staged=False, since=None)) assert result["ok"] is True - contract_keys = set(contracts.CheckResult.__annotations__) - assert set(result.keys()) <= contract_keys, ( - f"check.run() returned keys not in contracts.CheckResult: " - f"{set(result.keys()) - contract_keys}" - ) + _assert_keys_subset(result, contracts.CheckResult, "check.run()") + + +def test_preflight_result_matches_contract_keys(tmp_path): + result = preflight.run(SimpleNamespace(root=str(tmp_path))) + assert result["ok"] is True + _assert_keys_subset(result, contracts.PreflightResult, "preflight.run()") + + +def test_discover_result_matches_contract_keys(tmp_path): + result = discover.run(SimpleNamespace(root=str(tmp_path))) + assert result["ok"] is True + _assert_keys_subset(result, contracts.DiscoverResult, "discover.run()") + + +def test_init_dry_run_result_matches_contract_keys(tmp_path): + result = init.run(SimpleNamespace(dir="docs/decisions", root=str(tmp_path), locale=None, dry_run=True)) + assert result["ok"] is True + _assert_keys_subset(result, contracts.InitResult, "init.run()") + + +def test_index_result_matches_contract_keys(tmp_path): + adr_dir = tmp_path / "docs" / "decisions" + adr_dir.mkdir(parents=True) + result = index.run(SimpleNamespace(dir=str(adr_dir), root=str(tmp_path), locale=None)) + assert result["ok"] is True + _assert_keys_subset(result, contracts.IndexResult, "index.run()") + + +def test_related_result_matches_contract_keys(tmp_path): + adr_dir = tmp_path / "docs" / "decisions" + adr_dir.mkdir(parents=True) + result = related.run(SimpleNamespace(dir=str(adr_dir), paths=None, tags=None, keyword=None)) + assert result["ok"] is True + _assert_keys_subset(result, contracts.RelatedResult, "related.run()") + + +def test_significance_result_matches_contract_keys(tmp_path): + input_path = tmp_path / "scores.json" + input_path.write_text(json.dumps({ + "reversal_cost": 0, "alternatives_considered": 0, "quality_attribute_impact": 0, + "boundary_or_pattern_change": 0, "multi_developer_relevance": 0, + "ops_security_data_impact": 0, "future_rationale_query_likelihood": 0, + }), encoding="utf-8") + result = significance.run(SimpleNamespace(input=str(input_path))) + assert result["ok"] is True + _assert_keys_subset(result, contracts.SignificanceResult, "significance.run()") + + +def test_validate_result_matches_contract_keys(tmp_path): + adr_dir = tmp_path / "docs" / "decisions" + adr_dir.mkdir(parents=True) + result = validate.run(SimpleNamespace(dir=str(adr_dir), root=str(tmp_path))) + assert result["ok"] is True + _assert_keys_subset(result, contracts.ValidateResult, "validate.run()") + + +_STATUS_FIXTURE_ADR = ( + "---\n" + "id: ADR-0001\n" + "title: A decision\n" + "status: proposed\n" + "date: 2026-01-01\n" + "decision_makers: []\n" + "related: []\n" + "affected_paths: []\n" + "tags: []\n" + "retrospective: false\n" + "---\n\nBody.\n" +) + + +def test_status_dry_run_result_matches_contract_keys(tmp_path): + adr_dir = tmp_path / "docs" / "decisions" + adr_dir.mkdir(parents=True) + (adr_dir / "0001-a-decision.md").write_text(_STATUS_FIXTURE_ADR, encoding="utf-8") + result = status.run(SimpleNamespace(adr_number=1, to="accepted", dir=str(adr_dir), dry_run=True)) + assert result["ok"] is True + _assert_keys_subset(result, contracts.StatusResult, "status.run()") + + +_SUPERSEDE_OLD_ADR = ( + "---\nid: ADR-0001\ntitle: Old\nstatus: accepted\ndate: 2026-01-01\n" + "decision_makers: []\nrelated: []\naffected_paths: []\ntags: []\n" + "retrospective: false\n---\n\nBody.\n" +) +_SUPERSEDE_NEW_ADR = ( + "---\nid: ADR-0002\ntitle: New\nstatus: accepted\ndate: 2026-01-02\n" + "decision_makers: []\nrelated: []\naffected_paths: []\ntags: []\n" + "retrospective: false\n---\n\nBody.\n" +) + + +def test_supersede_dry_run_result_matches_contract_keys(tmp_path): + adr_dir = tmp_path / "docs" / "decisions" + adr_dir.mkdir(parents=True) + (adr_dir / "0001-old.md").write_text(_SUPERSEDE_OLD_ADR, encoding="utf-8") + (adr_dir / "0002-new.md").write_text(_SUPERSEDE_NEW_ADR, encoding="utf-8") + result = supersede.run(SimpleNamespace(adr_number=1, by=2, dir=str(adr_dir), dry_run=True)) + assert result["ok"] is True + _assert_keys_subset(result, contracts.SupersedeResult, "supersede.run()") + + +def test_diff_result_matches_contract_keys(tmp_path): + subprocess.run(["git", "init", "-q"], cwd=tmp_path, check=True) + result = diff.run(SimpleNamespace(root=str(tmp_path), staged=False, since=None)) + assert result["ok"] is True + _assert_keys_subset(result, contracts.DiffResult, "diff.run()") + + +def test_exception_dry_run_result_matches_contract_keys(tmp_path): + draft_path = tmp_path / "draft.json" + draft_path.write_text(json.dumps({ + "adr_id": "ADR-0001", "rule_id": "r", "owner": "o", "reason": "r", + "scope": ["src/**"], "expiry": "2099-01-01", + }), encoding="utf-8") + result = exception.run(SimpleNamespace( + input=str(draft_path), dir="docs/decisions", root=str(tmp_path), dry_run=True, + )) + assert result["ok"] is True + _assert_keys_subset(result, contracts.ExceptionResult, "exception.run()") + + +def test_graph_result_matches_contract_keys(tmp_path): + adr_dir = tmp_path / "docs" / "decisions" + adr_dir.mkdir(parents=True) + result = graph.run(SimpleNamespace(dir=str(adr_dir), root=str(tmp_path), format="both", output=None)) + assert result["ok"] is True + _assert_keys_subset(result, contracts.GraphResult, "graph.run()") + + +def test_search_result_matches_contract_keys(tmp_path): + adr_dir = tmp_path / "docs" / "decisions" + adr_dir.mkdir(parents=True) + result = search.run(SimpleNamespace( + dir=str(adr_dir), id=None, keyword=None, tags=None, status=None, path=None, limit=None, + )) + assert result["ok"] is True + _assert_keys_subset(result, contracts.SearchResult, "search.run()") From 5859f99aa9bffef66fd1486ef57d783217193dc4 Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 11:11:08 +0900 Subject: [PATCH 33/58] docs: close out output contract coverage item improvements.md's Medium section is now down to exactly one item (the declined parsing-cache) -- nothing else remains open in this worktree's scope. handoff.md records the full-coverage extension and corrects its stale "future work" note about contracts.py. --- changelog.md | 1 + handoff.md | 42 ++++++++++++++++++++++++------------------ improvements.md | 3 --- 3 files changed, 25 insertions(+), 21 deletions(-) diff --git a/changelog.md b/changelog.md index 4964d22..29bc77b 100644 --- a/changelog.md +++ b/changelog.md @@ -4,6 +4,7 @@ Lightweight human-readable summary of meaningful repository changes. ## Unreleased +- `core/contracts.py` now covers all 16 commands' output shapes (was 2). - `PathEscapesRootError` (added in the prior session's path-escape fix) is now caught at all 7 call sites and reported as a structured `PATH_ESCAPES_ROOT` error instead of falling through to a generic diff --git a/handoff.md b/handoff.md index 1ce64ff..c9d0cf3 100644 --- a/handoff.md +++ b/handoff.md @@ -27,17 +27,22 @@ High pass left behind (all 7 `resolve_from_root` call sites now catch it), `e3d592b` schema-drift detection test (no `jsonschema` dependency added -- see rationale in that commit), `2933575` extended `contracts.py` to cover CHECK, `d7368f6` bulk-ADR performance sanity -check, `c3ed01d` TTY-only stderr summary line, (this commit) docs -closeout. +check, `c3ed01d` TTY-only stderr summary line, `77ce206` docs closeout. + +**Follow-up** (owner asked to continue per `improvements.md`/`handoff.md` +after the Medium pass): `1df6066` extended `core/contracts.py` to cover +the remaining 14 commands (was 2/16, now 16/16) -- each shape read from +the actual `run()` return statements, not guessed, and spot-checked +against real error-path output for `status`/`supersede` too. All 3 plan files are gitignored by convention (`docs/superpowers/plans/`) but still on disk in this worktree. `improvements.md` now has: an empty `### Critical` section, a `### High` section containing only the 2 items explicitly deferred to another -worktree, and a `### Medium` section with one item marked **declined with -rationale** (parsing-result caching -- see below) and one marked -partially done (output contract schema, 2 of 16 commands covered). +worktree, and a `### Medium` section with exactly one item -- the +parsing-result cache, marked **declined with rationale** (see below). +Nothing else is open in this worktree's scope. **One Medium item was declined, not silently skipped:** the audit's `functools.lru_cache` suggestion for parsing-result caching provides zero @@ -91,19 +96,21 @@ Excluded here, being handled elsewhere -- do not touch: ## Next step -Nothing is currently in flight. `improvements.md`'s remaining backlog -- -2 other-worktree-flagged High items, plus 1 declined and 1 -partially-done Medium item -- is unscheduled. **Ask the owner before -starting any of it.** Every scope decision across all 3 passes -(Critical-then-High-then-Medium ordering, domain 1/5 exclusion, the -other-worktree exclusions, the parsing-cache decline) was the owner's +Nothing is currently in flight, and nothing is left unscheduled inside +this worktree's scope -- `improvements.md`'s only remaining Open items +are the 2 explicitly deferred to another worktree. If a future session is +asked to "continue per improvements.md/handoff.md" again and finds +nothing left there, that is the correct, complete state -- don't invent +new work; ask the owner what's next. Every scope decision across all +passes (Critical-then-High-then-Medium ordering, domain 1/5 exclusion, +the other-worktree exclusions, the parsing-cache decline) was the owner's explicit call or a judgment call made and explained in-session, not something derivable from the audit report alone. ## Verification -Full suite: `python3 -m pytest tests/unit tests/integration -v` -> 452 -passed as of commit `c3ed01d` (451 on Windows, where the SIGKILL chaos +Full suite: `python3 -m pytest tests/unit tests/integration -v` -> 465 +passed as of commit `1df6066` (464 on Windows, where the SIGKILL chaos test is skipped). CI now also runs a `type-check` job (`mypy --strict`) and gates the @@ -121,11 +128,10 @@ CI now also runs a `type-check` job (`mypy --strict`) and gates the `.adr-toolkit.lock` (0-byte dotfile) permanently inside `docs/decisions/` and `docs/decisions/exceptions/` -- intentional (the cross-process mutex), doesn't match `*.md`/`*.json` globs. -- `core/contracts.py` covers only `CreateResult` and `CheckResult`; - extending to the other 14 commands, and extending `mypy --strict` - beyond the fully-typed core modules into the command modules - themselves (blocked on typing `argparse.Namespace` args), is future - work. +- `core/contracts.py` now covers all 16 commands' result shapes, but + extending `mypy --strict` beyond the fully-typed core modules into the + command modules themselves (blocked on typing `argparse.Namespace` + args) is still future work. - (carried over from the audit, still true) CHECK deliberately cannot prove prose, business rationale, or organizational claims. - (carried over, still true) GitHub branch/tag protection is unavailable diff --git a/improvements.md b/improvements.md index 0c9a5dd..8dfb35f 100644 --- a/improvements.md +++ b/improvements.md @@ -34,9 +34,6 @@ also touch. 진짜 도움이 되려면 mtime 키 영속 캐시가 필요한데, 이는 staleness 리스크 대비 ADR 실사용 규모(수백 개 미만, 감사 보고서 자체 진단)에 비해 과한 투자. (감사 보고서 §2.3 3.2) -- [ ] **출력 계약 스키마 고정(골든 파일)** — `CreateResult`, `CheckResult` - 두 개만 커버됨. 나머지 14개 커맨드는 여전히 미착수. (감사 보고서 §2.1 - 1.3) ## Done From 8f7cd46f33234b3c1522967c5ae9e4bb62c923fc Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 11:29:31 +0900 Subject: [PATCH 34/58] docs: populate cross-session handoff summary before switching sessions improvements.md's Done section is filled in (deviating from its usual "stays empty" convention) with a full summary of every Critical/High/ Medium item shipped this session, at the owner's explicit request ahead of moving to a new session. handoff.md's Next step section is rewritten as an explicit checklist for a cold-start session: no queued work exists, the 2 remaining Open items belong to a different worktree, and prior deferred decisions (branch finish, parsing-cache decline) should be re-asked rather than assumed. --- handoff.md | 45 +++++++++++++++++++++++++++++++++------------ improvements.md | 45 ++++++++++++++++++++++++++++++++++++++++++--- 2 files changed, 75 insertions(+), 15 deletions(-) diff --git a/handoff.md b/handoff.md index c9d0cf3..44db62d 100644 --- a/handoff.md +++ b/handoff.md @@ -94,18 +94,39 @@ Excluded here, being handled elsewhere -- do not touch: another worktree. Every fix across all 3 passes that touched adapter or generator code was a code fix, not README prose. -## Next step - -Nothing is currently in flight, and nothing is left unscheduled inside -this worktree's scope -- `improvements.md`'s only remaining Open items -are the 2 explicitly deferred to another worktree. If a future session is -asked to "continue per improvements.md/handoff.md" again and finds -nothing left there, that is the correct, complete state -- don't invent -new work; ask the owner what's next. Every scope decision across all -passes (Critical-then-High-then-Medium ordering, domain 1/5 exclusion, -the other-worktree exclusions, the parsing-cache decline) was the owner's -explicit call or a judgment call made and explained in-session, not -something derivable from the audit report alone. +## Next step (for a new session picking this up cold) + +**Nothing is currently in flight.** Working tree is clean, everything is +committed on `feature/analyzing-adr-toolkit`, nothing pushed/PR'd. +`improvements.md`'s `## Open` section has exactly 2 items left, both +explicitly flagged `(다른 워크트리 확인)` -- they belong to a *different* +worktree (automatic version sync), not this one. Do not start them here. + +So, concretely, for this worktree: + +1. There is no queued task. Do not invent one. +2. If the user says "continue" / "다음 작업 진행해줘" without naming a + task: tell them the in-scope backlog is empty, and ask what they want + next (new feature? merge/PR this branch? something outside the audit + report entirely?) -- do not restart already-declined work + (parsing-result cache) or reach into another worktree's items without + being told to. +3. If the user wants to finish this branch (merge to `develop` / open a + PR): that decision was deferred every time it came up this session + (owner chose "keep as-is" each time) -- ask again fresh, don't assume + the answer carried forward. +4. If the user references a new audit finding or a fresh problem: that's + genuinely new work -- use the same pattern this session established + (writing-plans -> executing-plans, TDD, one commit per task, verify + real test/mypy output before each commit) rather than skipping + straight to edits. + +Every scope decision across all passes (Critical-then-High-then-Medium +ordering, domain 1/5 exclusion, the other-worktree exclusions, the +parsing-cache decline) was the owner's explicit call or a judgment call +made and explained in-session, not something derivable from the audit +report alone -- see `improvements.md`'s `## Done` for the full rationale +on each. ## Verification diff --git a/improvements.md b/improvements.md index 8dfb35f..d7949fe 100644 --- a/improvements.md +++ b/improvements.md @@ -37,6 +37,45 @@ also touch. ## Done -Resolved items are recorded in `changelog.md` (what shipped) and git history -(exactly how) rather than kept here — this section stays empty between -sessions. +Normally this section stays empty between sessions (resolved items live in +`changelog.md` + git history instead). Populated once here as a +cross-session handoff summary at the owner's explicit request — clear this +back out next time a session does routine cleanup, per the usual rule. + +All of it is on branch `feature/analyzing-adr-toolkit`, not merged/PR'd +yet (owner's explicit choice: keep as-is). Full detail, code, and +rationale for every item lives in `docs/adr-toolkit-audit-report.md` and +the 3 (gitignored) plan files under `docs/superpowers/plans/2026-09-01-*`. + +**Critical** — atomic writes + directory locking (`core/atomic_io.py`, +wired into create/exception/supersede so concurrent invocations can't +duplicate ADR/exception IDs or corrupt files); ReDoS timeout guard on +CHECK's author-supplied regex patterns; Markdown link-injection escape in +the generated `docs/decisions/README.md`; structured stderr logging with +correlation IDs (`core/telemetry.py`). + +**High** — `--dir`/`--root` path-escape guard (`PathEscapesRootError`); +CI branch-coverage gate at 85% (measured baseline: 93.32%); `mypy --strict` +CI gate + `core/contracts.py` (typed result shapes); `adr.py --diagnostic` +timing flag; OS-level (fork+SIGKILL) proof that a mid-write crash never +tears an ADR file; shared adapter-manifest validator +(`scripts/adapter_sdk.py`) used by all 4 manifest-based harness adapters. + +**Medium** — common `AdrToolkitError` base class for all 6 domain +exceptions (also closed a gap: `PathEscapesRootError` was raised but never +actually caught at any of its 7 call sites until this pass); schema-drift +regression test between `schemas/*.json` and the runtime validators (no +`jsonschema` dependency added, by design); bulk-ADR (200 fixtures) +performance sanity check; TTY-only human summary line on stderr +(`ADR_TOOLKIT_NO_COLOR` to suppress); `core/contracts.py` extended from +2/16 to 16/16 commands. + +**Declined, not done** — parsing-result caching (`functools.lru_cache`): +this CLI is a fresh process per invocation, so an in-process cache can't +reduce the actual cross-invocation re-parsing the audit worried about, and +no single command re-parses a file more than once internally either. Left +as a Critical-domain note in `docs/adr-toolkit-audit-report.md`, not +reopened without a real usage signal that changes this analysis. + +Test suite: 395 → 465 passing, zero regressions. CI gained a `type-check` +job and an 85% coverage gate. From a7f2d3b268662cadda20c61b880c03657169bb68 Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 13:17:17 +0900 Subject: [PATCH 35/58] docs: add Low-priority tier to improvements.md from enterprise-adoption.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sourced from docs/enterprise-adoption.md §4/§6-9 (a separate governance/ adoption-maturity report, distinct from the code/architecture audit). Three of the four items are precondition-gated on real-world facts (repo going public, 2+ qualified maintainers, 2+ repositories existing) rather than blocked by missing code -- flagged accordingly so a future session doesn't try to "implement" a GitHub ruleset change or multi-repo tooling against a single private repo. The fourth (adoption-metrics collection from existing ADR/exception frontmatter) has no such precondition and is flagged as the one actually startable item in this tier. Also updated the Done section's branch/test-count notes to reflect the origin/develop merge completed this session. --- improvements.md | 25 ++++++++++++++++++++----- 1 file changed, 20 insertions(+), 5 deletions(-) diff --git a/improvements.md b/improvements.md index d7949fe..e5966a8 100644 --- a/improvements.md +++ b/improvements.md @@ -35,6 +35,15 @@ also touch. 대비 ADR 실사용 규모(수백 개 미만, 감사 보고서 자체 진단)에 비해 과한 투자. (감사 보고서 §2.3 3.2) +### Low + +`docs/enterprise-adoption.md` §4/§6-9 기반 — 이건 `docs/adr-toolkit-audit-report.md`(코드/아키텍처 감사)와는 별개의, 조직 도입·거버넌스 성숙도를 다루는 문서다. 아래 항목 대부분은 코드로 "구현"할 수 있는 게 아니라 실제 세계의 전제조건(저장소 public 전환, 유지관리자 인원, 저장소 개수)에 막혀 있으니, 시작 전에 전제조건부터 확인할 것. + +- [ ] *(전제조건: 저장소 public 전환)* **Public 전환 게이트 실제 적용** — PR template/`CONTRIBUTING.md`/`SECURITY.md`는 이미 존재함(v0.2.1에 포함, `origin/develop` 병합으로 확인). 남은 건 `master`/`develop`/`v*` 태그에 대한 실제 GitHub ruleset(PR 필수, required CI, conversation resolution, force-push/삭제 차단) 적용과 API로 실제 상태 재조회뿐 — 코드 작업이 아니라 저장소를 public 전환한 뒤 GitHub 설정/API에서 해야 하는 작업. `project_v1_public_release_plan` 메모리 참고(1.0.0 시점 public 전환 계획). (enterprise-adoption.md §4, §9) +- [ ] *(전제조건: qualified maintainer 2명 이상)* **CODEOWNERS 독립 승인 활성화** — 현재 1인 운영 상태에서 필수 code-owner review를 켜면 운영을 막거나 형식적 self-review만 만든다고 보고서 자체가 명시적으로 경고함. 인원 조건 충족 전엔 시작하지 않음. (enterprise-adoption.md §4, §9 "지금 구현하지 않을 것") +- [ ] *(전제조건: 저장소 2개 이상)* **조직 단위 ruleset/reusable workflow/audit export/taxonomy** — 여러 저장소가 같은 운영 문제를 반복할 때 설계 시작. 지금은 저장소가 1개뿐이라 시작 조건 미충족. (enterprise-adoption.md §6, §8 항목 5) +- [ ] **도입 지표(adoption metrics) 수집 스크립트** — decision lead time, exception age, unresolved violations 같은 지표는 이미 존재하는 ADR frontmatter(`date`, `status`)와 exception JSON(`created`, `expiry`)만으로 계산 가능해 public 전환이나 멀티레포 없이도 지금 시작할 수 있음(이 Low 섹션에서 유일하게 전제조건이 없는 항목). 다만 "이 지표를 수집한다는 사실만으로 성숙도가 올라가지 않는다"는 보고서 자체의 경고를 유념 — 지표 정의 버전 관리, 실제 운영 개선 연결까지 되어야 의미가 있음. (enterprise-adoption.md §7) + ## Done Normally this section stays empty between sessions (resolved items live in @@ -43,9 +52,13 @@ cross-session handoff summary at the owner's explicit request — clear this back out next time a session does routine cleanup, per the usual rule. All of it is on branch `feature/analyzing-adr-toolkit`, not merged/PR'd -yet (owner's explicit choice: keep as-is). Full detail, code, and -rationale for every item lives in `docs/adr-toolkit-audit-report.md` and -the 3 (gitignored) plan files under `docs/superpowers/plans/2026-09-01-*`. +into `develop` yet (owner's explicit choice: keep as-is). `origin/develop` +was merged **into** this branch (not the other way around) to pick up its +`v0.2.1` release, Antigravity plugin work, and CI additions — see +`handoff.md` for the 3-file conflict resolution. Full detail, code, and +rationale for every hardening item lives in +`docs/adr-toolkit-audit-report.md` and the 3 (gitignored) plan files under +`docs/superpowers/plans/2026-09-01-*`. **Critical** — atomic writes + directory locking (`core/atomic_io.py`, wired into create/exception/supersede so concurrent invocations can't @@ -77,5 +90,7 @@ no single command re-parses a file more than once internally either. Left as a Critical-domain note in `docs/adr-toolkit-audit-report.md`, not reopened without a real usage signal that changes this analysis. -Test suite: 395 → 465 passing, zero regressions. CI gained a `type-check` -job and an 85% coverage gate. +Test suite: 395 → 465 passing (this branch's own work), zero regressions; +469 passing after merging `origin/develop`'s own new tests in. CI gained +a `type-check` job and an 85% coverage gate (this branch), plus +`examples-drift` and `pr-title-check` jobs (from `origin/develop`). From c46b2e6be6f2140bcee6bf779a9e99e486d193d0 Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 13:58:45 +0900 Subject: [PATCH 36/58] docs: add audit-report Low-risk items to improvements.md (corrects earlier miss) The prior "Low priority" pass only pulled from docs/enterprise-adoption.md; the actual ask was to also mine docs/adr-toolkit-audit-report.md's own Low-risk findings. Reviewed all 8 of that report's Low badges: 4 were "no action needed" or already resolved (6.1, 8.3, 8.4's PR-title-check is now the merged-in pr-title-check job, 5.3's README-escaping inconsistency was fixed by this session's safe_md_link_text work) and are noted as such; the remaining 4 real gaps are now in improvements.md's ### Low section alongside (separately sourced and labeled) the enterprise-adoption.md items: CODEOWNERS doc for constraints: blocks, a trivial proposed->deprecated lifecycle transition, moving CHECK's constraint lint earlier to CREATE/STATUS time, and folding Antigravity into harness-parity once agy gets a public registry (blocked, same as the other-worktree agy work). --- handoff.md | 18 +++++++++++++++--- improvements.md | 35 +++++++++++++++++++++++++++++++++-- 2 files changed, 48 insertions(+), 5 deletions(-) diff --git a/handoff.md b/handoff.md index 2b4a1a4..057477c 100644 --- a/handoff.md +++ b/handoff.md @@ -65,9 +65,21 @@ but still on disk in this worktree. section containing only the 2 items explicitly deferred to another worktree, and a `### Medium` section with exactly one item -- the parsing-result cache, marked **declined with rationale** (see below). It -also has a **Low-priority tier**, added at the owner's request, pulled -from `docs/enterprise-adoption.md` §8 -- see that section below for why -most of those items are precondition-gated rather than pure code tasks. +also has a **Low-priority tier with two sourced sub-groups**, added at +the owner's request: +- 4 items pulled from `docs/adr-toolkit-audit-report.md`'s own 🟢 Low-risk + findings (the ones that actually still need action -- most of the 8 + Low findings in that report were "no action needed" or already resolved + by this session's work or the `origin/develop` merge, e.g. its + Conventional-Commits PR title suggestion is now the merged-in + `pr-title-check` CI job). +- 4 items pulled from `docs/enterprise-adoption.md` §8 (a separate + governance/adoption-maturity report) -- see that file's notes below for + why most of those are precondition-gated rather than pure code tasks. + +(An earlier pass in this session mistakenly added only the +enterprise-adoption.md half when asked to pull Low items "from the +report" -- corrected once the ambiguity was pointed out.) **One Medium item was declined, not silently skipped:** the audit's `functools.lru_cache` suggestion for parsing-result caching provides zero diff --git a/improvements.md b/improvements.md index e5966a8..3be8f06 100644 --- a/improvements.md +++ b/improvements.md @@ -37,12 +37,43 @@ also touch. ### Low -`docs/enterprise-adoption.md` §4/§6-9 기반 — 이건 `docs/adr-toolkit-audit-report.md`(코드/아키텍처 감사)와는 별개의, 조직 도입·거버넌스 성숙도를 다루는 문서다. 아래 항목 대부분은 코드로 "구현"할 수 있는 게 아니라 실제 세계의 전제조건(저장소 public 전환, 유지관리자 인원, 저장소 개수)에 막혀 있으니, 시작 전에 전제조건부터 확인할 것. +두 개의 서로 다른 출처가 섞여 있어 각 항목에 출처를 명시했다. + +**출처: `docs/adr-toolkit-audit-report.md`의 🟢 Low 리스크 항목 8개 중, +"추가 조치 불요"이거나 이번 세션/`origin/develop` 병합으로 이미 해소된 +것(1.3 관련 없음, 6.1, 8.3, 8.4 PR 제목 체크는 `pr-title-check` job으로 +이미 존재, 5.3의 README 이스케이프 비일관성은 이번 세션 `safe_md_link_text` +작업으로 이미 해소)을 제외하고, 실제로 아직 안 한 것만 남긴 4건:** + +- [ ] *(전제조건: Antigravity CLI가 공개 패키지 레지스트리 지원)* + **harness-parity CI에 Antigravity 편입** — 현재 Codex/Gemini만 CI에서 + 실제 설치까지 검증하고 Antigravity(`agy`)는 README에 "수동 검증"으로 + 명시됨. `agy` 작업 자체는 다른 브랜치 소관이라 이 항목도 그쪽과 함께 + 검토. (감사 보고서 §2.1 1.2) +- [ ] **CODEOWNERS로 `constraints:` 블록 보호 문서화** — 격리 계층 대신 + 프로세스 통제로: ADR 디렉터리의 `constraints:` 블록을 승인 권한자만 + 병합하도록 `CONTRIBUTING.md`에 명문화. 코드 변경 없이 문서 한 줄이면 + 됨 — 지금 바로 가능. (감사 보고서 §2.2 2.1) +- [ ] **`proposed → deprecated` 전이 추가** — `core/lifecycle.py`의 + `ALLOWED_TRANSITIONS["proposed"]`에 `"deprecated"` 한 줄 추가 + + `test_lifecycle.py`에 파라미터 1건 추가. 합의 없이 제안을 철회하는 + 실무 케이스 지원. 아키텍처 변경 불요, 지금 바로 가능. (감사 보고서 + §2.5 5.1) +- [ ] **CHECK 사전 린트를 CREATE/STATUS 시점으로 앞당기기** — 현재는 + `constraints:` 블록 오탈자(ReDoS 포함)가 CHECK 실행 시점에야 발견됨. + `validate.py`에 이미 있는 파싱 흐름을 `create.py`/`status.py` 종료 + 직전에도 실행해 ADR 작성 시점에 조기 경고. (감사 보고서 §2.5 5.2) + +**출처: `docs/enterprise-adoption.md` §4/§6-9** — 코드/아키텍처 감사와는 +별개의, 조직 도입·거버넌스 성숙도를 다루는 문서. 아래 항목 대부분은 +코드로 "구현"할 수 있는 게 아니라 실제 세계의 전제조건(저장소 public +전환, 유지관리자 인원, 저장소 개수)에 막혀 있으니, 시작 전에 +전제조건부터 확인할 것. - [ ] *(전제조건: 저장소 public 전환)* **Public 전환 게이트 실제 적용** — PR template/`CONTRIBUTING.md`/`SECURITY.md`는 이미 존재함(v0.2.1에 포함, `origin/develop` 병합으로 확인). 남은 건 `master`/`develop`/`v*` 태그에 대한 실제 GitHub ruleset(PR 필수, required CI, conversation resolution, force-push/삭제 차단) 적용과 API로 실제 상태 재조회뿐 — 코드 작업이 아니라 저장소를 public 전환한 뒤 GitHub 설정/API에서 해야 하는 작업. `project_v1_public_release_plan` 메모리 참고(1.0.0 시점 public 전환 계획). (enterprise-adoption.md §4, §9) - [ ] *(전제조건: qualified maintainer 2명 이상)* **CODEOWNERS 독립 승인 활성화** — 현재 1인 운영 상태에서 필수 code-owner review를 켜면 운영을 막거나 형식적 self-review만 만든다고 보고서 자체가 명시적으로 경고함. 인원 조건 충족 전엔 시작하지 않음. (enterprise-adoption.md §4, §9 "지금 구현하지 않을 것") - [ ] *(전제조건: 저장소 2개 이상)* **조직 단위 ruleset/reusable workflow/audit export/taxonomy** — 여러 저장소가 같은 운영 문제를 반복할 때 설계 시작. 지금은 저장소가 1개뿐이라 시작 조건 미충족. (enterprise-adoption.md §6, §8 항목 5) -- [ ] **도입 지표(adoption metrics) 수집 스크립트** — decision lead time, exception age, unresolved violations 같은 지표는 이미 존재하는 ADR frontmatter(`date`, `status`)와 exception JSON(`created`, `expiry`)만으로 계산 가능해 public 전환이나 멀티레포 없이도 지금 시작할 수 있음(이 Low 섹션에서 유일하게 전제조건이 없는 항목). 다만 "이 지표를 수집한다는 사실만으로 성숙도가 올라가지 않는다"는 보고서 자체의 경고를 유념 — 지표 정의 버전 관리, 실제 운영 개선 연결까지 되어야 의미가 있음. (enterprise-adoption.md §7) +- [ ] **도입 지표(adoption metrics) 수집 스크립트** — decision lead time, exception age, unresolved violations 같은 지표는 이미 존재하는 ADR frontmatter(`date`, `status`)와 exception JSON(`created`, `expiry`)만으로 계산 가능해 public 전환이나 멀티레포 없이도 지금 시작할 수 있음. 다만 "이 지표를 수집한다는 사실만으로 성숙도가 올라가지 않는다"는 보고서 자체의 경고를 유념 — 지표 정의 버전 관리, 실제 운영 개선 연결까지 되어야 의미가 있음. (enterprise-adoption.md §7) ## Done From 0307a1c1d93c447a7b02b754ffb42696240f0509 Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 14:07:57 +0900 Subject: [PATCH 37/58] feat: allow proposed->deprecated transition; document constraints: block review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - core/lifecycle.py: ALLOWED_TRANSITIONS["proposed"] now includes "deprecated", supporting withdrawing a proposal that never reached consensus (previously only accepted/rejected were reachable from proposed). docs/adr-toolkit-audit-report.md §2.5 5.1. - CONTRIBUTING.md: documents that a constraints: block change needs sign-off from someone with authority over its affected_paths, since it's enforced policy text, not prose -- a process control in place of code sandboxing, per §2.2 2.1's reasoning (no third-party code executes here, so isolation isn't the applicable defense). --- CONTRIBUTING.md | 13 +++++++++++++ skills/adr-toolkit/scripts/core/lifecycle.py | 2 +- tests/unit/test_lifecycle.py | 6 ++++++ 3 files changed, 20 insertions(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 817e529..c037e64 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -54,6 +54,19 @@ python3 skills/adr-toolkit/scripts/adr.py check --uncommitted --dir docs/decisio - Accepted ADRs are append-only in spirit: use supersession for changed decisions instead of rewriting history. +## `constraints:` Block Review + +An ADR's `constraints:` block is enforced by `adr.py check` against every +future diff, so a change to one is a change to the repository's policy +surface, not just prose. Until CODEOWNERS-backed independent review is +active (see "Public Repository Hygiene" below), a PR that adds or edits a +`constraints:` block requires sign-off from someone with authority over +the affected `affected_paths`, in addition to normal review. This is a +process control, not a code sandbox -- see +`docs/adr-toolkit-audit-report.md` §2.2 2.1 for why isolation isn't the +right defense here (no third-party code executes; the block is +structured policy text). + ## Public Repository Hygiene Public repository branch/tag protection is expected to enforce PRs, required diff --git a/skills/adr-toolkit/scripts/core/lifecycle.py b/skills/adr-toolkit/scripts/core/lifecycle.py index b3b932b..59aabd8 100644 --- a/skills/adr-toolkit/scripts/core/lifecycle.py +++ b/skills/adr-toolkit/scripts/core/lifecycle.py @@ -4,7 +4,7 @@ STATUSES = {"proposed", "accepted", "rejected", "deprecated", "superseded"} ALLOWED_TRANSITIONS = { - "proposed": {"accepted", "rejected"}, + "proposed": {"accepted", "rejected", "deprecated"}, "accepted": {"deprecated", "superseded"}, "rejected": set(), "deprecated": set(), diff --git a/tests/unit/test_lifecycle.py b/tests/unit/test_lifecycle.py index f88a705..967349c 100644 --- a/tests/unit/test_lifecycle.py +++ b/tests/unit/test_lifecycle.py @@ -11,6 +11,12 @@ def test_proposed_can_become_rejected(): lifecycle.validate_transition("proposed", "rejected") # must not raise +def test_proposed_can_become_deprecated(): + # Withdrawing a proposal without going through accepted first (e.g. no + # consensus was ever reached) -- docs/adr-toolkit-audit-report.md §2.5 5.1. + lifecycle.validate_transition("proposed", "deprecated") # must not raise + + def test_accepted_cannot_go_back_to_proposed(): with pytest.raises(lifecycle.InvalidTransitionError): lifecycle.validate_transition("accepted", "proposed") From 9a44342efb0d1ee8fc9afe33c1a8769df4b60934 Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 14:11:50 +0900 Subject: [PATCH 38/58] feat: lint constraints: blocks at CREATE/STATUS time, not just CHECK time MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit New core/constraints.lint(body) wraps extract_constraints() and returns a BAD_CONSTRAINTS warning instead of raising, so a typo in a constraints: block surfaces as soon as the ADR is authored or accepted, rather than silently going unenforced until CHECK happens to run against it later. - create.py: lints the draft body on every path (dry-run and real write), added to a new "warnings" field alongside the existing response keys. - status.py: lints only on transition to "accepted", since that's the status CHECK actually enforces constraints against -- other transitions return warnings: [] without parsing the body. - core/contracts.py: CreateResult and StatusResult gain the warnings field to match. docs/adr-toolkit-audit-report.md §2.5 5.2. --- skills/adr-toolkit/scripts/commands/create.py | 11 ++++- skills/adr-toolkit/scripts/commands/status.py | 8 ++++ .../adr-toolkit/scripts/core/constraints.py | 12 ++++++ skills/adr-toolkit/scripts/core/contracts.py | 2 + tests/unit/test_constraints.py | 16 ++++++- tests/unit/test_create.py | 43 +++++++++++++++++++ tests/unit/test_status.py | 41 ++++++++++++++++++ 7 files changed, 131 insertions(+), 2 deletions(-) diff --git a/skills/adr-toolkit/scripts/commands/create.py b/skills/adr-toolkit/scripts/commands/create.py index e2363d2..a050b31 100644 --- a/skills/adr-toolkit/scripts/commands/create.py +++ b/skills/adr-toolkit/scripts/commands/create.py @@ -5,6 +5,7 @@ from pathlib import Path from scripts.core import atomic_io +from scripts.core import constraints from scripts.core import frontmatter as fm from scripts.core import identifiers from scripts.core.config import ConfigError, resolve_locale @@ -176,7 +177,14 @@ def run(args) -> dict: "errors": [{"code": "SCHEMA_ERROR", "detail": e} for e in schema_errors], } - return {"ok": True, "operation": "create", "dry_run": True, "would_create": str(target), "id": frontmatter_data["id"]} + return { + "ok": True, + "operation": "create", + "dry_run": True, + "would_create": str(target), + "id": frontmatter_data["id"], + "warnings": constraints.lint(draft["body"]), + } with atomic_io.adr_directory_lock(adr_dir): next_num = identifiers.next_id(adr_dir) @@ -208,4 +216,5 @@ def run(args) -> dict: "dry_run": False, "created": str(target), "id": frontmatter_data["id"], + "warnings": constraints.lint(draft["body"]), } diff --git a/skills/adr-toolkit/scripts/commands/status.py b/skills/adr-toolkit/scripts/commands/status.py index 647f96b..2effe22 100644 --- a/skills/adr-toolkit/scripts/commands/status.py +++ b/skills/adr-toolkit/scripts/commands/status.py @@ -1,6 +1,7 @@ """Change an ADR's status through the deterministic lifecycle state machine.""" from pathlib import Path +from scripts.core import constraints from scripts.core import frontmatter as fm from scripts.core import identifiers from scripts.core.lifecycle import InvalidTransitionError, validate_transition @@ -35,6 +36,11 @@ def run(args) -> dict: "errors": [{"code": "INVALID_TRANSITION", "detail": str(exc)}], } + # Constraints only get enforced by CHECK once an ADR is accepted, so + # that's the one transition where a malformed block would otherwise go + # silently unenforced (docs/adr-toolkit-audit-report.md §2.5 5.2). + warnings = constraints.lint(body) if args.to == "accepted" else [] + if getattr(args, "dry_run", False): return { "ok": True, @@ -42,6 +48,7 @@ def run(args) -> dict: "dry_run": True, "would_update": str(target_file), "to": args.to, + "warnings": warnings, } data["status"] = args.to @@ -55,4 +62,5 @@ def run(args) -> dict: "dry_run": False, "updated": str(target_file), "to": args.to, + "warnings": warnings, } diff --git a/skills/adr-toolkit/scripts/core/constraints.py b/skills/adr-toolkit/scripts/core/constraints.py index f84c91d..953c090 100644 --- a/skills/adr-toolkit/scripts/core/constraints.py +++ b/skills/adr-toolkit/scripts/core/constraints.py @@ -33,6 +33,18 @@ class ConstraintsError(AdrToolkitError): error_code = "BAD_CONSTRAINTS" +def lint(body: str) -> list: + """Best-effort pre-flight check for a malformed constraints: block, so a + typo surfaces at CREATE/STATUS time instead of silently going + unenforced until CHECK runs against it later + (docs/adr-toolkit-audit-report.md §2.5 5.2).""" + try: + extract_constraints(body) + except ConstraintsError as exc: + return [{"code": "BAD_CONSTRAINTS", "detail": str(exc)}] + return [] + + def extract_constraints(body: str) -> list: rules = [] for fence_match in FENCE_RE.finditer(body): diff --git a/skills/adr-toolkit/scripts/core/contracts.py b/skills/adr-toolkit/scripts/core/contracts.py index b5cb279..8b9c21e 100644 --- a/skills/adr-toolkit/scripts/core/contracts.py +++ b/skills/adr-toolkit/scripts/core/contracts.py @@ -30,6 +30,7 @@ class CreateResult(BaseResult, total=False): created: str would_create: str id: str + warnings: List[Dict[str, Any]] errors: List[CommandError] @@ -103,6 +104,7 @@ class StatusResult(BaseResult, total=False): would_update: str updated: str to: str + warnings: List[Dict[str, Any]] errors: List[Dict[str, Any]] diff --git a/tests/unit/test_constraints.py b/tests/unit/test_constraints.py index 39ee026..5ad06cc 100644 --- a/tests/unit/test_constraints.py +++ b/tests/unit/test_constraints.py @@ -1,6 +1,6 @@ import pytest -from scripts.core.constraints import ConstraintsError, extract_constraints +from scripts.core.constraints import ConstraintsError, extract_constraints, lint BODY_WITH_CONSTRAINTS = """# Use a provider port @@ -91,3 +91,17 @@ def test_all_six_known_kinds_are_accepted(): " severity: major\n message: \"m\"\n```\n" ) assert extract_constraints(body)[0]["kind"] == kind + + +def test_lint_returns_no_warnings_for_a_valid_body(): + assert lint(BODY_WITH_CONSTRAINTS) == [] + + +def test_lint_returns_no_warnings_for_a_body_with_no_constraints_block(): + assert lint("# Just prose\n\nNo constraints here.\n") == [] + + +def test_lint_returns_a_bad_constraints_warning_for_a_malformed_block(): + warnings = lint(BODY_WITH_UNKNOWN_KIND) + assert warnings == [{"code": "BAD_CONSTRAINTS", "detail": warnings[0]["detail"]}] + assert "forbidden_imports" in warnings[0]["detail"] diff --git a/tests/unit/test_create.py b/tests/unit/test_create.py index 0469cb0..552827d 100644 --- a/tests/unit/test_create.py +++ b/tests/unit/test_create.py @@ -223,3 +223,46 @@ def test_conflicting_cli_and_draft_slugs_return_structured_error(tmp_path): assert result["ok"] is False assert result["errors"][0]["code"] == "CONFLICTING_SLUG_INPUT" + + +def test_create_warns_on_malformed_constraints_block(tmp_path): + adr_dir = tmp_path / "docs" / "decisions" + draft_path = _write_draft( + tmp_path, + body=( + "# Decision\n\nBody.\n\n" + "```yaml\nconstraints:\n - id: r\n kind: forbidden_imports\n```\n" + ), + ) + + result = create.run(_args(tmp_path, draft_path, adr_dir)) + + assert result["ok"] is True + assert result["warnings"] == [{ + "code": "BAD_CONSTRAINTS", + "detail": result["warnings"][0]["detail"], + }] + assert "forbidden_imports" in result["warnings"][0]["detail"] + + +def test_create_dry_run_also_warns_on_malformed_constraints_block(tmp_path): + adr_dir = tmp_path / "docs" / "decisions" + draft_path = _write_draft( + tmp_path, + body="# Decision\n\n```yaml\nconstraints:\n - id: r\n kind: bogus\n```\n", + ) + + result = create.run(_args(tmp_path, draft_path, adr_dir, dry_run=True)) + + assert result["ok"] is True + assert result["warnings"][0]["code"] == "BAD_CONSTRAINTS" + + +def test_create_has_no_warnings_for_a_clean_body(tmp_path): + adr_dir = tmp_path / "docs" / "decisions" + draft_path = _write_draft(tmp_path) + + result = create.run(_args(tmp_path, draft_path, adr_dir)) + + assert result["ok"] is True + assert result["warnings"] == [] diff --git a/tests/unit/test_status.py b/tests/unit/test_status.py index c9c8048..ad2608c 100644 --- a/tests/unit/test_status.py +++ b/tests/unit/test_status.py @@ -105,3 +105,44 @@ def test_valid_transition_preserves_parsed_body_whitespace(tmp_path): assert result["ok"] is True after_body = fm.parse(target.read_text(encoding="utf-8"))[1] assert after_body == before_body + + +def test_status_warns_on_malformed_constraints_when_accepting(tmp_path): + adr = ACCEPTED_ADR.replace( + "# Use Kafka\n", + "# Use Kafka\n\n```yaml\nconstraints:\n - id: r\n kind: bogus\n```\n", + ) + (tmp_path / "0001-use-kafka.md").write_text(adr, encoding="utf-8") + + result = status.run( + SimpleNamespace(adr_number=1, to="accepted", dir=str(tmp_path), dry_run=False) + ) + + assert result["ok"] is True + assert result["warnings"][0]["code"] == "BAD_CONSTRAINTS" + + +def test_status_does_not_lint_constraints_for_a_non_accepted_transition(tmp_path): + adr = ACCEPTED_ADR.replace( + "# Use Kafka\n", + "# Use Kafka\n\n```yaml\nconstraints:\n - id: r\n kind: bogus\n```\n", + ) + (tmp_path / "0001-use-kafka.md").write_text(adr, encoding="utf-8") + + result = status.run( + SimpleNamespace(adr_number=1, to="rejected", dir=str(tmp_path), dry_run=False) + ) + + assert result["ok"] is True + assert result["warnings"] == [] + + +def test_status_has_no_warnings_for_a_clean_body_when_accepting(tmp_path): + (tmp_path / "0001-use-kafka.md").write_text(ACCEPTED_ADR, encoding="utf-8") + + result = status.run( + SimpleNamespace(adr_number=1, to="accepted", dir=str(tmp_path), dry_run=False) + ) + + assert result["ok"] is True + assert result["warnings"] == [] From 466ffed07284ad6fb037df31464e1b7854cc9649 Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 14:13:36 +0900 Subject: [PATCH 39/58] docs: close out 3 of 4 audit-report Low items, note parallel Codex work improvements.md's audit-report Low sub-group is down to 1 item (Antigravity/harness-parity, blocked on an external precondition). handoff.md records the new commits, the parallel Codex session working on the adoption-metrics item in this same worktree/branch, and updates the Next-step checklist to reflect that almost nothing remains open in this worktree's own scope. --- handoff.md | 80 +++++++++++++++++++++++++++++++------------------ improvements.md | 35 ++++++++++------------ 2 files changed, 66 insertions(+), 49 deletions(-) diff --git a/handoff.md b/handoff.md index 057477c..6d8b8d5 100644 --- a/handoff.md +++ b/handoff.md @@ -58,6 +58,28 @@ the remaining 14 commands (was 2/16, now 16/16) -- each shape read from the actual `run()` return statements, not guessed, and spot-checked against real error-path output for `status`/`supersede` too. +**Low-priority follow-up** (3 of 4 audit-report Low items): +`0307a1c` allows `proposed -> deprecated` and documents `constraints:` +block review in `CONTRIBUTING.md`; `9a44342` adds `core/constraints.lint()` +and wires it into `create.py` (always) and `status.py` (only on +transition to `accepted`, since that's the status CHECK actually enforces +constraints against) so a typo surfaces at authoring time instead of only +at CHECK time -- `CreateResult`/`StatusResult` gained a `warnings` field +to match. The 4th item (Antigravity in `harness-parity`) stays open, +blocked on `agy` getting a public package registry. + +**Concurrent work (owner's own coordination, not this session's):** the +owner assigned `improvements.md`'s "도입 지표 수집 스크립트" +(adoption-metrics script, from `docs/enterprise-adoption.md` §7) to a +**Codex session running in this same worktree/branch** in parallel with +this session, specifically because it's a new-file-only task with no +overlap against the files this session was touching. If you see an +uncommitted or newly-committed `scripts/adoption_metrics.py` (or +similarly named) plus a matching test file that you don't recognize +authoring, that's Codex's work landing -- don't revert it, and check +`improvements.md`'s enterprise-adoption.md sub-group for whether it's +already been checked off before restarting it. + All 3 plan files are gitignored by convention (`docs/superpowers/plans/`) but still on disk in this worktree. @@ -134,37 +156,37 @@ Excluded here, being handled elsewhere -- do not touch: ## Next step (for a new session picking this up cold) -**Nothing is currently in flight in this worktree's own scope**, but -`improvements.md` now carries a **Low-priority tier** (added this -session from `docs/enterprise-adoption.md` §8) that a future session can -pick up -- see that file's `### Low` section for the exact items and -their preconditions. Concretely: +**Only one small item is truly open in this worktree's own scope** +(the Antigravity/harness-parity one below); everything else left in +`improvements.md` is either someone else's worktree, a precondition-gated +enterprise-adoption item, or in flight in a parallel Codex session (see +above). Concretely: 1. `improvements.md`'s `## Open` → `### High` still has exactly 2 items left, both flagged `(다른 워크트리 확인)` -- still someone else's. Do not start them here. -2. `improvements.md`'s new `### Low` section has items sourced from - `docs/enterprise-adoption.md`. Read that section's notes carefully - before starting any of them: most are **not pure code tasks** -- - they're gated on real-world preconditions (the repository actually - going public, 2+ repositories existing) that no amount of local - editing satisfies. Don't "implement" a GitHub ruleset change by - writing a script that doesn't actually call the GitHub API against a - real public repo, and don't build multi-repo tooling against a - single-repo reality. -3. If the user says "continue" / "다음 작업 진행해줘" without naming a - task: check `improvements.md`'s `### Low` section first (that's the - one open, actionable-albeit-constrained tier); don't restart - already-declined work (parsing-result cache) or reach into another - worktree's High items without being told to. -4. If the user wants to finish this branch (merge to `develop` / open a +2. `improvements.md`'s `### Low` → audit-report sub-group has exactly 1 + item left (Antigravity in `harness-parity`), blocked on an external + fact (agy public registry support) -- don't start it, just note it's + blocked if asked. +3. `improvements.md`'s `### Low` → enterprise-adoption.md sub-group: check + whether the Codex session's adoption-metrics work has landed and been + checked off before assuming it's still open. The other 3 items there + remain precondition-gated (repository going public, 2+ maintainers, + 2+ repositories) -- **not pure code tasks**, don't "implement" a + GitHub ruleset change or multi-repo tooling against a single private + repo's reality. +4. If the user says "continue" / "다음 작업 진행해줘" without naming a + task: at this point the honest answer may be "nothing is open here" -- + say so and ask what's next, rather than inventing scope. +5. If the user wants to finish this branch (merge to `develop` / open a PR): that decision was deferred every time it came up this session (owner chose "keep as-is" each time) -- ask again fresh, don't assume the answer carried forward. Note this branch now includes the merged `origin/develop` history (see above), so a future merge/PR back to `develop` should be a clean fast-forward-friendly merge, not a repeat of this session's conflict resolution. -5. If the user references a new audit finding or a fresh problem: that's +6. If the user references a new audit finding or a fresh problem: that's genuinely new work -- use the same pattern this session established (writing-plans -> executing-plans, TDD, one commit per task, verify real test/mypy output before each commit) rather than skipping @@ -179,14 +201,14 @@ on each. ## Verification -Full suite before the `origin/develop` merge: -`python3 -m pytest tests/unit tests/integration -v` -> 465 passed. Re-run -this after the merge to get the current combined count (develop's new -`tests/integration/test_examples.py` and expanded -`test_antigravity_adapter.py` add more). CI now also runs `type-check` -(`mypy --strict`), `examples-drift` (from develop), and `pr-title-check` -(from develop) jobs alongside the existing `pytest` (now coverage-gated -at 85%), `version-drift`, and `harness-parity` jobs. +`python3 -m pytest tests/unit tests/integration -v` -> 479 passed as of +commit `9a44342` (395 at session start -> 465 before the `origin/develop` +merge -> 469 after merging in develop's own new tests -> 479 after the +Low-priority follow-up work). Re-run to pick up whatever Codex's parallel +adoption-metrics work adds. CI now also runs `type-check` (`mypy +--strict`), `examples-drift` (from develop), and `pr-title-check` (from +develop) jobs alongside the existing `pytest` (now coverage-gated at +85%), `version-drift`, and `harness-parity` jobs. ## Open risks diff --git a/improvements.md b/improvements.md index 3be8f06..8be570c 100644 --- a/improvements.md +++ b/improvements.md @@ -40,29 +40,14 @@ also touch. 두 개의 서로 다른 출처가 섞여 있어 각 항목에 출처를 명시했다. **출처: `docs/adr-toolkit-audit-report.md`의 🟢 Low 리스크 항목 8개 중, -"추가 조치 불요"이거나 이번 세션/`origin/develop` 병합으로 이미 해소된 -것(1.3 관련 없음, 6.1, 8.3, 8.4 PR 제목 체크는 `pr-title-check` job으로 -이미 존재, 5.3의 README 이스케이프 비일관성은 이번 세션 `safe_md_link_text` -작업으로 이미 해소)을 제외하고, 실제로 아직 안 한 것만 남긴 4건:** +"추가 조치 불요"이거나 이미 해소된 것을 제외하고 실제로 안 한 것 4건 +중 3건 완료(`0307a1c`, `9a44342`). 남은 건 1건뿐:** - [ ] *(전제조건: Antigravity CLI가 공개 패키지 레지스트리 지원)* **harness-parity CI에 Antigravity 편입** — 현재 Codex/Gemini만 CI에서 실제 설치까지 검증하고 Antigravity(`agy`)는 README에 "수동 검증"으로 명시됨. `agy` 작업 자체는 다른 브랜치 소관이라 이 항목도 그쪽과 함께 검토. (감사 보고서 §2.1 1.2) -- [ ] **CODEOWNERS로 `constraints:` 블록 보호 문서화** — 격리 계층 대신 - 프로세스 통제로: ADR 디렉터리의 `constraints:` 블록을 승인 권한자만 - 병합하도록 `CONTRIBUTING.md`에 명문화. 코드 변경 없이 문서 한 줄이면 - 됨 — 지금 바로 가능. (감사 보고서 §2.2 2.1) -- [ ] **`proposed → deprecated` 전이 추가** — `core/lifecycle.py`의 - `ALLOWED_TRANSITIONS["proposed"]`에 `"deprecated"` 한 줄 추가 + - `test_lifecycle.py`에 파라미터 1건 추가. 합의 없이 제안을 철회하는 - 실무 케이스 지원. 아키텍처 변경 불요, 지금 바로 가능. (감사 보고서 - §2.5 5.1) -- [ ] **CHECK 사전 린트를 CREATE/STATUS 시점으로 앞당기기** — 현재는 - `constraints:` 블록 오탈자(ReDoS 포함)가 CHECK 실행 시점에야 발견됨. - `validate.py`에 이미 있는 파싱 흐름을 `create.py`/`status.py` 종료 - 직전에도 실행해 ADR 작성 시점에 조기 경고. (감사 보고서 §2.5 5.2) **출처: `docs/enterprise-adoption.md` §4/§6-9** — 코드/아키텍처 감사와는 별개의, 조직 도입·거버넌스 성숙도를 다루는 문서. 아래 항목 대부분은 @@ -121,7 +106,17 @@ no single command re-parses a file more than once internally either. Left as a Critical-domain note in `docs/adr-toolkit-audit-report.md`, not reopened without a real usage signal that changes this analysis. +**Low (audit-report-sourced, 3 of 4)** — `constraints:` block review +documented in `CONTRIBUTING.md`; `proposed → deprecated` lifecycle +transition added; CHECK's constraints-block lint moved earlier to +CREATE/STATUS time (`core/constraints.lint()`, new `warnings` field on +`CreateResult`/`StatusResult`) so a typo surfaces at authoring time +instead of only when CHECK later runs against it. The 4th item +(Antigravity in harness-parity CI) stays open, blocked on `agy` getting a +public package registry. + Test suite: 395 → 465 passing (this branch's own work), zero regressions; -469 passing after merging `origin/develop`'s own new tests in. CI gained -a `type-check` job and an 85% coverage gate (this branch), plus -`examples-drift` and `pr-title-check` jobs (from `origin/develop`). +469 after merging `origin/develop`; 479 after the Low-priority follow-up +work above. CI gained a `type-check` job and an 85% coverage gate (this +branch), plus `examples-drift` and `pr-title-check` jobs (from +`origin/develop`). From 9a0de457d9722c4bc486bcca7187e344edf4a944 Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 14:14:26 +0900 Subject: [PATCH 40/58] docs: design adoption metrics collector --- .../2026-09-01-adoption-metrics-design.md | 304 ++++++++++++++++++ 1 file changed, 304 insertions(+) create mode 100644 docs/superpowers/specs/2026-09-01-adoption-metrics-design.md diff --git a/docs/superpowers/specs/2026-09-01-adoption-metrics-design.md b/docs/superpowers/specs/2026-09-01-adoption-metrics-design.md new file mode 100644 index 0000000..6b74a55 --- /dev/null +++ b/docs/superpowers/specs/2026-09-01-adoption-metrics-design.md @@ -0,0 +1,304 @@ +# Adoption Metrics Collector Design + +## Context + +`docs/enterprise-adoption.md` section 7 defines five operational metrics for +finding workflow bottlenecks and policy debt: decision lead time, review +latency, supersession rate, unresolved violations, and exception age. The ADR +Toolkit already stores the current ADR state in Markdown frontmatter and active +policy exceptions in JSON, but those snapshots do not preserve every event the +definitions require. In particular, a single `date` cannot represent both the +proposal and decision times, CHECK output is not persisted, and a file does not +identify when review was requested or completed. + +The collector must not turn missing evidence into a plausible-looking number. +It will therefore separate metric calculation from evidence collection and +report provenance and coverage with every value. Git is the broadly portable +default history source. GitHub review events improve the common hosted case, +while a provider-neutral JSONL event contract supports GitLab, Bitbucket, +non-Git SCMs, and manually exported audit data. + +## Goals + +- Produce one deterministic JSON document covering all five section 7 metrics. +- Work without GitHub and degrade cleanly when Git or provider data is absent. +- Use local ADR and exception files as the current-state source of truth. +- Reconstruct lifecycle transitions from local Git when explicit events are + absent. +- Use GitHub only to supply review events that local Git cannot represent. +- Accept the same events through JSONL so the calculation layer is not tied to + a hosting provider. +- Make incomplete evidence visible through `available`, `coverage`, `sources`, + and warnings instead of silently estimating missing history. +- Keep all existing ADR command modules unchanged. + +## Non-goals + +- Ranking people, teams, or repositories by productivity. +- Modifying ADR frontmatter, exception files, or Git history. +- Persisting CHECK observations during this first collector iteration. +- Supporting GitLab or Bitbucket APIs directly. Their exporters can target the + provider-neutral event contract. +- Treating an active exception as resolution of a `VIOLATED` finding. Existing + CHECK semantics deliberately annotate rather than hide violations. +- Calculating stale review coverage, which section 7 lists separately but the + requested backlog item does not include. + +## Architecture + +The implementation is a single repository tool, `scripts/adoption_metrics.py`, +split internally into four boundaries: + +1. Local readers parse ADR frontmatter, exception JSON, and optional historical + CHECK observation files without importing the skill-internal `scripts` + package. +2. Collectors normalize explicit JSONL, local Git history, and optional GitHub + review data into provider-neutral events. +3. Pure calculation functions consume current records and normalized events. +4. The CLI validates arguments, assembles evidence in precedence order, and + serializes the result. + +The dependency direction is one-way: + +```text +ADR / exceptions -----------+ +explicit JSONL events ------+--> normalized records --> metric calculators --> JSON +local Git history ----------+ +optional GitHub review data + +``` + +Metric calculators never invoke Git or GitHub. This keeps edge cases testable +without network access and prevents provider-specific fields from becoming part +of the public metrics contract. + +## Command Interface + +```text +python3 scripts/adoption_metrics.py \ + --root . \ + --dir docs/decisions \ + [--since 2026-01-01] \ + [--until 2026-12-31] \ + [--events path/to/events.jsonl] \ + [--github] \ + [--check-results path/to/check-observations.jsonl] \ + --json +``` + +`--root` defaults to the current directory and `--dir` is resolved beneath it. +`--until` defaults to the current UTC date; `--since` defaults to the earliest +available evidence. `--json` is required in v1 so no unstable human-readable +format becomes an accidental contract. + +Local Git collection is attempted when `--root` belongs to a Git repository. +Absence of Git is not an error. `--github` opts into review collection through +the installed and authenticated `gh` CLI; failure becomes a warning unless the +entire requested metric has no other evidence. Explicit `--events` always works +without Git or GitHub. + +All timestamps are parsed as ISO 8601. Date-only inputs represent midnight UTC +for interval filtering and whole-day age calculations. Invalid inputs produce a +non-zero exit and a JSON error object; an individual malformed ADR, exception, +or event becomes a warning while other valid records are still processed. + +## Normalized Events + +Each line of `--events` and `--check-results` is one JSON object with a versioned +envelope: + +```json +{ + "schema_version": 1, + "event": "adr_status_changed", + "occurred_at": "2026-08-30T11:36:40Z", + "adr_id": "ADR-0003", + "from": "accepted", + "to": "superseded", + "source": "git" +} +``` + +Supported event names are: + +| Event | Required payload | Purpose | +| --- | --- | --- | +| `adr_created` | `adr_id`, `status` | Establish first known lifecycle state | +| `adr_status_changed` | `adr_id`, `from`, `to` | Establish decision and supersession transitions | +| `review_requested` | `adr_id`, `reviewer` | Start review latency | +| `review_submitted` | `adr_id`, `reviewer`, `qualified` | End review latency at first qualified review | +| `violation_observed` | `fingerprint`, `adr_id`, `rule_id` | Open or continue a violation | +| `violation_resolved` | `fingerprint`, `adr_id`, `rule_id` | Close a previously observed violation | + +Event identity is the tuple of event type, occurrence time, and its natural +entity key. Duplicate events from explicit input, Git reconstruction, and +GitHub are deduplicated. Evidence precedence is explicit JSONL, then local Git, +then GitHub. A higher-precedence event wins when two sources disagree, and the +conflict is emitted as a warning. + +## Git And GitHub Collection + +The Git collector follows each `docs/decisions/[0-9]*.md` path through history, +parses frontmatter at every content-changing commit, and emits creation and +status-transition events using the commit author timestamp. It does not infer a +`proposed` event for an ADR whose first committed state is `accepted`; that ADR +is excluded from lead-time coverage. Rename following is best effort and emits +a warning if an ADR ID changes. + +The GitHub collector queries pull requests that touched an ADR file during the +requested interval. A review is qualified when it is submitted by a reviewer +who was explicitly requested for that pull request; self-review by the PR author +does not qualify. The first qualified submitted review after the first review +request ends the interval. The normalized event keeps no provider-specific URL +or numeric ID in the calculation path, though diagnostics may report them. + +GitHub collection is an enhancement, not a prerequisite. Repositories on other +providers can export equivalent `review_requested` and `review_submitted` +events. A repository with neither source receives an unavailable review metric, +not a guessed value. + +## Metric Definitions + +### Decision Lead Time + +For each ADR, measure elapsed hours from its earliest observed `proposed` state +to its first transition into `accepted` or `rejected`. Report the median across +decisions completed within `[since, until]`. ADRs first observed in a terminal +state are excluded and reduce coverage. + +### Review Latency + +For each ADR review cycle, measure elapsed hours from the first review request +to the first subsequent qualified review. Report the median for review cycles +completed within the interval. Requests with no qualified review are reported +as open review cycles but are not included in the median. + +### Supersession Rate + +The numerator is the number of ADRs transitioning to `superseded` within the +interval. The denominator is the number of ADRs transitioning to `accepted` +within the same interval. Report a JSON number in the range 0 through 1, or +`null` when the denominator is zero. Also report both raw counts so consumers do +not over-interpret a small sample. + +### Unresolved Violations + +A violation is identified by the stable fingerprint supplied by the CHECK +observation producer. It is open after its latest `violation_observed` event and +closed after a later `violation_resolved` event. At `until`, report the open +count and each open violation's whole-day age from first uninterrupted +observation. If only a current CHECK result exists, count is available but age +is unavailable. Active exceptions remain visible and do not close violations. + +### Exception Age + +For every schema-valid exception, calculate whole days from `created` to +`until`. Report active exception count, median active age, maximum active age, +and count whose `expiry` is before `until`. Expired exceptions are excluded from +active-age aggregates but included in the expired count. + +## Output Contract + +```json +{ + "ok": true, + "operation": "adoption_metrics", + "schema_version": 1, + "period": {"since": "2026-01-01", "until": "2026-12-31"}, + "metrics": { + "decision_lead_time": { + "available": true, + "median_hours": 26.5, + "sample_size": 4, + "coverage": {"eligible": 5, "measured": 4, "ratio": 0.8}, + "sources": ["git"] + }, + "review_latency": { + "available": false, + "median_hours": null, + "sample_size": 0, + "coverage": {"eligible": 0, "measured": 0, "ratio": null}, + "sources": [], + "reason": "No review events were available." + }, + "supersession_rate": { + "available": true, + "rate": 0.25, + "superseded": 1, + "accepted": 4, + "sources": ["git"] + }, + "unresolved_violations": { + "available": true, + "open_count": 2, + "age_available": true, + "median_age_days": 3, + "max_age_days": 5, + "sources": ["events"] + }, + "exception_age": { + "available": true, + "active_count": 2, + "median_age_days": 9.5, + "max_age_days": 12, + "expired_count": 1, + "sources": ["exceptions"] + } + }, + "warnings": [] +} +``` + +Metric objects keep stable names even when unavailable. `available: false` +requires a machine-readable `reason`; numeric values remain `null`. A source is +listed only if it contributed evidence to that metric. Warnings contain stable +codes plus file/event context and never go to stdout outside the JSON document. + +## Error Handling And Safety + +The collector is read-only. It does not rewrite ADRs, exceptions, events, or +Git metadata. Paths are resolved beneath `--root`; path escape is rejected. +Subprocess calls use argument arrays and never invoke a shell. GitHub tokens are +left to `gh` credential management and never appear in output. + +Missing optional sources degrade metric availability. Invalid CLI arguments, +an absent ADR directory, path escape, or failure to serialize the final result +is fatal. Malformed individual records are warnings because discarding every +metric would hide otherwise valid evidence; their omission is reflected in +coverage. + +## Testing Strategy + +`tests/unit/test_adoption_metrics.py` loads the repository-root module with +`importlib.util.spec_from_file_location`, matching +`tests/unit/test_sync_version.py`, because the distributable skill has an +unrelated top-level `scripts` package. + +Tests use temporary ADR directories, synthetic Git repositories, inline JSONL, +and pure normalized GitHub response fixtures. No test requires network access +or a logged-in GitHub account. TDD cycles cover: + +- current ADR and exception parsing, including malformed-record warnings; +- Git lifecycle reconstruction without inventing missing proposed history; +- event deduplication and precedence; +- qualified review selection and open review cycles; +- all five calculations, empty denominators, median behavior, and interval + boundaries; +- partial availability and coverage reporting; +- deterministic CLI JSON and fatal error JSON; +- operation without Git, GitHub, or event history; +- path containment and subprocess argument safety. + +The focused unit file runs first. Final verification runs the full unit and +integration suites plus the repository's strict type check and any formatting +or lint checks configured for scripts. + +## Files And Scope + +Implementation creates: + +- `scripts/adoption_metrics.py` +- `tests/unit/test_adoption_metrics.py` + +This design document and the normal closeout updates to `changelog.md`, +`handoff.md`, and `improvements.md` are the only other expected changes. No +file under `skills/adr-toolkit/scripts/commands/` is modified. From 70e2dd241040ba393eaec9c5b6cef5e3898708cd Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 14:31:40 +0900 Subject: [PATCH 41/58] feat: calculate adoption metrics from normalized events --- scripts/adoption_metrics.py | 309 +++++++++++++++++++++++++ tests/unit/test_adoption_metrics.py | 341 ++++++++++++++++++++++++++++ 2 files changed, 650 insertions(+) create mode 100644 scripts/adoption_metrics.py create mode 100644 tests/unit/test_adoption_metrics.py diff --git a/scripts/adoption_metrics.py b/scripts/adoption_metrics.py new file mode 100644 index 0000000..bdf7966 --- /dev/null +++ b/scripts/adoption_metrics.py @@ -0,0 +1,309 @@ +#!/usr/bin/env python3 +"""Collect provider-neutral ADR adoption metrics as deterministic JSON.""" + +import json +import re +import statistics +from datetime import datetime, timezone +from pathlib import Path +from typing import Any, Dict, List, Tuple + + +FRONTMATTER_RE = re.compile(r"\A---\n(.*?)\n---(?:\n|\Z)", re.DOTALL) +REQUIRED_EXCEPTION_FIELDS = { + "id", + "adr_id", + "rule_id", + "owner", + "reason", + "scope", + "created", + "expiry", +} + + +def _parse_scalar_frontmatter(text: str) -> Dict[str, str]: + match = FRONTMATTER_RE.match(text) + if match is None: + raise ValueError("No YAML frontmatter block found") + + data: Dict[str, str] = {} + for line in match.group(1).splitlines(): + if not line.strip() or line.startswith(" - "): + continue + if ":" not in line: + raise ValueError("Malformed frontmatter line: {!r}".format(line)) + key, value = line.split(":", 1) + value = value.strip() + if value: + data[key.strip()] = value.strip('"').strip("'") + return data + + +def read_adrs(adr_dir: Path) -> Tuple[List[Dict[str, str]], List[Dict[str, str]]]: + records: List[Dict[str, str]] = [] + warnings: List[Dict[str, str]] = [] + for path in sorted(adr_dir.glob("[0-9]*.md")): + try: + data = _parse_scalar_frontmatter(path.read_text(encoding="utf-8")) + for field in ("id", "title", "status", "date"): + if not data.get(field): + raise ValueError("missing required field: {}".format(field)) + except (OSError, UnicodeError, ValueError) as exc: + warnings.append( + {"code": "BAD_FRONTMATTER", "file": path.name, "detail": str(exc)} + ) + continue + records.append( + { + "id": data["id"], + "title": data["title"], + "status": data["status"], + "date": data["date"], + "file": path.name, + } + ) + return records, warnings + + +def read_exceptions(adr_dir: Path) -> Tuple[List[Dict[str, Any]], List[Dict[str, str]]]: + records: List[Dict[str, Any]] = [] + warnings: List[Dict[str, str]] = [] + exceptions_dir = adr_dir / "exceptions" + if not exceptions_dir.is_dir(): + return records, warnings + + for path in sorted(exceptions_dir.glob("*.json")): + try: + data = json.loads(path.read_text(encoding="utf-8")) + if not isinstance(data, dict): + raise ValueError("exception must be a JSON object") + missing = sorted(REQUIRED_EXCEPTION_FIELDS - set(data)) + if missing: + raise ValueError("missing required field(s): {}".format(", ".join(missing))) + except (json.JSONDecodeError, OSError, UnicodeError, ValueError) as exc: + warnings.append( + {"code": "BAD_EXCEPTION", "file": path.name, "detail": str(exc)} + ) + continue + records.append(data) + return records, warnings + + +def parse_timestamp(value: str) -> datetime: + normalized = value[:-1] + "+00:00" if value.endswith("Z") else value + parsed = datetime.fromisoformat(normalized) + if parsed.tzinfo is None: + parsed = parsed.replace(tzinfo=timezone.utc) + return parsed.astimezone(timezone.utc) + + +def _event_time(event: Dict[str, Any]) -> datetime: + return parse_timestamp(str(event["occurred_at"])) + + +def _in_period(event: Dict[str, Any], since: datetime, until: datetime) -> bool: + occurred_at = _event_time(event) + return since <= occurred_at <= until + + +def _coverage(eligible: int, measured: int) -> Dict[str, Any]: + ratio = measured / eligible if eligible else None + return {"eligible": eligible, "measured": measured, "ratio": ratio} + + +def _decision_lead_time( + events: List[Dict[str, Any]], since: datetime, until: datetime +) -> Dict[str, Any]: + by_adr: Dict[str, List[Dict[str, Any]]] = {} + for event in events: + if event.get("event") in {"adr_created", "adr_status_changed"}: + by_adr.setdefault(str(event.get("adr_id")), []).append(event) + + eligible = 0 + durations: List[float] = [] + sources = set() + for adr_events in by_adr.values(): + ordered = sorted(adr_events, key=_event_time) + proposed_at = None + outcome = None + for event in ordered: + status = event.get("status") if event["event"] == "adr_created" else event.get("to") + if status == "proposed" and proposed_at is None: + proposed_at = _event_time(event) + if status in {"accepted", "rejected"} and _in_period(event, since, until): + outcome = event + break + if outcome is None: + continue + eligible += 1 + if proposed_at is not None and proposed_at <= _event_time(outcome): + durations.append((_event_time(outcome) - proposed_at).total_seconds() / 3600) + sources.update( + str(event.get("source")) + for event in ordered + if proposed_at <= _event_time(event) <= _event_time(outcome) + ) + + result: Dict[str, Any] = { + "available": bool(durations), + "median_hours": statistics.median(durations) if durations else None, + "sample_size": len(durations), + "coverage": _coverage(eligible, len(durations)), + "sources": sorted(sources), + } + if not durations: + result["reason"] = "No completed decision had observable proposed history." + return result + + +def _review_latency( + events: List[Dict[str, Any]], since: datetime, until: datetime +) -> Dict[str, Any]: + requests = [event for event in events if event.get("event") == "review_requested"] + submissions = [event for event in events if event.get("event") == "review_submitted"] + durations: List[float] = [] + open_cycles = 0 + sources = set() + for request in sorted(requests, key=_event_time): + if not (since <= _event_time(request) <= until): + continue + candidates = [ + event + for event in submissions + if event.get("adr_id") == request.get("adr_id") + and event.get("qualified") is True + and _event_time(event) >= _event_time(request) + and _event_time(event) <= until + ] + if not candidates: + open_cycles += 1 + sources.add(str(request.get("source"))) + continue + submitted = min(candidates, key=_event_time) + durations.append( + (_event_time(submitted) - _event_time(request)).total_seconds() / 3600 + ) + sources.update((str(request.get("source")), str(submitted.get("source")))) + + eligible = len(durations) + open_cycles + result: Dict[str, Any] = { + "available": bool(durations), + "median_hours": statistics.median(durations) if durations else None, + "sample_size": len(durations), + "open_cycles": open_cycles, + "coverage": _coverage(eligible, len(durations)), + "sources": sorted(sources), + } + if not durations: + result["reason"] = "No completed qualified review cycle was available." + return result + + +def _supersession_rate( + events: List[Dict[str, Any]], since: datetime, until: datetime +) -> Dict[str, Any]: + transitions = [ + event + for event in events + if event.get("event") == "adr_status_changed" and _in_period(event, since, until) + ] + accepted = sum(event.get("to") == "accepted" for event in transitions) + superseded = sum(event.get("to") == "superseded" for event in transitions) + sources = sorted( + { + str(event.get("source")) + for event in transitions + if event.get("to") in {"accepted", "superseded"} + } + ) + result: Dict[str, Any] = { + "available": accepted > 0, + "rate": superseded / accepted if accepted else None, + "superseded": superseded, + "accepted": accepted, + "sources": sources, + } + if not accepted: + result["reason"] = "No accepted transitions were observed in the period." + return result + + +def _unresolved_violations( + events: List[Dict[str, Any]], until: datetime +) -> Dict[str, Any]: + violation_events = [ + event + for event in events + if event.get("event") in {"violation_observed", "violation_resolved"} + and _event_time(event) <= until + ] + if not violation_events: + return { + "available": False, + "open_count": None, + "age_available": False, + "median_age_days": None, + "max_age_days": None, + "sources": [], + "reason": "No CHECK violation observations were available.", + } + + open_since: Dict[str, datetime] = {} + sources = set() + for event in sorted(violation_events, key=_event_time): + fingerprint = str(event.get("fingerprint")) + sources.add(str(event.get("source"))) + if event["event"] == "violation_resolved": + open_since.pop(fingerprint, None) + elif fingerprint not in open_since: + open_since[fingerprint] = _event_time(event) + + ages = [(until.date() - opened.date()).days for opened in open_since.values()] + return { + "available": True, + "open_count": len(open_since), + "age_available": True, + "median_age_days": statistics.median(ages) if ages else None, + "max_age_days": max(ages) if ages else None, + "sources": sorted(sources), + } + + +def _exception_age( + exceptions: List[Dict[str, Any]], until: datetime +) -> Dict[str, Any]: + ages: List[int] = [] + expired_count = 0 + for exception in exceptions: + created = parse_timestamp(str(exception["created"])).date() + expiry = parse_timestamp(str(exception["expiry"])).date() + if expiry < until.date(): + expired_count += 1 + else: + ages.append((until.date() - created).days) + return { + "available": True, + "active_count": len(ages), + "median_age_days": statistics.median(ages) if ages else None, + "max_age_days": max(ages) if ages else None, + "expired_count": expired_count, + "sources": ["exceptions"], + } + + +def calculate_metrics( + adrs: List[Dict[str, Any]], + exceptions: List[Dict[str, Any]], + events: List[Dict[str, Any]], + since: datetime, + until: datetime, +) -> Dict[str, Any]: + del adrs # Current ADR snapshots are retained for future coverage extensions. + return { + "decision_lead_time": _decision_lead_time(events, since, until), + "review_latency": _review_latency(events, since, until), + "supersession_rate": _supersession_rate(events, since, until), + "unresolved_violations": _unresolved_violations(events, until), + "exception_age": _exception_age(exceptions, until), + } diff --git a/tests/unit/test_adoption_metrics.py b/tests/unit/test_adoption_metrics.py new file mode 100644 index 0000000..b1641cc --- /dev/null +++ b/tests/unit/test_adoption_metrics.py @@ -0,0 +1,341 @@ +import importlib.util +import json +from datetime import datetime, timezone +from pathlib import Path + + +_ADOPTION_METRICS_PATH = ( + Path(__file__).resolve().parents[2] / "scripts" / "adoption_metrics.py" +) +_spec = importlib.util.spec_from_file_location( + "_repo_root_adoption_metrics", _ADOPTION_METRICS_PATH +) +adoption_metrics = importlib.util.module_from_spec(_spec) +_spec.loader.exec_module(adoption_metrics) + +SINCE = datetime(2026, 1, 1, tzinfo=timezone.utc) +UNTIL = datetime(2026, 1, 31, tzinfo=timezone.utc) + + +def _event(event, occurred_at, source="events", **payload): + return { + "schema_version": 1, + "event": event, + "occurred_at": occurred_at, + "source": source, + **payload, + } + + +def test_read_adrs_returns_valid_frontmatter_and_warns_for_malformed_file(tmp_path): + adr_dir = tmp_path / "docs" / "decisions" + adr_dir.mkdir(parents=True) + (adr_dir / "0001-a.md").write_text( + "---\n" + "id: ADR-0001\n" + "title: A\n" + "status: accepted\n" + "date: 2026-01-02\n" + "---\n" + "Body\n", + encoding="utf-8", + ) + (adr_dir / "0002-b.md").write_text("not frontmatter\n", encoding="utf-8") + + records, warnings = adoption_metrics.read_adrs(adr_dir) + + assert records == [ + { + "id": "ADR-0001", + "title": "A", + "status": "accepted", + "date": "2026-01-02", + "file": "0001-a.md", + } + ] + assert warnings[0]["code"] == "BAD_FRONTMATTER" + assert warnings[0]["file"] == "0002-b.md" + + +def test_read_exceptions_keeps_valid_records_and_warns_for_bad_json(tmp_path): + exceptions_dir = tmp_path / "exceptions" + exceptions_dir.mkdir() + valid = { + "id": "EXC-0001", + "adr_id": "ADR-0001", + "rule_id": "r1", + "owner": "team", + "reason": "migration", + "scope": ["src/a.py"], + "created": "2026-01-01", + "expiry": "2026-01-10", + } + (exceptions_dir / "0001.json").write_text(json.dumps(valid), encoding="utf-8") + (exceptions_dir / "0002.json").write_text("{", encoding="utf-8") + + records, warnings = adoption_metrics.read_exceptions(tmp_path) + + assert records == [valid] + assert warnings[0]["code"] == "BAD_EXCEPTION" + assert warnings[0]["file"] == "0002.json" + + +def test_decision_lead_time_is_median_completed_cycle_hours(): + events = [ + _event( + "adr_created", + "2026-01-01T00:00:00Z", + adr_id="ADR-0001", + status="proposed", + ), + _event( + "adr_status_changed", + "2026-01-02T00:00:00Z", + adr_id="ADR-0001", + **{"from": "proposed", "to": "accepted"}, + ), + _event( + "adr_created", + "2026-01-01T00:00:00Z", + adr_id="ADR-0002", + status="proposed", + ), + _event( + "adr_status_changed", + "2026-01-04T00:00:00Z", + adr_id="ADR-0002", + **{"from": "proposed", "to": "rejected"}, + ), + ] + + result = adoption_metrics.calculate_metrics([], [], events, SINCE, UNTIL)[ + "decision_lead_time" + ] + + assert result == { + "available": True, + "median_hours": 48.0, + "sample_size": 2, + "coverage": {"eligible": 2, "measured": 2, "ratio": 1.0}, + "sources": ["events"], + } + + +def test_decision_lead_time_excludes_terminal_first_observation_from_coverage(): + events = [ + _event( + "adr_created", + "2026-01-02T00:00:00Z", + source="git", + adr_id="ADR-0001", + status="accepted", + ) + ] + + result = adoption_metrics.calculate_metrics([], [], events, SINCE, UNTIL)[ + "decision_lead_time" + ] + + assert result["available"] is False + assert result["median_hours"] is None + assert result["coverage"] == {"eligible": 1, "measured": 0, "ratio": 0.0} + assert result["reason"] == "No completed decision had observable proposed history." + + +def test_review_latency_uses_first_qualified_review_after_request(): + events = [ + _event( + "review_requested", + "2026-01-01T00:00:00Z", + adr_id="ADR-0001", + reviewer="alice", + ), + _event( + "review_submitted", + "2026-01-01T01:00:00Z", + adr_id="ADR-0001", + reviewer="bob", + qualified=False, + ), + _event( + "review_submitted", + "2026-01-01T06:00:00Z", + adr_id="ADR-0001", + reviewer="alice", + qualified=True, + ), + ] + + result = adoption_metrics.calculate_metrics([], [], events, SINCE, UNTIL)[ + "review_latency" + ] + + assert result["median_hours"] == 6.0 + assert result["sample_size"] == 1 + assert result["open_cycles"] == 0 + assert result["coverage"] == {"eligible": 1, "measured": 1, "ratio": 1.0} + + +def test_review_latency_reports_open_cycle_without_adding_it_to_median(): + events = [ + _event( + "review_requested", + "2026-01-01T00:00:00Z", + adr_id="ADR-0001", + reviewer="alice", + ), + _event( + "review_requested", + "2026-01-02T00:00:00Z", + adr_id="ADR-0002", + reviewer="bob", + ), + _event( + "review_submitted", + "2026-01-02T12:00:00Z", + adr_id="ADR-0002", + reviewer="bob", + qualified=True, + ), + ] + + result = adoption_metrics.calculate_metrics([], [], events, SINCE, UNTIL)[ + "review_latency" + ] + + assert result["median_hours"] == 12.0 + assert result["open_cycles"] == 1 + assert result["coverage"] == {"eligible": 2, "measured": 1, "ratio": 0.5} + + +def test_supersession_rate_uses_transitions_within_period(): + events = [ + _event( + "adr_status_changed", + "2026-01-02T00:00:00Z", + adr_id="ADR-0001", + **{"from": "proposed", "to": "accepted"}, + ), + _event( + "adr_status_changed", + "2026-01-03T00:00:00Z", + adr_id="ADR-0002", + **{"from": "proposed", "to": "accepted"}, + ), + _event( + "adr_status_changed", + "2026-01-04T00:00:00Z", + adr_id="ADR-0001", + **{"from": "accepted", "to": "superseded"}, + ), + _event( + "adr_status_changed", + "2025-12-31T00:00:00Z", + adr_id="ADR-0003", + **{"from": "accepted", "to": "superseded"}, + ), + ] + + result = adoption_metrics.calculate_metrics([], [], events, SINCE, UNTIL)[ + "supersession_rate" + ] + + assert result == { + "available": True, + "rate": 0.5, + "superseded": 1, + "accepted": 2, + "sources": ["events"], + } + + +def test_supersession_rate_is_unavailable_when_no_acceptance_is_observed(): + result = adoption_metrics.calculate_metrics([], [], [], SINCE, UNTIL)[ + "supersession_rate" + ] + + assert result["available"] is False + assert result["rate"] is None + assert result["accepted"] == 0 + assert result["reason"] == "No accepted transitions were observed in the period." + + +def test_unresolved_violations_use_first_observation_after_latest_resolution(): + events = [ + _event( + "violation_observed", + "2026-01-01T00:00:00Z", + fingerprint="f1", + adr_id="ADR-0001", + rule_id="r1", + ), + _event( + "violation_resolved", + "2026-01-05T00:00:00Z", + fingerprint="f1", + adr_id="ADR-0001", + rule_id="r1", + ), + _event( + "violation_observed", + "2026-01-21T00:00:00Z", + fingerprint="f1", + adr_id="ADR-0001", + rule_id="r1", + ), + ] + + result = adoption_metrics.calculate_metrics([], [], events, SINCE, UNTIL)[ + "unresolved_violations" + ] + + assert result == { + "available": True, + "open_count": 1, + "age_available": True, + "median_age_days": 10, + "max_age_days": 10, + "sources": ["events"], + } + + +def test_unresolved_violations_are_unavailable_without_observations(): + result = adoption_metrics.calculate_metrics([], [], [], SINCE, UNTIL)[ + "unresolved_violations" + ] + + assert result["available"] is False + assert result["open_count"] is None + assert result["reason"] == "No CHECK violation observations were available." + + +def test_exception_age_separates_active_age_from_expired_count(): + exceptions = [ + {"id": "EXC-0001", "created": "2026-01-01", "expiry": "2026-02-01"}, + {"id": "EXC-0002", "created": "2026-01-11", "expiry": "2026-02-02"}, + {"id": "EXC-0003", "created": "2026-01-01", "expiry": "2026-01-02"}, + ] + + result = adoption_metrics.calculate_metrics([], exceptions, [], SINCE, UNTIL)[ + "exception_age" + ] + + assert result == { + "available": True, + "active_count": 2, + "median_age_days": 25.0, + "max_age_days": 30, + "expired_count": 1, + "sources": ["exceptions"], + } + + +def test_exception_age_is_available_for_an_empty_exception_directory(): + result = adoption_metrics.calculate_metrics([], [], [], SINCE, UNTIL)[ + "exception_age" + ] + + assert result["available"] is True + assert result["active_count"] == 0 + assert result["median_age_days"] is None + assert result["expired_count"] == 0 From 8ee84c079aa6fb3c95441a215b4e652dc209126d Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 14:36:15 +0900 Subject: [PATCH 42/58] feat: collect portable Git and GitHub metric evidence --- scripts/adoption_metrics.py | 441 ++++++++++++++++++++++++++++ tests/unit/test_adoption_metrics.py | 312 ++++++++++++++++++++ 2 files changed, 753 insertions(+) diff --git a/scripts/adoption_metrics.py b/scripts/adoption_metrics.py index bdf7966..a359d85 100644 --- a/scripts/adoption_metrics.py +++ b/scripts/adoption_metrics.py @@ -4,6 +4,7 @@ import json import re import statistics +import subprocess from datetime import datetime, timezone from pathlib import Path from typing import Any, Dict, List, Tuple @@ -20,6 +21,50 @@ "created", "expiry", } +EVENT_REQUIRED_FIELDS = { + "adr_created": {"adr_id", "status"}, + "adr_status_changed": {"adr_id", "from", "to"}, + "review_requested": {"adr_id", "reviewer"}, + "review_submitted": {"adr_id", "reviewer", "qualified"}, + "violation_observed": {"fingerprint", "adr_id", "rule_id"}, + "violation_resolved": {"fingerprint", "adr_id", "rule_id"}, +} +GITHUB_REVIEW_QUERY = """ +query($owner: String!, $name: String!) { + repository(owner: $owner, name: $name) { + pullRequests(first: 100, orderBy: {field: UPDATED_AT, direction: DESC}) { + nodes { + number + author { login } + files(first: 100) { nodes { path } pageInfo { hasNextPage } } + timelineItems( + first: 100, + itemTypes: [REVIEW_REQUESTED_EVENT, PULL_REQUEST_REVIEW] + ) { + nodes { + __typename + ... on ReviewRequestedEvent { + createdAt + requestedReviewer { + __typename + ... on User { login } + ... on Team { slug } + ... on Mannequin { login } + } + } + ... on PullRequestReview { + submittedAt + author { login } + } + } + pageInfo { hasNextPage } + } + } + pageInfo { hasNextPage } + } + } +} +""" def _parse_scalar_frontmatter(text: str) -> Dict[str, str]: @@ -98,6 +143,402 @@ def parse_timestamp(value: str) -> datetime: return parsed.astimezone(timezone.utc) +def _validate_event(data: Any) -> None: + if not isinstance(data, dict): + raise ValueError("event must be a JSON object") + if data.get("schema_version") != 1: + raise ValueError("schema_version must be 1") + event_name = data.get("event") + if event_name not in EVENT_REQUIRED_FIELDS: + raise ValueError("unknown event: {!r}".format(event_name)) + missing = ( + {"occurred_at", "source"} | EVENT_REQUIRED_FIELDS[str(event_name)] + ) - set(data) + if missing: + raise ValueError("missing required field(s): {}".format(", ".join(sorted(missing)))) + parse_timestamp(str(data["occurred_at"])) + + +def read_events( + paths: List[Path], +) -> Tuple[List[Dict[str, Any]], List[Dict[str, Any]]]: + events: List[Dict[str, Any]] = [] + warnings: List[Dict[str, Any]] = [] + for path in paths: + try: + lines = path.read_text(encoding="utf-8").splitlines() + except (OSError, UnicodeError) as exc: + warnings.append( + {"code": "BAD_EVENT_FILE", "file": str(path), "detail": str(exc)} + ) + continue + for line_number, line in enumerate(lines, start=1): + if not line.strip(): + continue + try: + data = json.loads(line) + except json.JSONDecodeError as exc: + warnings.append( + { + "code": "BAD_EVENT_JSON", + "file": str(path), + "line": line_number, + "detail": str(exc), + } + ) + continue + try: + _validate_event(data) + except (TypeError, ValueError) as exc: + warnings.append( + { + "code": "BAD_EVENT_SCHEMA", + "file": str(path), + "line": line_number, + "detail": str(exc), + } + ) + continue + events.append(data) + return events, warnings + + +def _event_entity(event: Dict[str, Any]) -> str: + if "fingerprint" in event: + return str(event["fingerprint"]) + if event.get("event") in {"review_requested", "review_submitted"}: + return "{}:{}".format(event.get("adr_id"), event.get("reviewer")) + return str(event.get("adr_id")) + + +def _event_identity(event: Dict[str, Any]) -> Tuple[str, str, str]: + return ( + str(event["event"]), + parse_timestamp(str(event["occurred_at"])).isoformat(), + _event_entity(event), + ) + + +def _event_payload(event: Dict[str, Any]) -> Dict[str, Any]: + return {key: value for key, value in event.items() if key != "source"} + + +def merge_events( + source_events: List[Tuple[str, List[Dict[str, Any]]]], +) -> Tuple[List[Dict[str, Any]], List[Dict[str, Any]]]: + merged: Dict[Tuple[str, str, str], Tuple[str, Dict[str, Any]]] = {} + warnings: List[Dict[str, Any]] = [] + for source_group, events in source_events: + for event in events: + identity = _event_identity(event) + existing = merged.get(identity) + if existing is None: + merged[identity] = (source_group, event) + continue + kept_group, kept_event = existing + if _event_payload(kept_event) == _event_payload(event): + continue + warnings.append( + { + "code": "EVENT_CONFLICT", + "event": str(event["event"]), + "entity": _event_entity(event).split(":", 1)[0], + "kept_source": kept_group, + "discarded_source": source_group, + } + ) + + ordered = sorted( + (event for _, event in merged.values()), + key=lambda event: (_event_time(event), str(event["event"]), _event_entity(event)), + ) + return ordered, warnings + + +def _run_git(root: Path, arguments: List[str]) -> subprocess.CompletedProcess: + return subprocess.run( + ["git"] + arguments, + cwd=str(root), + capture_output=True, + text=True, + check=False, + ) + + +def collect_git_events( + root: Path, adr_dir: Path +) -> Tuple[List[Dict[str, Any]], List[Dict[str, Any]]]: + probe = _run_git(root, ["rev-parse", "--is-inside-work-tree"]) + if probe.returncode != 0 or probe.stdout.strip() != "true": + return [], [ + {"code": "GIT_UNAVAILABLE", "detail": "Root is not a Git work tree."} + ] + + events: List[Dict[str, Any]] = [] + warnings: List[Dict[str, Any]] = [] + for path in sorted(adr_dir.glob("[0-9]*.md")): + try: + relative_path = path.resolve().relative_to(root.resolve()).as_posix() + except ValueError: + warnings.append( + { + "code": "GIT_PATH_OUTSIDE_ROOT", + "file": str(path), + "detail": "ADR path is outside the Git root.", + } + ) + continue + history = _run_git( + root, + ["log", "--follow", "--format=%H%x1f%aI", "--", relative_path], + ) + if history.returncode != 0: + warnings.append( + { + "code": "GIT_HISTORY_FAILED", + "file": relative_path, + "detail": "Could not read ADR history.", + } + ) + continue + + versions: List[Tuple[str, str, Dict[str, str]]] = [] + for line in reversed([item for item in history.stdout.splitlines() if item]): + commit, separator, timestamp = line.partition("\x1f") + if not separator: + continue + shown = _run_git(root, ["show", "{}:{}".format(commit, relative_path)]) + if shown.returncode != 0: + warnings.append( + { + "code": "GIT_VERSION_UNAVAILABLE", + "file": relative_path, + "commit": commit, + "detail": "Could not read historical ADR content.", + } + ) + continue + try: + data = _parse_scalar_frontmatter(shown.stdout) + if not data.get("id") or not data.get("status"): + raise ValueError("historical ADR is missing id or status") + occurred_at = parse_timestamp(timestamp).isoformat() + except (TypeError, ValueError) as exc: + warnings.append( + { + "code": "BAD_GIT_FRONTMATTER", + "file": relative_path, + "commit": commit, + "detail": str(exc), + } + ) + continue + versions.append((commit, occurred_at, data)) + + previous_status = None + previous_id = None + for _, occurred_at, data in versions: + adr_id = data["id"] + status = data["status"] + if previous_status is None: + events.append( + { + "schema_version": 1, + "event": "adr_created", + "occurred_at": occurred_at, + "source": "git", + "adr_id": adr_id, + "status": status, + } + ) + elif adr_id != previous_id: + warnings.append( + { + "code": "ADR_ID_CHANGED", + "file": relative_path, + "detail": "ADR ID changed from {} to {}.".format( + previous_id, adr_id + ), + } + ) + elif status != previous_status: + events.append( + { + "schema_version": 1, + "event": "adr_status_changed", + "occurred_at": occurred_at, + "source": "git", + "adr_id": adr_id, + "from": previous_status, + "to": status, + } + ) + previous_id = adr_id + previous_status = status + + return sorted(events, key=_event_time), warnings + + +def _run_gh(root: Path, arguments: List[str]) -> subprocess.CompletedProcess: + return subprocess.run( + ["gh"] + arguments, + cwd=str(root), + capture_output=True, + text=True, + check=False, + ) + + +def collect_github_payload( + root: Path, +) -> Tuple[Any, List[Dict[str, str]]]: + repository_result = _run_gh(root, ["repo", "view", "--json", "nameWithOwner"]) + if repository_result.returncode != 0: + return None, [ + { + "code": "GITHUB_UNAVAILABLE", + "detail": "Could not determine the GitHub repository.", + } + ] + try: + repository_data = json.loads(repository_result.stdout) + owner, name = str(repository_data["nameWithOwner"]).split("/", 1) + except (KeyError, TypeError, ValueError, json.JSONDecodeError): + return None, [ + { + "code": "GITHUB_BAD_RESPONSE", + "detail": "GitHub returned an invalid repository identity.", + } + ] + + result = _run_gh( + root, + [ + "api", + "graphql", + "-f", + "query={}".format(GITHUB_REVIEW_QUERY), + "-F", + "owner={}".format(owner), + "-F", + "name={}".format(name), + ], + ) + if result.returncode != 0: + return None, [ + { + "code": "GITHUB_UNAVAILABLE", + "detail": "Could not collect GitHub review history.", + } + ] + try: + return json.loads(result.stdout), [] + except json.JSONDecodeError: + return None, [ + { + "code": "GITHUB_BAD_RESPONSE", + "detail": "GitHub returned invalid review JSON.", + } + ] + + +def _reviewer_login(node: Any) -> Any: + if not isinstance(node, dict): + return None + return node.get("login") or node.get("slug") + + +def normalize_github_reviews( + payload: Any, adr_paths: Dict[str, str] +) -> Tuple[List[Dict[str, Any]], List[Dict[str, str]]]: + warnings: List[Dict[str, str]] = [] + try: + pull_requests = payload["data"]["repository"]["pullRequests"] + nodes = pull_requests["nodes"] + except (KeyError, TypeError): + return [], [ + { + "code": "GITHUB_BAD_RESPONSE", + "detail": "GitHub review response is missing pull request data.", + } + ] + if pull_requests.get("pageInfo", {}).get("hasNextPage"): + warnings.append( + { + "code": "GITHUB_RESULTS_TRUNCATED", + "detail": "GitHub returned more pull requests than this collection fetched.", + } + ) + + events: List[Dict[str, Any]] = [] + for pull_request in nodes: + files = pull_request.get("files", {}) + timeline = pull_request.get("timelineItems", {}) + if files.get("pageInfo", {}).get("hasNextPage") or timeline.get( + "pageInfo", {} + ).get("hasNextPage"): + warnings.append( + { + "code": "GITHUB_PR_RESULTS_TRUNCATED", + "detail": "GitHub truncated files or review events for pull request {}.".format( + pull_request.get("number") + ), + } + ) + adr_ids = sorted( + { + adr_paths[file_node.get("path")] + for file_node in files.get("nodes", []) + if file_node.get("path") in adr_paths + } + ) + if not adr_ids: + continue + author = _reviewer_login(pull_request.get("author")) + requested = set() + timeline_nodes = sorted( + timeline.get("nodes", []), + key=lambda node: str(node.get("createdAt") or node.get("submittedAt") or ""), + ) + for node in timeline_nodes: + if node.get("__typename") == "ReviewRequestedEvent": + reviewer = _reviewer_login(node.get("requestedReviewer")) + occurred_at = node.get("createdAt") + if not reviewer or not occurred_at: + continue + requested.add(reviewer) + for adr_id in adr_ids: + events.append( + { + "schema_version": 1, + "event": "review_requested", + "occurred_at": parse_timestamp(str(occurred_at)).isoformat(), + "source": "github", + "adr_id": adr_id, + "reviewer": reviewer, + } + ) + elif node.get("__typename") == "PullRequestReview": + reviewer = _reviewer_login(node.get("author")) + occurred_at = node.get("submittedAt") + if not reviewer or not occurred_at: + continue + qualified = reviewer in requested and reviewer != author + for adr_id in adr_ids: + events.append( + { + "schema_version": 1, + "event": "review_submitted", + "occurred_at": parse_timestamp(str(occurred_at)).isoformat(), + "source": "github", + "adr_id": adr_id, + "reviewer": reviewer, + "qualified": qualified, + } + ) + return sorted(events, key=_event_time), warnings + + def _event_time(event: Dict[str, Any]) -> datetime: return parse_timestamp(str(event["occurred_at"])) diff --git a/tests/unit/test_adoption_metrics.py b/tests/unit/test_adoption_metrics.py index b1641cc..ec6660f 100644 --- a/tests/unit/test_adoption_metrics.py +++ b/tests/unit/test_adoption_metrics.py @@ -1,5 +1,7 @@ import importlib.util import json +import os +import subprocess from datetime import datetime, timezone from pathlib import Path @@ -27,6 +29,33 @@ def _event(event, occurred_at, source="events", **payload): } +def _git(repo, *args, timestamp=None): + env = os.environ.copy() + if timestamp is not None: + env["GIT_AUTHOR_DATE"] = timestamp + env["GIT_COMMITTER_DATE"] = timestamp + return subprocess.run( + ["git", *args], + cwd=repo, + env=env, + capture_output=True, + text=True, + check=True, + ) + + +def _write_adr(path, adr_id, status, date="2026-01-01"): + path.write_text( + "---\n" + "id: {}\n".format(adr_id) + + "title: Test decision\n" + + "status: {}\n".format(status) + + "date: {}\n".format(date) + + "---\nBody\n", + encoding="utf-8", + ) + + def test_read_adrs_returns_valid_frontmatter_and_warns_for_malformed_file(tmp_path): adr_dir = tmp_path / "docs" / "decisions" adr_dir.mkdir(parents=True) @@ -339,3 +368,286 @@ def test_exception_age_is_available_for_an_empty_exception_directory(): assert result["active_count"] == 0 assert result["median_age_days"] is None assert result["expired_count"] == 0 + + +def test_read_events_keeps_valid_lines_and_warns_for_invalid_records(tmp_path): + events_path = tmp_path / "events.jsonl" + events_path.write_text( + json.dumps( + _event( + "adr_created", + "2026-01-01T00:00:00Z", + adr_id="ADR-0001", + status="proposed", + ) + ) + + "\n" + + "{\n" + + json.dumps( + { + "schema_version": 2, + "event": "adr_created", + "occurred_at": "2026-01-01T00:00:00Z", + "source": "events", + "adr_id": "ADR-0002", + "status": "accepted", + } + ) + + "\n", + encoding="utf-8", + ) + + events, warnings = adoption_metrics.read_events([events_path]) + + assert len(events) == 1 + assert events[0]["adr_id"] == "ADR-0001" + assert [warning["code"] for warning in warnings] == [ + "BAD_EVENT_JSON", + "BAD_EVENT_SCHEMA", + ] + assert [warning["line"] for warning in warnings] == [2, 3] + + +def test_read_events_rejects_unknown_event_and_missing_required_payload(tmp_path): + events_path = tmp_path / "events.jsonl" + events_path.write_text( + json.dumps( + { + "schema_version": 1, + "event": "unknown", + "occurred_at": "2026-01-01T00:00:00Z", + "source": "events", + } + ) + + "\n" + + json.dumps( + { + "schema_version": 1, + "event": "review_requested", + "occurred_at": "2026-01-01T00:00:00Z", + "source": "events", + "adr_id": "ADR-0001", + } + ) + + "\n", + encoding="utf-8", + ) + + events, warnings = adoption_metrics.read_events([events_path]) + + assert events == [] + assert [warning["code"] for warning in warnings] == [ + "BAD_EVENT_SCHEMA", + "BAD_EVENT_SCHEMA", + ] + + +def test_merge_events_deduplicates_and_prefers_explicit_conflicting_payload(): + explicit = _event( + "adr_created", + "2026-01-01T00:00:00Z", + source="manual_export", + adr_id="ADR-0001", + status="proposed", + ) + reconstructed_duplicate = dict(explicit, source="git") + reconstructed_conflict = dict(explicit, source="git", status="accepted") + + events, warnings = adoption_metrics.merge_events( + [("events", [explicit]), ("git", [reconstructed_duplicate, reconstructed_conflict])] + ) + + assert events == [explicit] + assert warnings == [ + { + "code": "EVENT_CONFLICT", + "event": "adr_created", + "entity": "ADR-0001", + "kept_source": "events", + "discarded_source": "git", + } + ] + + +def test_collect_git_events_reconstructs_proposed_to_accepted_in_path_with_spaces( + tmp_path, +): + root = tmp_path / "repo with spaces" + adr_dir = root / "docs" / "decisions" + adr_dir.mkdir(parents=True) + _git(root, "init") + _git(root, "config", "user.name", "Test") + _git(root, "config", "user.email", "test@example.com") + adr_path = adr_dir / "0001-test.md" + _write_adr(adr_path, "ADR-0001", "proposed") + _git(root, "add", str(adr_path.relative_to(root))) + _git(root, "commit", "-m", "propose", timestamp="2026-01-01T00:00:00Z") + _write_adr(adr_path, "ADR-0001", "accepted") + _git(root, "add", str(adr_path.relative_to(root))) + _git(root, "commit", "-m", "accept", timestamp="2026-01-02T00:00:00Z") + + events, warnings = adoption_metrics.collect_git_events(root, adr_dir) + + assert warnings == [] + assert events == [ + _event( + "adr_created", + "2026-01-01T00:00:00+00:00", + source="git", + adr_id="ADR-0001", + status="proposed", + ), + _event( + "adr_status_changed", + "2026-01-02T00:00:00+00:00", + source="git", + adr_id="ADR-0001", + **{"from": "proposed", "to": "accepted"}, + ), + ] + + +def test_collect_git_events_does_not_invent_proposed_for_terminal_first_commit(tmp_path): + root = tmp_path / "repo" + adr_dir = root / "docs" / "decisions" + adr_dir.mkdir(parents=True) + _git(root, "init") + _git(root, "config", "user.name", "Test") + _git(root, "config", "user.email", "test@example.com") + adr_path = adr_dir / "0001-test.md" + _write_adr(adr_path, "ADR-0001", "accepted") + _git(root, "add", str(adr_path.relative_to(root))) + _git(root, "commit", "-m", "record", timestamp="2026-01-01T00:00:00Z") + + events, warnings = adoption_metrics.collect_git_events(root, adr_dir) + + assert warnings == [] + assert [event["status"] for event in events if event["event"] == "adr_created"] == [ + "accepted" + ] + assert not any( + event.get("status") == "proposed" or event.get("to") == "proposed" + for event in events + ) + + +def test_collect_git_events_degrades_cleanly_outside_a_git_repository(tmp_path): + adr_dir = tmp_path / "docs" / "decisions" + adr_dir.mkdir(parents=True) + + events, warnings = adoption_metrics.collect_git_events(tmp_path, adr_dir) + + assert events == [] + assert warnings == [ + { + "code": "GIT_UNAVAILABLE", + "detail": "Root is not a Git work tree.", + } + ] + + +def test_normalize_github_reviews_qualifies_requested_non_author_reviewer(): + payload = { + "data": { + "repository": { + "pullRequests": { + "nodes": [ + { + "number": 7, + "author": {"login": "owner"}, + "files": { + "nodes": [ + {"path": "docs/decisions/0001-test.md"}, + {"path": "src/app.py"}, + ] + }, + "timelineItems": { + "nodes": [ + { + "__typename": "ReviewRequestedEvent", + "createdAt": "2026-01-01T00:00:00Z", + "requestedReviewer": { + "__typename": "User", + "login": "alice", + }, + }, + { + "__typename": "PullRequestReview", + "submittedAt": "2026-01-01T01:00:00Z", + "author": {"login": "bob"}, + }, + { + "__typename": "PullRequestReview", + "submittedAt": "2026-01-01T02:00:00Z", + "author": {"login": "owner"}, + }, + { + "__typename": "PullRequestReview", + "submittedAt": "2026-01-01T03:00:00Z", + "author": {"login": "alice"}, + }, + ] + }, + } + ], + "pageInfo": {"hasNextPage": False}, + } + } + } + } + + events, warnings = adoption_metrics.normalize_github_reviews( + payload, {"docs/decisions/0001-test.md": "ADR-0001"} + ) + + assert warnings == [] + submitted = [event for event in events if event["event"] == "review_submitted"] + assert [(event["reviewer"], event["qualified"]) for event in submitted] == [ + ("bob", False), + ("owner", False), + ("alice", True), + ] + assert all(event["adr_id"] == "ADR-0001" for event in events) + + +def test_normalize_github_reviews_warns_when_provider_result_is_truncated(): + payload = { + "data": { + "repository": { + "pullRequests": { + "nodes": [], + "pageInfo": {"hasNextPage": True}, + } + } + } + } + + events, warnings = adoption_metrics.normalize_github_reviews(payload, {}) + + assert events == [] + assert warnings == [ + { + "code": "GITHUB_RESULTS_TRUNCATED", + "detail": "GitHub returned more pull requests than this collection fetched.", + } + ] + + +def test_collect_github_payload_hides_provider_stderr_on_failure(tmp_path, monkeypatch): + def fail_gh(root, arguments): + return subprocess.CompletedProcess( + ["gh", *arguments], returncode=1, stdout="", stderr="token=secret-value" + ) + + monkeypatch.setattr(adoption_metrics, "_run_gh", fail_gh) + + payload, warnings = adoption_metrics.collect_github_payload(tmp_path) + + assert payload is None + assert warnings == [ + { + "code": "GITHUB_UNAVAILABLE", + "detail": "Could not determine the GitHub repository.", + } + ] + assert "secret-value" not in json.dumps(warnings) From 45b34729e06bcd8388ce796294b409171ce8c847 Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 14:39:52 +0900 Subject: [PATCH 43/58] feat: expose adoption metrics JSON collector --- scripts/adoption_metrics.py | 211 +++++++++++++++++++++++- tests/unit/test_adoption_metrics.py | 241 ++++++++++++++++++++++++++++ 2 files changed, 445 insertions(+), 7 deletions(-) diff --git a/scripts/adoption_metrics.py b/scripts/adoption_metrics.py index a359d85..f2e7c02 100644 --- a/scripts/adoption_metrics.py +++ b/scripts/adoption_metrics.py @@ -1,13 +1,15 @@ #!/usr/bin/env python3 """Collect provider-neutral ADR adoption metrics as deterministic JSON.""" +import argparse import json import re import statistics import subprocess +import sys from datetime import datetime, timezone from pathlib import Path -from typing import Any, Dict, List, Tuple +from typing import Any, Dict, List, Optional, Tuple FRONTMATTER_RE = re.compile(r"\A---\n(.*?)\n---(?:\n|\Z)", re.DOTALL) @@ -94,6 +96,10 @@ def read_adrs(adr_dir: Path) -> Tuple[List[Dict[str, str]], List[Dict[str, str]] for field in ("id", "title", "status", "date"): if not data.get(field): raise ValueError("missing required field: {}".format(field)) + try: + parse_timestamp(data["date"]) + except ValueError as exc: + raise ValueError("invalid date: {}".format(exc)) except (OSError, UnicodeError, ValueError) as exc: warnings.append( {"code": "BAD_FRONTMATTER", "file": path.name, "detail": str(exc)} @@ -126,6 +132,11 @@ def read_exceptions(adr_dir: Path) -> Tuple[List[Dict[str, Any]], List[Dict[str, missing = sorted(REQUIRED_EXCEPTION_FIELDS - set(data)) if missing: raise ValueError("missing required field(s): {}".format(", ".join(missing))) + for field in ("created", "expiry"): + try: + parse_timestamp(str(data[field])) + except ValueError as exc: + raise ValueError("invalid {}: {}".format(field, exc)) except (json.JSONDecodeError, OSError, UnicodeError, ValueError) as exc: warnings.append( {"code": "BAD_EXCEPTION", "file": path.name, "detail": str(exc)} @@ -644,18 +655,30 @@ def _review_latency( def _supersession_rate( events: List[Dict[str, Any]], since: datetime, until: datetime ) -> Dict[str, Any]: - transitions = [ + lifecycle_events = [ event for event in events - if event.get("event") == "adr_status_changed" and _in_period(event, since, until) + if event.get("event") in {"adr_created", "adr_status_changed"} + and _in_period(event, since, until) ] - accepted = sum(event.get("to") == "accepted" for event in transitions) - superseded = sum(event.get("to") == "superseded" for event in transitions) + accepted_ids = { + str(event.get("adr_id")) + for event in lifecycle_events + if event.get("to") == "accepted" + or (event.get("event") == "adr_created" and event.get("status") == "accepted") + } + superseded_ids = { + str(event.get("adr_id")) + for event in lifecycle_events + if event.get("to") == "superseded" + } + accepted = len(accepted_ids) + superseded = len(superseded_ids) sources = sorted( { str(event.get("source")) - for event in transitions - if event.get("to") in {"accepted", "superseded"} + for event in lifecycle_events + if event.get("adr_id") in accepted_ids | superseded_ids } ) result: Dict[str, Any] = { @@ -748,3 +771,177 @@ def calculate_metrics( "unresolved_violations": _unresolved_violations(events, until), "exception_age": _exception_age(exceptions, until), } + + +class CollectionError(Exception): + def __init__(self, error: Dict[str, Any]): + super().__init__(str(error)) + self.error = error + + +class JsonArgumentParser(argparse.ArgumentParser): + def error(self, message: str) -> None: + raise CollectionError({"code": "INVALID_ARGUMENT", "detail": message}) + + +def _resolve_within_root(root: Path, value: str) -> Path: + candidate = Path(value) + resolved = (candidate if candidate.is_absolute() else root / candidate).resolve() + try: + resolved.relative_to(root) + except ValueError: + raise CollectionError({"code": "PATH_ESCAPES_ROOT", "path": str(resolved)}) + return resolved + + +def _default_since( + adrs: List[Dict[str, Any]], events: List[Dict[str, Any]], until: datetime +) -> datetime: + candidates: List[datetime] = [] + for event in events: + try: + candidates.append(_event_time(event)) + except (KeyError, TypeError, ValueError): + continue + for adr in adrs: + try: + candidates.append(parse_timestamp(str(adr["date"]))) + except (KeyError, TypeError, ValueError): + continue + return min(candidates) if candidates else until + + +def build_report( + root: Path, + adr_dir: Path, + since: Optional[datetime], + until: datetime, + event_paths: List[Path], + check_paths: List[Path], + use_github: bool, +) -> Dict[str, Any]: + if not adr_dir.is_dir(): + raise CollectionError({"code": "ADR_DIR_NOT_FOUND", "path": str(adr_dir)}) + + adrs, adr_warnings = read_adrs(adr_dir) + exceptions, exception_warnings = read_exceptions(adr_dir) + explicit_events, explicit_warnings = read_events(event_paths + check_paths) + git_events, git_warnings = collect_git_events(root, adr_dir) + + github_events: List[Dict[str, Any]] = [] + github_warnings: List[Dict[str, Any]] = [] + if use_github: + payload, github_warnings = collect_github_payload(root) + if payload is not None: + adr_paths = { + (adr_dir / str(adr["file"])).relative_to(root).as_posix(): str(adr["id"]) + for adr in adrs + } + github_events, normalization_warnings = normalize_github_reviews( + payload, adr_paths + ) + github_warnings += normalization_warnings + + events, merge_warnings = merge_events( + [ + ("events", explicit_events), + ("git", git_events), + ("github", github_events), + ] + ) + effective_since = since if since is not None else _default_since(adrs, events, until) + if effective_since > until: + raise CollectionError( + { + "code": "INVALID_PERIOD", + "detail": "--since must be earlier than or equal to --until.", + } + ) + + return { + "ok": True, + "operation": "adoption_metrics", + "schema_version": 1, + "period": { + "since": effective_since.date().isoformat(), + "until": until.date().isoformat(), + }, + "metrics": calculate_metrics( + adrs, exceptions, events, effective_since, until + ), + "warnings": ( + adr_warnings + + exception_warnings + + explicit_warnings + + git_warnings + + github_warnings + + merge_warnings + ), + } + + +def _parse_cli_timestamp(value: Optional[str], option: str) -> Optional[datetime]: + if value is None: + return None + try: + return parse_timestamp(value) + except ValueError: + raise CollectionError( + {"code": "INVALID_DATE", "option": option, "value": value} + ) + + +def _parser() -> JsonArgumentParser: + parser = JsonArgumentParser() + parser.add_argument("--root", default=".") + parser.add_argument("--dir", default="docs/decisions") + parser.add_argument("--since") + parser.add_argument("--until") + parser.add_argument("--events", action="append", default=[]) + parser.add_argument("--check-results", action="append", default=[]) + parser.add_argument("--github", action="store_true") + parser.add_argument("--json", action="store_true") + return parser + + +def main(argv: Optional[List[str]] = None) -> int: + try: + args = _parser().parse_args(argv) + if not args.json: + raise CollectionError( + {"code": "JSON_REQUIRED", "detail": "--json is required."} + ) + root = Path(args.root).resolve() + adr_dir = _resolve_within_root(root, args.dir) + event_paths = [_resolve_within_root(root, value) for value in args.events] + check_paths = [ + _resolve_within_root(root, value) for value in args.check_results + ] + since = _parse_cli_timestamp(args.since, "--since") + until = _parse_cli_timestamp(args.until, "--until") + if until is None: + now = datetime.now(timezone.utc) + until = datetime(now.year, now.month, now.day, tzinfo=timezone.utc) + report = build_report( + root, + adr_dir, + since, + until, + event_paths, + check_paths, + args.github, + ) + return_code = 0 + except CollectionError as exc: + report = { + "ok": False, + "operation": "adoption_metrics", + "errors": [exc.error], + } + return_code = 1 + print(json.dumps(report, ensure_ascii=False, sort_keys=True)) + return return_code + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tests/unit/test_adoption_metrics.py b/tests/unit/test_adoption_metrics.py index ec6660f..2ef208c 100644 --- a/tests/unit/test_adoption_metrics.py +++ b/tests/unit/test_adoption_metrics.py @@ -109,6 +109,40 @@ def test_read_exceptions_keeps_valid_records_and_warns_for_bad_json(tmp_path): assert warnings[0]["file"] == "0002.json" +def test_read_adrs_warns_and_skips_an_invalid_date(tmp_path): + adr_dir = tmp_path / "docs" / "decisions" + adr_dir.mkdir(parents=True) + _write_adr(adr_dir / "0001-test.md", "ADR-0001", "accepted", date="not-a-date") + + records, warnings = adoption_metrics.read_adrs(adr_dir) + + assert records == [] + assert warnings[0]["code"] == "BAD_FRONTMATTER" + assert "date" in warnings[0]["detail"] + + +def test_read_exceptions_warns_and_skips_an_invalid_expiry(tmp_path): + exceptions_dir = tmp_path / "exceptions" + exceptions_dir.mkdir() + invalid = { + "id": "EXC-0001", + "adr_id": "ADR-0001", + "rule_id": "r1", + "owner": "team", + "reason": "migration", + "scope": ["src/a.py"], + "created": "2026-01-01", + "expiry": "not-a-date", + } + (exceptions_dir / "0001.json").write_text(json.dumps(invalid), encoding="utf-8") + + records, warnings = adoption_metrics.read_exceptions(tmp_path) + + assert records == [] + assert warnings[0]["code"] == "BAD_EXCEPTION" + assert "expiry" in warnings[0]["detail"] + + def test_decision_lead_time_is_median_completed_cycle_hours(): events = [ _event( @@ -289,6 +323,34 @@ def test_supersession_rate_is_unavailable_when_no_acceptance_is_observed(): assert result["reason"] == "No accepted transitions were observed in the period." +def test_supersession_rate_counts_an_adr_first_observed_as_accepted(): + events = [ + _event( + "adr_created", + "2026-01-02T00:00:00Z", + source="git", + adr_id="ADR-0001", + status="accepted", + ), + _event( + "adr_status_changed", + "2026-01-03T00:00:00Z", + source="git", + adr_id="ADR-0001", + **{"from": "accepted", "to": "superseded"}, + ), + ] + + result = adoption_metrics.calculate_metrics([], [], events, SINCE, UNTIL)[ + "supersession_rate" + ] + + assert result["available"] is True + assert result["accepted"] == 1 + assert result["superseded"] == 1 + assert result["rate"] == 1.0 + + def test_unresolved_violations_use_first_observation_after_latest_resolution(): events = [ _event( @@ -651,3 +713,182 @@ def fail_gh(root, arguments): } ] assert "secret-value" not in json.dumps(warnings) + + +def test_cli_emits_one_json_report_and_degrades_without_optional_history(tmp_path, capsys): + adr_dir = tmp_path / "docs" / "decisions" + adr_dir.mkdir(parents=True) + _write_adr(adr_dir / "0001-test.md", "ADR-0001", "accepted") + + return_code = adoption_metrics.main( + [ + "--root", + str(tmp_path), + "--dir", + "docs/decisions", + "--until", + "2026-01-31", + "--json", + ] + ) + + captured = capsys.readouterr() + result = json.loads(captured.out) + assert return_code == 0 + assert captured.err == "" + assert result["ok"] is True + assert result["operation"] == "adoption_metrics" + assert result["schema_version"] == 1 + assert result["period"] == {"since": "2026-01-01", "until": "2026-01-31"} + assert set(result["metrics"]) == { + "decision_lead_time", + "review_latency", + "supersession_rate", + "unresolved_violations", + "exception_age", + } + assert result["metrics"]["decision_lead_time"]["available"] is False + assert result["metrics"]["exception_age"]["available"] is True + assert [warning["code"] for warning in result["warnings"]] == ["GIT_UNAVAILABLE"] + + +def test_cli_uses_explicit_events_to_calculate_lead_time(tmp_path, capsys): + adr_dir = tmp_path / "docs" / "decisions" + adr_dir.mkdir(parents=True) + _write_adr(adr_dir / "0001-test.md", "ADR-0001", "accepted") + events_path = tmp_path / "events.jsonl" + events_path.write_text( + json.dumps( + _event( + "adr_created", + "2026-01-01T00:00:00Z", + adr_id="ADR-0001", + status="proposed", + ) + ) + + "\n" + + json.dumps( + _event( + "adr_status_changed", + "2026-01-02T00:00:00Z", + adr_id="ADR-0001", + **{"from": "proposed", "to": "accepted"}, + ) + ) + + "\n", + encoding="utf-8", + ) + + return_code = adoption_metrics.main( + [ + "--root", + str(tmp_path), + "--dir", + "docs/decisions", + "--since", + "2026-01-01", + "--until", + "2026-01-31", + "--events", + "events.jsonl", + "--json", + ] + ) + + result = json.loads(capsys.readouterr().out) + assert return_code == 0 + assert result["metrics"]["decision_lead_time"]["median_hours"] == 24.0 + assert result["metrics"]["decision_lead_time"]["sources"] == ["events"] + + +def test_cli_opt_in_github_failure_is_a_warning(tmp_path, capsys, monkeypatch): + adr_dir = tmp_path / "docs" / "decisions" + adr_dir.mkdir(parents=True) + _write_adr(adr_dir / "0001-test.md", "ADR-0001", "accepted") + monkeypatch.setattr( + adoption_metrics, + "collect_github_payload", + lambda root: ( + None, + [{"code": "GITHUB_UNAVAILABLE", "detail": "No authenticated provider."}], + ), + ) + + return_code = adoption_metrics.main( + [ + "--root", + str(tmp_path), + "--dir", + "docs/decisions", + "--until", + "2026-01-31", + "--github", + "--json", + ] + ) + + result = json.loads(capsys.readouterr().out) + assert return_code == 0 + assert [warning["code"] for warning in result["warnings"]] == [ + "GIT_UNAVAILABLE", + "GITHUB_UNAVAILABLE", + ] + + +def test_cli_returns_json_error_for_missing_adr_directory(tmp_path, capsys): + return_code = adoption_metrics.main( + ["--root", str(tmp_path), "--dir", "docs/missing", "--json"] + ) + + captured = capsys.readouterr() + result = json.loads(captured.out) + assert return_code == 1 + assert captured.err == "" + assert result["ok"] is False + assert result["errors"][0]["code"] == "ADR_DIR_NOT_FOUND" + + +def test_cli_returns_json_error_when_adr_directory_escapes_root(tmp_path, capsys): + outside = tmp_path.parent / "outside-decisions" + outside.mkdir(exist_ok=True) + + return_code = adoption_metrics.main( + ["--root", str(tmp_path), "--dir", str(outside), "--json"] + ) + + result = json.loads(capsys.readouterr().out) + assert return_code == 1 + assert result["errors"] == [ + { + "code": "PATH_ESCAPES_ROOT", + "path": str(outside.resolve()), + } + ] + + +def test_cli_returns_json_error_for_invalid_period(tmp_path, capsys): + adr_dir = tmp_path / "docs" / "decisions" + adr_dir.mkdir(parents=True) + + return_code = adoption_metrics.main( + [ + "--root", + str(tmp_path), + "--dir", + "docs/decisions", + "--since", + "2026-02-01", + "--until", + "2026-01-01", + "--json", + ] + ) + + result = json.loads(capsys.readouterr().out) + assert return_code == 1 + assert result["errors"] == [ + { + "code": "INVALID_PERIOD", + "detail": "--since must be earlier than or equal to --until.", + } + ] From 26021a97e503910292560498f8ddb4c9879810bc Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 14:44:32 +0900 Subject: [PATCH 44/58] fix: reject nested-quantifier ReDoS patterns in constraints: blocks statically MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit New _reject_if_redos_prone() in core/constraints.py rejects a `pattern` value at parse time if it contains a quantified group whose own body ends in a quantifier (e.g. (a+)+, (a*)*, (x{1,3})+) -- the classic catastrophic- backtracking shape. This is a static, string-level check, not a runtime guard, so it works identically on every platform and closes the Windows gap left by rules/conflict.py's SIGALRM-based timeout (POSIX-only, confirmed unguarded on Windows per docs/adr-toolkit-audit-report.md §2.2 2.3's Open Risk). Scoped to forbidden_import/dependency_forbidden only -- required_path/ forbidden_path treat `pattern` as glob syntax via core/globs.py, which can't produce catastrophic backtracking, so flagging it there would be a false positive. Verified the real dogfooded ADR-0011 constraints block (genuine regex patterns) still validates and checks cleanly, and that a rejected pattern never reaches re.compile() (monkeypatched re.compile to assert it's never called for a dangerous pattern). This is a heuristic covering the single most common ReDoS shape, not a full detector -- alternation-based patterns like (a|a)* are a different dangerous shape and remain uncaught, noted in the code comment. --- .../adr-toolkit/scripts/core/constraints.py | 35 +++++++++++ tests/unit/test_constraints.py | 62 +++++++++++++++++++ 2 files changed, 97 insertions(+) diff --git a/skills/adr-toolkit/scripts/core/constraints.py b/skills/adr-toolkit/scripts/core/constraints.py index 953c090..9217a99 100644 --- a/skills/adr-toolkit/scripts/core/constraints.py +++ b/skills/adr-toolkit/scripts/core/constraints.py @@ -28,6 +28,26 @@ "test_must_exist", } +# Only these two kinds ever pass their `pattern` values to re.compile() as +# regex (rules/conflict.py::_content_pattern) -- required_path/forbidden_path +# treat `pattern` as glob syntax via core/globs.py instead, which is built +# from a fixed, safe translation and can't produce catastrophic +# backtracking. Kept as a small local set rather than importing +# rules/conflict.py's CONTENT_PATTERN_KINDS, since core/ must not depend on +# rules/ (the opposite direction already holds throughout this codebase). +_REGEX_PATTERN_KINDS = {"forbidden_import", "dependency_forbidden"} + +# A quantified group whose own body ends in a quantifier -- e.g. (a+)+, +# (a*)*, (x{1,3})+ -- is the single most common shape behind catastrophic +# regex backtracking (ReDoS). This is a static, string-level heuristic, not +# a full ReDoS detector: alternation-based patterns like (a|a)* are a +# different dangerous shape and are not caught here. It exists because the +# runtime SIGALRM-based timeout guard in rules/conflict.py is POSIX-only; +# rejecting the pattern here, before it is ever compiled or executed, +# protects Windows too (docs/adr-toolkit-audit-report.md §2.2 2.3). +_QUANTIFIER = r"(?:[+*]|\{\d*,?\d*\})" +_NESTED_QUANTIFIER_RE = re.compile(r"\([^()]*" + _QUANTIFIER + r"\)" + _QUANTIFIER) + class ConstraintsError(AdrToolkitError): error_code = "BAD_CONSTRAINTS" @@ -99,4 +119,19 @@ def _parse_rules(lines) -> list: if current is not None: rules.append(current) + + for rule in rules: + if rule.get("kind") in _REGEX_PATTERN_KINDS: + for pattern in rule.get("pattern", []): + _reject_if_redos_prone(pattern) + return rules + + +def _reject_if_redos_prone(pattern: str) -> None: + if _NESTED_QUANTIFIER_RE.search(pattern): + raise ConstraintsError( + f"pattern {pattern!r} has a nested quantifier and risks catastrophic " + f"backtracking (ReDoS) -- rewrite it without a repeated group inside " + f"another repeated group" + ) diff --git a/tests/unit/test_constraints.py b/tests/unit/test_constraints.py index 5ad06cc..45731f0 100644 --- a/tests/unit/test_constraints.py +++ b/tests/unit/test_constraints.py @@ -1,3 +1,5 @@ +import json + import pytest from scripts.core.constraints import ConstraintsError, extract_constraints, lint @@ -105,3 +107,63 @@ def test_lint_returns_a_bad_constraints_warning_for_a_malformed_block(): warnings = lint(BODY_WITH_UNKNOWN_KIND) assert warnings == [{"code": "BAD_CONSTRAINTS", "detail": warnings[0]["detail"]}] assert "forbidden_imports" in warnings[0]["detail"] + + +def _body_with_pattern(kind: str, pattern: list) -> str: + return ( + "```yaml\nconstraints:\n" + f" - id: r\n kind: {kind}\n paths: [\"src/**\"]\n" + f" pattern: {json.dumps(pattern)}\n" + " severity: major\n message: \"m\"\n```\n" + ) + + +@pytest.mark.parametrize("dangerous_pattern", [ + r"(a+)+$", + r"(a*)*", + r"(a+)*", + r"((a+)+)+", + r"(x{1,3})+", +]) +def test_nested_quantifier_pattern_is_rejected_for_content_pattern_kinds(dangerous_pattern): + for kind in ("forbidden_import", "dependency_forbidden"): + with pytest.raises(ConstraintsError) as exc: + extract_constraints(_body_with_pattern(kind, [dangerous_pattern])) + assert "nested quantifier" in str(exc.value) + + +def test_nested_quantifier_check_never_actually_runs_the_pattern(monkeypatch): + # The whole point is that a dangerous pattern is rejected by inspecting + # the pattern *string* -- it must never reach re.compile()/re.search(), + # which is what makes this protection work identically without a + # runtime timeout (i.e. on Windows, where signal.SIGALRM doesn't exist). + import re as re_module + + def _boom(*args, **kwargs): + raise AssertionError("a rejected pattern must never be compiled") + + monkeypatch.setattr(re_module, "compile", _boom) + + with pytest.raises(ConstraintsError): + extract_constraints(_body_with_pattern("forbidden_import", ["(a+)+"])) + + +def test_ordinary_patterns_are_not_flagged_as_dangerous(): + body = _body_with_pattern("forbidden_import", ["openai", "anthropic", "^foo.*bar$"]) + rules = extract_constraints(body) + assert rules[0]["pattern"] == ["openai", "anthropic", "^foo.*bar$"] + + +def test_nested_quantifier_in_a_non_regex_kind_is_not_rejected(): + # required_path/forbidden_path treat `pattern` as glob syntax (via + # core/globs.py), which can't produce catastrophic backtracking -- + # only forbidden_import/dependency_forbidden compile it as regex. + body = _body_with_pattern("required_path", ["src/(a+)+/registry.py"]) + rules = extract_constraints(body) + assert rules[0]["pattern"] == ["src/(a+)+/registry.py"] + + +def test_lint_reports_nested_quantifier_as_bad_constraints_warning(): + warnings = lint(_body_with_pattern("forbidden_import", ["(a+)+"])) + assert warnings[0]["code"] == "BAD_CONSTRAINTS" + assert "nested quantifier" in warnings[0]["detail"] From c65165863bc9eee7cbeb53d4e70296991480d154 Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 14:45:56 +0900 Subject: [PATCH 45/58] docs: record the Windows ReDoS mitigation, note parallel Codex work exists improvements.md and handoff.md updated for this session's work only (the Windows static-complexity-linter item promoted from Open Risks). Codex's already-committed adoption-metrics commits are acknowledged as present in this branch but not itemized here, per the owner's explicit request not to reflect that in-progress work from this session's docs. --- handoff.md | 35 ++++++++++++++++++++++++++--------- improvements.md | 16 +++++++++++++--- 2 files changed, 39 insertions(+), 12 deletions(-) diff --git a/handoff.md b/handoff.md index 6d8b8d5..df42b98 100644 --- a/handoff.md +++ b/handoff.md @@ -68,6 +68,17 @@ at CHECK time -- `CreateResult`/`StatusResult` gained a `warnings` field to match. The 4th item (Antigravity in `harness-parity`) stays open, blocked on `agy` getting a public package registry. +**Windows ReDoS gap closed** (`26021a9`, promoted from this file's own +Open Risks below, not a numbered backlog item): `core/constraints.py` +statically rejects nested-quantifier `pattern` values (`(a+)+`-shaped) at +parse time for `forbidden_import`/`dependency_forbidden` rules -- a +string-level check that works the same on every OS, unlike +`rules/conflict.py`'s SIGALRM-based runtime timeout (POSIX-only). Verified +against the real dogfooded `ADR-0011` constraints block (no false +positive) and that a rejected pattern never reaches `re.compile()`. +Heuristic, not a full ReDoS detector -- alternation-based shapes like +`(a|a)*` remain uncaught, noted in the code. + **Concurrent work (owner's own coordination, not this session's):** the owner assigned `improvements.md`'s "도입 지표 수집 스크립트" (adoption-metrics script, from `docs/enterprise-adoption.md` §7) to a @@ -201,19 +212,25 @@ on each. ## Verification -`python3 -m pytest tests/unit tests/integration -v` -> 479 passed as of -commit `9a44342` (395 at session start -> 465 before the `origin/develop` +`python3 -m pytest tests/unit tests/integration -v` -> 518 passed as of +commit `26021a9` (395 at session start -> 465 before the `origin/develop` merge -> 469 after merging in develop's own new tests -> 479 after the -Low-priority follow-up work). Re-run to pick up whatever Codex's parallel -adoption-metrics work adds. CI now also runs `type-check` (`mypy ---strict`), `examples-drift` (from develop), and `pr-title-check` (from -develop) jobs alongside the existing `pytest` (now coverage-gated at -85%), `version-drift`, and `harness-parity` jobs. +Low-priority follow-up work -> 518 current, which also includes the +parallel Codex session's adoption-metrics tests landing in this branch). +CI now also runs `type-check` (`mypy --strict`), `examples-drift` (from +develop), and `pr-title-check` (from develop) jobs alongside the existing +`pytest` (now coverage-gated at 85%), `version-drift`, and +`harness-parity` jobs. ## Open risks -- The ReDoS guard is POSIX-only (`signal.SIGALRM`); Windows CI is - unaffected but unguarded -- a known, documented gap. +- ~~The ReDoS guard is POSIX-only...~~ **Mitigated (`26021a9`)**: the + runtime SIGALRM timeout is still POSIX-only, but a static + nested-quantifier check in `core/constraints.py` now rejects the most + common ReDoS shape at parse time on every platform, so Windows is no + longer completely unguarded. Not a full ReDoS detector -- alternation- + based patterns (`(a|a)*`-shaped) still rely on the POSIX-only runtime + guard and remain unmitigated on Windows. - `supersede.py`'s two-file update guarantees each individual file is never torn by a mid-write crash, but not that the *pair* stays consistent if killed between the two writes -- true two-phase commit diff --git a/improvements.md b/improvements.md index 8be570c..1f5ca57 100644 --- a/improvements.md +++ b/improvements.md @@ -115,8 +115,18 @@ instead of only when CHECK later runs against it. The 4th item (Antigravity in harness-parity CI) stays open, blocked on `agy` getting a public package registry. +**Windows ReDoS static complexity linter** (promoted from `handoff.md`'s +Open Risks, not originally a numbered backlog item) — `core/constraints.py` +now statically rejects a nested-quantifier `pattern` value (e.g. `(a+)+`) +at parse time for `forbidden_import`/`dependency_forbidden` rules, closing +the gap where `rules/conflict.py`'s runtime SIGALRM timeout guard is +POSIX-only and Windows had zero ReDoS protection. Heuristic, not a full +detector -- alternation-based ReDoS shapes remain uncaught. + Test suite: 395 → 465 passing (this branch's own work), zero regressions; 469 after merging `origin/develop`; 479 after the Low-priority follow-up -work above. CI gained a `type-check` job and an 85% coverage gate (this -branch), plus `examples-drift` and `pr-title-check` jobs (from -`origin/develop`). +work; 518 as of this note (includes a parallel Codex session's own +adoption-metrics commits landing in this same branch -- see `handoff.md`, +not itemized here since that work isn't this session's to describe). CI +gained a `type-check` job and an 85% coverage gate (this branch), plus +`examples-drift` and `pr-title-check` jobs (from `origin/develop`). From 7f09a5da87da0e26f25f03bc6a1f2354c0c8a1f2 Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 14:54:09 +0900 Subject: [PATCH 46/58] fix: harden adoption metrics evidence handling --- .../2026-09-01-adoption-metrics-design.md | 25 +- scripts/adoption_metrics.py | 278 ++++++++++--- tests/unit/test_adoption_metrics.py | 371 ++++++++++++++++++ 3 files changed, 610 insertions(+), 64 deletions(-) diff --git a/docs/superpowers/specs/2026-09-01-adoption-metrics-design.md b/docs/superpowers/specs/2026-09-01-adoption-metrics-design.md index 6b74a55..30cee1a 100644 --- a/docs/superpowers/specs/2026-09-01-adoption-metrics-design.md +++ b/docs/superpowers/specs/2026-09-01-adoption-metrics-design.md @@ -144,8 +144,9 @@ status-transition events using the commit author timestamp. It does not infer a is excluded from lead-time coverage. Rename following is best effort and emits a warning if an ADR ID changes. -The GitHub collector queries pull requests that touched an ADR file during the -requested interval. A review is qualified when it is submitted by a reviewer +The GitHub collector paginates through pull requests and selects review cycles +completed during the requested interval after normalization. A review is +qualified when it is submitted by a reviewer who was explicitly requested for that pull request; self-review by the PR author does not qualify. The first qualified submitted review after the first review request ends the interval. The normalized event keeps no provider-specific URL @@ -156,6 +157,10 @@ providers can export equivalent `review_requested` and `review_submitted` events. A repository with neither source receives an unavailable review metric, not a guessed value. +If GitHub truncates the file or timeline connection inside any pull request, +the collector discards GitHub review evidence for that run and emits a warning; +partial provider evidence must not produce a confident latency. + ## Metric Definitions ### Decision Lead Time @@ -174,11 +179,12 @@ as open review cycles but are not included in the median. ### Supersession Rate -The numerator is the number of ADRs transitioning to `superseded` within the -interval. The denominator is the number of ADRs transitioning to `accepted` -within the same interval. Report a JSON number in the range 0 through 1, or -`null` when the denominator is zero. Also report both raw counts so consumers do -not over-interpret a small sample. +The denominator is the cohort of ADRs first observed as `accepted` or +transitioning to `accepted` within the interval. The numerator is the members +of that same cohort that also transition to `superseded` within the interval. +Report a JSON number in the range 0 through 1, or `null` when the denominator is +zero. Also report both raw counts so consumers do not over-interpret a small +sample. ### Unresolved Violations @@ -186,8 +192,9 @@ A violation is identified by the stable fingerprint supplied by the CHECK observation producer. It is open after its latest `violation_observed` event and closed after a later `violation_resolved` event. At `until`, report the open count and each open violation's whole-day age from first uninterrupted -observation. If only a current CHECK result exists, count is available but age -is unavailable. Active exceptions remain visible and do not close violations. +observation. `--check-results` is a current snapshot: count is available, but +age is unavailable unless matching historical observations are supplied with +`--events`. Active exceptions remain visible and do not close violations. ### Exception Age diff --git a/scripts/adoption_metrics.py b/scripts/adoption_metrics.py index f2e7c02..f3e2d5f 100644 --- a/scripts/adoption_metrics.py +++ b/scripts/adoption_metrics.py @@ -23,6 +23,18 @@ "created", "expiry", } +EXCEPTION_FIELD_TYPES = { + "id": str, + "adr_id": str, + "rule_id": str, + "owner": str, + "reason": str, + "scope": list, + "created": str, + "expiry": str, +} +EXCEPTION_ID_RE = re.compile(r"^EXC-\d{4}$") +ADR_ID_RE = re.compile(r"^ADR-\d{4}$") EVENT_REQUIRED_FIELDS = { "adr_created": {"adr_id", "status"}, "adr_status_changed": {"adr_id", "from", "to"}, @@ -32,9 +44,9 @@ "violation_resolved": {"fingerprint", "adr_id", "rule_id"}, } GITHUB_REVIEW_QUERY = """ -query($owner: String!, $name: String!) { +query($owner: String!, $name: String!, $cursor: String) { repository(owner: $owner, name: $name) { - pullRequests(first: 100, orderBy: {field: UPDATED_AT, direction: DESC}) { + pullRequests(first: 100, after: $cursor, orderBy: {field: UPDATED_AT, direction: DESC}) { nodes { number author { login } @@ -132,6 +144,22 @@ def read_exceptions(adr_dir: Path) -> Tuple[List[Dict[str, Any]], List[Dict[str, missing = sorted(REQUIRED_EXCEPTION_FIELDS - set(data)) if missing: raise ValueError("missing required field(s): {}".format(", ".join(missing))) + for field, expected_type in EXCEPTION_FIELD_TYPES.items(): + if not isinstance(data[field], expected_type): + raise ValueError( + "field {!r} must be {}, got {}".format( + field, expected_type.__name__, type(data[field]).__name__ + ) + ) + if not EXCEPTION_ID_RE.fullmatch(data["id"]): + raise ValueError("id does not match EXC-NNNN") + if not ADR_ID_RE.fullmatch(data["adr_id"]): + raise ValueError("adr_id does not match ADR-NNNN") + for field in ("owner", "reason", "rule_id"): + if not data[field].strip(): + raise ValueError("{} must not be empty".format(field)) + if not data["scope"]: + raise ValueError("scope must contain at least one path pattern") for field in ("created", "expiry"): try: parse_timestamp(str(data[field])) @@ -167,6 +195,16 @@ def _validate_event(data: Any) -> None: ) - set(data) if missing: raise ValueError("missing required field(s): {}".format(", ".join(sorted(missing)))) + string_fields = EVENT_REQUIRED_FIELDS[str(event_name)] - {"qualified"} + for field in string_fields | {"occurred_at", "source"}: + if not isinstance(data[field], str) or not data[field].strip(): + raise ValueError("field {!r} must be a non-empty string".format(field)) + if event_name == "review_submitted" and not isinstance(data["qualified"], bool): + raise ValueError("field 'qualified' must be bool") + if "review_cycle" in data and ( + not isinstance(data["review_cycle"], str) or not data["review_cycle"].strip() + ): + raise ValueError("field 'review_cycle' must be a non-empty string") parse_timestamp(str(data["occurred_at"])) @@ -210,6 +248,7 @@ def read_events( } ) continue + data["occurred_at"] = parse_timestamp(data["occurred_at"]).isoformat() events.append(data) return events, warnings @@ -231,7 +270,9 @@ def _event_identity(event: Dict[str, Any]) -> Tuple[str, str, str]: def _event_payload(event: Dict[str, Any]) -> Dict[str, Any]: - return {key: value for key, value in event.items() if key != "source"} + payload = {key: value for key, value in event.items() if key != "source"} + payload["occurred_at"] = parse_timestamp(str(event["occurred_at"])).isoformat() + return payload def merge_events( @@ -267,13 +308,18 @@ def merge_events( def _run_git(root: Path, arguments: List[str]) -> subprocess.CompletedProcess: - return subprocess.run( - ["git"] + arguments, - cwd=str(root), - capture_output=True, - text=True, - check=False, - ) + command = ["git"] + arguments + try: + return subprocess.run( + command, + cwd=str(root), + capture_output=True, + encoding="utf-8", + errors="replace", + check=False, + ) + except OSError: + return subprocess.CompletedProcess(command, 127, "", "") def collect_git_events( @@ -284,12 +330,18 @@ def collect_git_events( return [], [ {"code": "GIT_UNAVAILABLE", "detail": "Root is not a Git work tree."} ] + top_result = _run_git(root, ["rev-parse", "--show-toplevel"]) + if top_result.returncode != 0: + return [], [ + {"code": "GIT_UNAVAILABLE", "detail": "Could not resolve Git top-level."} + ] + git_root = Path(top_result.stdout.strip()).resolve() events: List[Dict[str, Any]] = [] warnings: List[Dict[str, Any]] = [] for path in sorted(adr_dir.glob("[0-9]*.md")): try: - relative_path = path.resolve().relative_to(root.resolve()).as_posix() + relative_path = path.resolve().relative_to(git_root).as_posix() except ValueError: warnings.append( { @@ -300,7 +352,7 @@ def collect_git_events( ) continue history = _run_git( - root, + git_root, ["log", "--follow", "--format=%H%x1f%aI", "--", relative_path], ) if history.returncode != 0: @@ -313,12 +365,13 @@ def collect_git_events( ) continue - versions: List[Tuple[str, str, Dict[str, str]]] = [] - for line in reversed([item for item in history.stdout.splitlines() if item]): + versions_newest_first: List[Tuple[str, str, Dict[str, str]]] = [] + historical_path = relative_path + for line in [item for item in history.stdout.splitlines() if item]: commit, separator, timestamp = line.partition("\x1f") if not separator: continue - shown = _run_git(root, ["show", "{}:{}".format(commit, relative_path)]) + shown = _run_git(git_root, ["show", "{}:{}".format(commit, historical_path)]) if shown.returncode != 0: warnings.append( { @@ -344,7 +397,22 @@ def collect_git_events( } ) continue - versions.append((commit, occurred_at, data)) + versions_newest_first.append((commit, occurred_at, data)) + + names = _run_git( + git_root, + ["diff-tree", "--no-commit-id", "--name-status", "-r", "-M", commit], + ) + if names.returncode == 0: + for changed_line in names.stdout.splitlines(): + parts = changed_line.split("\t") + if len(parts) == 3 and parts[0].startswith("R"): + old_name, new_name = parts[1], parts[2] + if new_name == historical_path: + historical_path = old_name + break + + versions = list(reversed(versions_newest_first)) previous_status = None previous_id = None @@ -391,13 +459,18 @@ def collect_git_events( def _run_gh(root: Path, arguments: List[str]) -> subprocess.CompletedProcess: - return subprocess.run( - ["gh"] + arguments, - cwd=str(root), - capture_output=True, - text=True, - check=False, - ) + command = ["gh"] + arguments + try: + return subprocess.run( + command, + cwd=str(root), + capture_output=True, + encoding="utf-8", + errors="replace", + check=False, + ) + except OSError: + return subprocess.CompletedProcess(command, 127, "", "") def collect_github_payload( @@ -422,9 +495,10 @@ def collect_github_payload( } ] - result = _run_gh( - root, - [ + all_nodes: List[Dict[str, Any]] = [] + cursor = None + while True: + arguments = [ "api", "graphql", "-f", @@ -433,24 +507,40 @@ def collect_github_payload( "owner={}".format(owner), "-F", "name={}".format(name), - ], - ) - if result.returncode != 0: - return None, [ - { - "code": "GITHUB_UNAVAILABLE", - "detail": "Could not collect GitHub review history.", - } - ] - try: - return json.loads(result.stdout), [] - except json.JSONDecodeError: - return None, [ - { - "code": "GITHUB_BAD_RESPONSE", - "detail": "GitHub returned invalid review JSON.", - } ] + if cursor is not None: + arguments += ["-F", "cursor={}".format(cursor)] + result = _run_gh(root, arguments) + if result.returncode != 0: + return None, [ + { + "code": "GITHUB_UNAVAILABLE", + "detail": "Could not collect GitHub review history.", + } + ] + try: + page_payload = json.loads(result.stdout) + pull_requests = page_payload["data"]["repository"]["pullRequests"] + all_nodes.extend(pull_requests["nodes"]) + page_info = pull_requests["pageInfo"] + except (json.JSONDecodeError, KeyError, TypeError): + return None, [ + { + "code": "GITHUB_BAD_RESPONSE", + "detail": "GitHub returned invalid review JSON.", + } + ] + if not page_info.get("hasNextPage"): + pull_requests["nodes"] = all_nodes + return page_payload, [] + cursor = page_info.get("endCursor") + if not cursor: + return None, [ + { + "code": "GITHUB_BAD_RESPONSE", + "detail": "GitHub pagination omitted its next cursor.", + } + ] def _reviewer_login(node: Any) -> Any: @@ -482,12 +572,14 @@ def normalize_github_reviews( ) events: List[Dict[str, Any]] = [] + incomplete = False for pull_request in nodes: files = pull_request.get("files", {}) timeline = pull_request.get("timelineItems", {}) if files.get("pageInfo", {}).get("hasNextPage") or timeline.get( "pageInfo", {} ).get("hasNextPage"): + incomplete = True warnings.append( { "code": "GITHUB_PR_RESULTS_TRUNCATED", @@ -496,6 +588,7 @@ def normalize_github_reviews( ), } ) + continue adr_ids = sorted( { adr_paths[file_node.get("path")] @@ -506,6 +599,7 @@ def normalize_github_reviews( if not adr_ids: continue author = _reviewer_login(pull_request.get("author")) + review_cycle = "github-pr-{}".format(pull_request.get("number")) requested = set() timeline_nodes = sorted( timeline.get("nodes", []), @@ -527,6 +621,7 @@ def normalize_github_reviews( "source": "github", "adr_id": adr_id, "reviewer": reviewer, + "review_cycle": review_cycle, } ) elif node.get("__typename") == "PullRequestReview": @@ -545,8 +640,11 @@ def normalize_github_reviews( "adr_id": adr_id, "reviewer": reviewer, "qualified": qualified, + "review_cycle": review_cycle, } ) + if incomplete: + return [], warnings return sorted(events, key=_event_time), warnings @@ -565,7 +663,10 @@ def _coverage(eligible: int, measured: int) -> Dict[str, Any]: def _decision_lead_time( - events: List[Dict[str, Any]], since: datetime, until: datetime + adrs: List[Dict[str, Any]], + events: List[Dict[str, Any]], + since: datetime, + until: datetime, ) -> Dict[str, Any]: by_adr: Dict[str, List[Dict[str, Any]]] = {} for event in events: @@ -583,10 +684,10 @@ def _decision_lead_time( status = event.get("status") if event["event"] == "adr_created" else event.get("to") if status == "proposed" and proposed_at is None: proposed_at = _event_time(event) - if status in {"accepted", "rejected"} and _in_period(event, since, until): + if status in {"accepted", "rejected"}: outcome = event break - if outcome is None: + if outcome is None or not _in_period(outcome, since, until): continue eligible += 1 if proposed_at is not None and proposed_at <= _event_time(outcome): @@ -597,6 +698,16 @@ def _decision_lead_time( if proposed_at <= _event_time(event) <= _event_time(outcome) ) + event_adr_ids = set(by_adr) + for adr in adrs: + if str(adr.get("id")) in event_adr_ids: + continue + if adr.get("status") not in {"accepted", "rejected", "superseded"}: + continue + observed_at = parse_timestamp(str(adr.get("date"))) + if since <= observed_at <= until: + eligible += 1 + result: Dict[str, Any] = { "available": bool(durations), "median_hours": statistics.median(durations) if durations else None, @@ -617,13 +728,21 @@ def _review_latency( durations: List[float] = [] open_cycles = 0 sources = set() - for request in sorted(requests, key=_event_time): - if not (since <= _event_time(request) <= until): + cycles: Dict[Tuple[str, str], List[Dict[str, Any]]] = {} + for request in requests: + adr_id = str(request.get("adr_id")) + cycle_id = str(request.get("review_cycle") or adr_id) + cycles.setdefault((adr_id, cycle_id), []).append(request) + + for (adr_id, cycle_id), cycle_requests in cycles.items(): + request = min(cycle_requests, key=_event_time) + if _event_time(request) > until: continue candidates = [ event for event in submissions - if event.get("adr_id") == request.get("adr_id") + if str(event.get("adr_id")) == adr_id + and str(event.get("review_cycle") or adr_id) == cycle_id and event.get("qualified") is True and _event_time(event) >= _event_time(request) and _event_time(event) <= until @@ -633,10 +752,13 @@ def _review_latency( sources.add(str(request.get("source"))) continue submitted = min(candidates, key=_event_time) + if _event_time(submitted) < since: + continue durations.append( (_event_time(submitted) - _event_time(request)).total_seconds() / 3600 ) - sources.update((str(request.get("source")), str(submitted.get("source")))) + sources.update(str(item.get("source")) for item in cycle_requests) + sources.add(str(submitted.get("source"))) eligible = len(durations) + open_cycles result: Dict[str, Any] = { @@ -667,11 +789,12 @@ def _supersession_rate( if event.get("to") == "accepted" or (event.get("event") == "adr_created" and event.get("status") == "accepted") } - superseded_ids = { + superseded_in_period = { str(event.get("adr_id")) for event in lifecycle_events if event.get("to") == "superseded" } + superseded_ids = superseded_in_period & accepted_ids accepted = len(accepted_ids) superseded = len(superseded_ids) sources = sorted( @@ -713,9 +836,31 @@ def _unresolved_violations( "reason": "No CHECK violation observations were available.", } + current_events = [ + event for event in violation_events if event.get("observation_mode") == "current" + ] + historical_events = [ + event for event in violation_events if event.get("observation_mode") != "current" + ] + current_fingerprints = { + str(event.get("fingerprint")) + for event in current_events + if event.get("event") == "violation_observed" + } + if current_events and not historical_events: + return { + "available": True, + "open_count": len(current_fingerprints), + "age_available": False, + "median_age_days": None, + "max_age_days": None, + "sources": sorted({str(event.get("source")) for event in current_events}), + "reason": "Current CHECK results have no historical first-observed evidence.", + } + open_since: Dict[str, datetime] = {} sources = set() - for event in sorted(violation_events, key=_event_time): + for event in sorted(historical_events, key=_event_time): fingerprint = str(event.get("fingerprint")) sources.add(str(event.get("source"))) if event["event"] == "violation_resolved": @@ -723,6 +868,22 @@ def _unresolved_violations( elif fingerprint not in open_since: open_since[fingerprint] = _event_time(event) + if current_events: + sources.update(str(event.get("source")) for event in current_events) + if not current_fingerprints.issubset(open_since): + return { + "available": True, + "open_count": len(current_fingerprints), + "age_available": False, + "median_age_days": None, + "max_age_days": None, + "sources": sorted(sources), + "reason": "Some current violations lack historical first-observed evidence.", + } + open_since = { + fingerprint: open_since[fingerprint] for fingerprint in current_fingerprints + } + ages = [(until.date() - opened.date()).days for opened in open_since.values()] return { "available": True, @@ -742,6 +903,8 @@ def _exception_age( for exception in exceptions: created = parse_timestamp(str(exception["created"])).date() expiry = parse_timestamp(str(exception["expiry"])).date() + if created > until.date(): + continue if expiry < until.date(): expired_count += 1 else: @@ -763,9 +926,8 @@ def calculate_metrics( since: datetime, until: datetime, ) -> Dict[str, Any]: - del adrs # Current ADR snapshots are retained for future coverage extensions. return { - "decision_lead_time": _decision_lead_time(events, since, until), + "decision_lead_time": _decision_lead_time(adrs, events, since, until), "review_latency": _review_latency(events, since, until), "supersession_rate": _supersession_rate(events, since, until), "unresolved_violations": _unresolved_violations(events, until), @@ -825,7 +987,11 @@ def build_report( adrs, adr_warnings = read_adrs(adr_dir) exceptions, exception_warnings = read_exceptions(adr_dir) - explicit_events, explicit_warnings = read_events(event_paths + check_paths) + explicit_events, explicit_warnings = read_events(event_paths) + check_events, check_warnings = read_events(check_paths) + for event in check_events: + event["source"] = "check_results" + event["observation_mode"] = "current" git_events, git_warnings = collect_git_events(root, adr_dir) github_events: List[Dict[str, Any]] = [] @@ -845,6 +1011,7 @@ def build_report( events, merge_warnings = merge_events( [ ("events", explicit_events), + ("check_results", check_events), ("git", git_events), ("github", github_events), ] @@ -873,6 +1040,7 @@ def build_report( adr_warnings + exception_warnings + explicit_warnings + + check_warnings + git_warnings + github_warnings + merge_warnings diff --git a/tests/unit/test_adoption_metrics.py b/tests/unit/test_adoption_metrics.py index 2ef208c..eaa6fef 100644 --- a/tests/unit/test_adoption_metrics.py +++ b/tests/unit/test_adoption_metrics.py @@ -143,6 +143,40 @@ def test_read_exceptions_warns_and_skips_an_invalid_expiry(tmp_path): assert "expiry" in warnings[0]["detail"] +def test_read_exceptions_rejects_wrong_types_and_invalid_ids(tmp_path): + exceptions_dir = tmp_path / "exceptions" + exceptions_dir.mkdir() + invalid = { + "id": "bad-id", + "adr_id": "bad-adr", + "rule_id": ["r1"], + "owner": ["team"], + "reason": "migration", + "scope": "src/a.py", + "created": "2026-01-01", + "expiry": "2026-02-01", + } + (exceptions_dir / "0001.json").write_text(json.dumps(invalid), encoding="utf-8") + + records, warnings = adoption_metrics.read_exceptions(tmp_path) + + assert records == [] + assert warnings[0]["code"] == "BAD_EXCEPTION" + + +def test_exception_created_after_report_date_is_not_counted_as_negative_age(): + exceptions = [ + {"id": "EXC-0001", "created": "2026-02-01", "expiry": "2026-03-01"} + ] + + result = adoption_metrics.calculate_metrics([], exceptions, [], SINCE, UNTIL)[ + "exception_age" + ] + + assert result["active_count"] == 0 + assert result["expired_count"] == 0 + + def test_decision_lead_time_is_median_completed_cycle_hours(): events = [ _event( @@ -205,6 +239,53 @@ def test_decision_lead_time_excludes_terminal_first_observation_from_coverage(): assert result["reason"] == "No completed decision had observable proposed history." +def test_decision_lead_time_uses_current_terminal_adrs_for_coverage_without_history(): + adrs = [ + { + "id": "ADR-0001", + "title": "A", + "status": "accepted", + "date": "2026-01-02", + "file": "0001-a.md", + } + ] + + result = adoption_metrics.calculate_metrics(adrs, [], [], SINCE, UNTIL)[ + "decision_lead_time" + ] + + assert result["coverage"] == {"eligible": 1, "measured": 0, "ratio": 0.0} + + +def test_decision_lead_time_does_not_replace_pre_period_first_outcome(): + events = [ + _event( + "adr_created", + "2025-12-01T00:00:00Z", + adr_id="ADR-0001", + status="proposed", + ), + _event( + "adr_status_changed", + "2025-12-02T00:00:00Z", + adr_id="ADR-0001", + **{"from": "proposed", "to": "accepted"}, + ), + _event( + "adr_status_changed", + "2026-01-02T00:00:00Z", + adr_id="ADR-0001", + **{"from": "deprecated", "to": "accepted"}, + ), + ] + + result = adoption_metrics.calculate_metrics([], [], events, SINCE, UNTIL)[ + "decision_lead_time" + ] + + assert result["coverage"] == {"eligible": 0, "measured": 0, "ratio": None} + + def test_review_latency_uses_first_qualified_review_after_request(): events = [ _event( @@ -271,6 +352,72 @@ def test_review_latency_reports_open_cycle_without_adding_it_to_median(): assert result["coverage"] == {"eligible": 2, "measured": 1, "ratio": 0.5} +def test_review_latency_filters_completed_cycles_by_submission_time(): + events = [ + _event( + "review_requested", + "2026-01-09T00:00:00Z", + adr_id="ADR-0001", + reviewer="alice", + review_cycle="pr-7", + ), + _event( + "review_submitted", + "2026-01-11T00:00:00Z", + adr_id="ADR-0001", + reviewer="alice", + qualified=True, + review_cycle="pr-7", + ), + ] + + result = adoption_metrics.calculate_metrics( + [], + [], + events, + datetime(2026, 1, 10, tzinfo=timezone.utc), + UNTIL, + )["review_latency"] + + assert result["sample_size"] == 1 + assert result["median_hours"] == 48.0 + + +def test_review_latency_groups_multiple_requested_reviewers_into_one_cycle(): + events = [ + _event( + "review_requested", + "2026-01-01T00:00:00Z", + adr_id="ADR-0001", + reviewer="alice", + review_cycle="pr-7", + ), + _event( + "review_requested", + "2026-01-01T01:00:00Z", + adr_id="ADR-0001", + reviewer="bob", + review_cycle="pr-7", + ), + _event( + "review_submitted", + "2026-01-01T06:00:00Z", + adr_id="ADR-0001", + reviewer="bob", + qualified=True, + review_cycle="pr-7", + ), + ] + + result = adoption_metrics.calculate_metrics([], [], events, SINCE, UNTIL)[ + "review_latency" + ] + + assert result["sample_size"] == 1 + assert result["median_hours"] == 6.0 + assert result["coverage"] == {"eligible": 1, "measured": 1, "ratio": 1.0} + + def test_supersession_rate_uses_transitions_within_period(): events = [ _event( @@ -351,6 +498,49 @@ def test_supersession_rate_counts_an_adr_first_observed_as_accepted(): assert result["rate"] == 1.0 +def test_supersession_rate_excludes_supersessions_outside_period_accepted_cohort(): + events = [ + _event( + "adr_created", + "2025-12-01T00:00:00Z", + adr_id="ADR-OLD-1", + status="accepted", + ), + _event( + "adr_created", + "2025-12-02T00:00:00Z", + adr_id="ADR-OLD-2", + status="accepted", + ), + _event( + "adr_created", + "2026-01-02T00:00:00Z", + adr_id="ADR-NEW", + status="accepted", + ), + _event( + "adr_status_changed", + "2026-01-03T00:00:00Z", + adr_id="ADR-OLD-1", + **{"from": "accepted", "to": "superseded"}, + ), + _event( + "adr_status_changed", + "2026-01-04T00:00:00Z", + adr_id="ADR-OLD-2", + **{"from": "accepted", "to": "superseded"}, + ), + ] + + result = adoption_metrics.calculate_metrics([], [], events, SINCE, UNTIL)[ + "supersession_rate" + ] + + assert result["accepted"] == 1 + assert result["superseded"] == 0 + assert result["rate"] == 0.0 + + def test_unresolved_violations_use_first_observation_after_latest_resolution(): events = [ _event( @@ -504,6 +694,47 @@ def test_read_events_rejects_unknown_event_and_missing_required_payload(tmp_path ] +def test_read_events_rejects_wrong_payload_types(tmp_path): + events_path = tmp_path / "events.jsonl" + events_path.write_text( + json.dumps( + _event( + "review_submitted", + "2026-01-01T00:00:00Z", + adr_id="ADR-0001", + reviewer="alice", + qualified="yes", + ) + ) + + "\n", + encoding="utf-8", + ) + + events, warnings = adoption_metrics.read_events([events_path]) + + assert events == [] + assert warnings[0]["code"] == "BAD_EVENT_SCHEMA" + + +def test_merge_events_treats_equivalent_utc_timestamp_spellings_as_duplicates(): + explicit = _event( + "adr_created", + "2026-01-01T00:00:00Z", + adr_id="ADR-0001", + status="accepted", + ) + reconstructed = dict( + explicit, occurred_at="2026-01-01T00:00:00+00:00", source="git" + ) + + events, warnings = adoption_metrics.merge_events( + [("events", [explicit]), ("git", [reconstructed])] + ) + + assert len(events) == 1 + assert warnings == [] + + def test_merge_events_deduplicates_and_prefers_explicit_conflicting_payload(): explicit = _event( "adr_created", @@ -593,6 +824,33 @@ def test_collect_git_events_does_not_invent_proposed_for_terminal_first_commit(t ) +def test_collect_git_events_follows_rename_inside_a_nested_project_root(tmp_path): + repository = tmp_path / "repository" + root = repository / "nested-project" + adr_dir = root / "docs" / "decisions" + adr_dir.mkdir(parents=True) + _git(repository, "init") + _git(repository, "config", "user.name", "Test") + _git(repository, "config", "user.email", "test@example.com") + old_path = adr_dir / "0001-old.md" + new_path = adr_dir / "0001-new.md" + _write_adr(old_path, "ADR-0001", "proposed") + _git(repository, "add", str(old_path.relative_to(repository))) + _git(repository, "commit", "-m", "propose", timestamp="2026-01-01T00:00:00Z") + _git(repository, "mv", str(old_path.relative_to(repository)), str(new_path.relative_to(repository))) + _write_adr(new_path, "ADR-0001", "accepted") + _git(repository, "add", str(new_path.relative_to(repository))) + _git(repository, "commit", "-m", "rename and accept", timestamp="2026-01-02T00:00:00Z") + + events, warnings = adoption_metrics.collect_git_events(root, adr_dir) + + assert warnings == [] + assert [(event["event"], event.get("status"), event.get("to")) for event in events] == [ + ("adr_created", "proposed", None), + ("adr_status_changed", None, "accepted"), + ] + + def test_collect_git_events_degrades_cleanly_outside_a_git_repository(tmp_path): adr_dir = tmp_path / "docs" / "decisions" adr_dir.mkdir(parents=True) @@ -608,6 +866,21 @@ def test_collect_git_events_degrades_cleanly_outside_a_git_repository(tmp_path): ] +def test_collect_git_events_degrades_when_git_executable_is_missing(tmp_path, monkeypatch): + adr_dir = tmp_path / "docs" / "decisions" + adr_dir.mkdir(parents=True) + + def missing_executable(*args, **kwargs): + raise FileNotFoundError("git") + + monkeypatch.setattr(adoption_metrics.subprocess, "run", missing_executable) + + events, warnings = adoption_metrics.collect_git_events(tmp_path, adr_dir) + + assert events == [] + assert warnings[0]["code"] == "GIT_UNAVAILABLE" + + def test_normalize_github_reviews_qualifies_requested_non_author_reviewer(): payload = { "data": { @@ -715,6 +988,62 @@ def fail_gh(root, arguments): assert "secret-value" not in json.dumps(warnings) +def test_collect_github_payload_degrades_when_gh_executable_is_missing( + tmp_path, monkeypatch +): + def missing_executable(*args, **kwargs): + raise FileNotFoundError("gh") + + monkeypatch.setattr(adoption_metrics.subprocess, "run", missing_executable) + + payload, warnings = adoption_metrics.collect_github_payload(tmp_path) + + assert payload is None + assert warnings[0]["code"] == "GITHUB_UNAVAILABLE" + + +def test_collect_github_payload_paginates_until_all_pull_requests_are_loaded( + tmp_path, monkeypatch +): + first_page = { + "data": { + "repository": { + "pullRequests": { + "nodes": [{"number": 1}], + "pageInfo": {"hasNextPage": True, "endCursor": "CURSOR-1"}, + } + } + } + } + second_page = { + "data": { + "repository": { + "pullRequests": { + "nodes": [{"number": 2}], + "pageInfo": {"hasNextPage": False, "endCursor": None}, + } + } + } + } + + def fake_gh(root, arguments): + if arguments[:2] == ["repo", "view"]: + stdout = json.dumps({"nameWithOwner": "owner/repo"}) + elif any(value == "cursor=CURSOR-1" for value in arguments): + stdout = json.dumps(second_page) + else: + stdout = json.dumps(first_page) + return subprocess.CompletedProcess(["gh", *arguments], 0, stdout, "") + + monkeypatch.setattr(adoption_metrics, "_run_gh", fake_gh) + + payload, warnings = adoption_metrics.collect_github_payload(tmp_path) + + nodes = payload["data"]["repository"]["pullRequests"]["nodes"] + assert [node["number"] for node in nodes] == [1, 2] + assert warnings == [] + + def test_cli_emits_one_json_report_and_degrades_without_optional_history(tmp_path, capsys): adr_dir = tmp_path / "docs" / "decisions" adr_dir.mkdir(parents=True) @@ -801,6 +1130,48 @@ def test_cli_uses_explicit_events_to_calculate_lead_time(tmp_path, capsys): assert result["metrics"]["decision_lead_time"]["sources"] == ["events"] +def test_cli_current_check_results_report_count_without_inventing_age(tmp_path, capsys): + adr_dir = tmp_path / "docs" / "decisions" + adr_dir.mkdir(parents=True) + _write_adr(adr_dir / "0001-test.md", "ADR-0001", "accepted") + check_path = tmp_path / "check.jsonl" + check_path.write_text( + json.dumps( + _event( + "violation_observed", + "2026-01-31T00:00:00Z", + adr_id="ADR-0001", + rule_id="r1", + fingerprint="ADR-0001:r1:src/a.py", + ) + ) + + "\n", + encoding="utf-8", + ) + + return_code = adoption_metrics.main( + [ + "--root", + str(tmp_path), + "--dir", + "docs/decisions", + "--until", + "2026-01-31", + "--check-results", + "check.jsonl", + "--json", + ] + ) + + result = json.loads(capsys.readouterr().out) + violations = result["metrics"]["unresolved_violations"] + assert return_code == 0 + assert violations["open_count"] == 1 + assert violations["age_available"] is False + assert violations["median_age_days"] is None + assert violations["sources"] == ["check_results"] + + def test_cli_opt_in_github_failure_is_a_warning(tmp_path, capsys, monkeypatch): adr_dir = tmp_path / "docs" / "decisions" adr_dir.mkdir(parents=True) From e7829d11e917e0a5ef42e6a764aa3c5702f9f289 Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 14:58:23 +0900 Subject: [PATCH 47/58] docs: reconcile High-priority backlog against the merged agy/version-sync work MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit PR #6 (feature/agy-plugin-implements-2) and PR #7 (feature/add-githooks) -- the "다른 워크트리" that improvements.md's 2 High items were deferred to -- are both already merged into origin/develop and pulled into this branch (0a0db8a). Re-checked both items against actual current code: - 8.4 auto-version-direction review: genuinely done. sync_version.py and release.yml still do manual-only version bumps, no conflict with the audit's recommendation. Closed, moved to Done. - Supply-chain signing: genuinely still unimplemented (release.yml has no checksum/signing step). Stays Open, stale "다른 워크트리 확인" framing removed since that worktree's work is already merged -- now startable here, though it touches the release pipeline so needs owner confirmation before starting. - Antigravity/harness-parity Low item: re-verified still correctly blocked (agy still has no public registry per its README) -- confirms not everything tied to that merged worktree is automatically resolved. Does not touch or itemize the parallel Codex session's adoption-metrics work, per explicit instruction. --- handoff.md | 89 ++++++++++++++++++++++++++++++------------------- improvements.md | 44 ++++++++++++++---------- 2 files changed, 82 insertions(+), 51 deletions(-) diff --git a/handoff.md b/handoff.md index df42b98..c256c03 100644 --- a/handoff.md +++ b/handoff.md @@ -79,6 +79,21 @@ positive) and that a rejected pattern never reaches `re.compile()`. Heuristic, not a full ReDoS detector -- alternation-based shapes like `(a|a)*` remain uncaught, noted in the code. +**Backlog reconciliation** (docs-only, no commit yet as of writing this): +the owner pointed out that PR #6 (`feature/agy-plugin-implements-2`) and +PR #7 (`feature/add-githooks`) -- the "다른 워크트리" that the High +section's 2 items were deferred to -- are both already merged into +`origin/develop` and pulled into this branch. Re-checked both items +against the actual current code rather than assuming: the 8.4 +auto-version-direction review is genuinely done (no conflict found, +moved to `## Done`) and removed from Open; the supply-chain signing item +is genuinely still unimplemented (confirmed by reading `release.yml`) and +stays Open, just with the stale "다른 워크트리 확인" framing removed. +Also re-verified the Antigravity/harness-parity Low item is still +correctly blocked (agy still has no public registry, per +`adapters/antigravity/README.md`) -- not everything merged from that +worktree closes every item tied to it. + **Concurrent work (owner's own coordination, not this session's):** the owner assigned `improvements.md`'s "도입 지표 수집 스크립트" (adoption-metrics script, from `docs/enterprise-adoption.md` §7) to a @@ -151,52 +166,58 @@ code again: ## Scope for this worktree -Excluded here, being handled elsewhere -- do not touch: - - Domains 1 (core/plugin architecture) and 5 (governance/FSM) from the - audit report. -- Anything Antigravity (`agy`) adapter-related -- another branch (now - merged into `develop` as of this session's merge -- see above). -- Automatic version sync -- another worktree; **do not touch - `.github/workflows/release.yml` for any reason.** The 2 remaining - "(다른 워크트리 확인)" items in `improvements.md`'s `### High` section - are deliberately left there for that other worktree. + audit report -- still out of scope, already scored well. +- **The Antigravity (`agy`) adapter and automatic-version-sync worktree + no longer exists as a separate concern** -- its work merged via GitHub + PR #6/#7 into `origin/develop`, which this branch pulled in (`0a0db8a`). + Re-verified against the actual merged code (not assumed): confirmed + `scripts/sync_version.py`/`release.yml` still do manual-only version + bumps (no conflict with the audit's recommendation -- that review item + is now closed, see `improvements.md`'s `## Done`), and confirmed + `.github/workflows/release.yml` still has no supply-chain checksum/ + signing step (that item is now open and startable in this worktree, + not blocked by a concurrent editor anymore -- but touches the release + pipeline, so confirm with the owner before starting). - README prose (root README.md, `adapters/*/README.md` content) -- - another worktree. Every fix across all 3 passes that touched adapter or - generator code was a code fix, not README prose. + still another worktree's; every fix across all passes that touched + adapter or generator code was a code fix, not README prose. +- **Do not touch what the parallel Codex session is doing** (the + adoption-metrics collector -- already 4 commits in as of `45b3472`, + see below). Don't revert, refactor, or duplicate its work. ## Next step (for a new session picking this up cold) -**Only one small item is truly open in this worktree's own scope** -(the Antigravity/harness-parity one below); everything else left in -`improvements.md` is either someone else's worktree, a precondition-gated -enterprise-adoption item, or in flight in a parallel Codex session (see -above). Concretely: +**One real, startable item exists in `improvements.md`'s `## Open`**: +supply-chain checksums/signing for `.github/workflows/release.yml` +(§2.2 2.2) -- verified not implemented, no longer blocked by a separate +worktree, but touches the release pipeline so confirm with the owner +before starting. Everything else is either precondition-gated or the +parallel Codex session's. Concretely: -1. `improvements.md`'s `## Open` → `### High` still has exactly 2 items - left, both flagged `(다른 워크트리 확인)` -- still someone else's. - Do not start them here. +1. `improvements.md`'s `### High` now has exactly 1 item (the + supply-chain one above) -- startable with owner confirmation, since + it modifies `release.yml`. 2. `improvements.md`'s `### Low` → audit-report sub-group has exactly 1 - item left (Antigravity in `harness-parity`), blocked on an external - fact (agy public registry support) -- don't start it, just note it's - blocked if asked. + item left (Antigravity in `harness-parity`), re-verified against + `adapters/antigravity/README.md` and still blocked on an external fact + (agy has no public package registry) -- don't start it. 3. `improvements.md`'s `### Low` → enterprise-adoption.md sub-group: check - whether the Codex session's adoption-metrics work has landed and been - checked off before assuming it's still open. The other 3 items there - remain precondition-gated (repository going public, 2+ maintainers, - 2+ repositories) -- **not pure code tasks**, don't "implement" a - GitHub ruleset change or multi-repo tooling against a single private - repo's reality. + whether the Codex session's adoption-metrics work has been checked off + before assuming it's still open (as of this note it's implemented -- + `9a0de45`..`45b3472` -- but not yet reflected in `improvements.md` + since this session was told not to touch that item's bookkeeping). The + other 3 items there remain precondition-gated (repository going + public, 2+ maintainers, 2+ repositories) -- **not pure code tasks**. 4. If the user says "continue" / "다음 작업 진행해줘" without naming a - task: at this point the honest answer may be "nothing is open here" -- - say so and ask what's next, rather than inventing scope. + task: the supply-chain item is the one thing to offer; otherwise ask + what's next rather than inventing scope. 5. If the user wants to finish this branch (merge to `develop` / open a PR): that decision was deferred every time it came up this session (owner chose "keep as-is" each time) -- ask again fresh, don't assume - the answer carried forward. Note this branch now includes the merged - `origin/develop` history (see above), so a future merge/PR back to - `develop` should be a clean fast-forward-friendly merge, not a repeat - of this session's conflict resolution. + the answer carried forward. This branch already includes the merged + `origin/develop` history, so a future merge/PR back to `develop` + should be a clean fast-forward-friendly merge. 6. If the user references a new audit finding or a fresh problem: that's genuinely new work -- use the same pattern this session established (writing-plans -> executing-plans, TDD, one commit per task, verify diff --git a/improvements.md b/improvements.md index 1f5ca57..8d0d448 100644 --- a/improvements.md +++ b/improvements.md @@ -7,22 +7,21 @@ Concrete implementation backlog. Unscheduled product bets belong in Backlog derived from `docs/adr-toolkit-audit-report.md`. Scope for this worktree excludes domains 1 (core/plugin architecture) and 5 (governance/ -FSM) — already scored 72/80 and mostly "no action needed" in the audit — -plus anything Antigravity-adapter-related, automatic version sync, and -README prose, which are being handled in other worktrees/branches. The two -items below flagged "(다른 워크트리 확인)" touch files those efforts may -also touch. +FSM) — already scored 72/80 and mostly "no action needed" in the audit. +The Antigravity-adapter and automatic-version-sync work that used to be +a separate worktree has since been merged via GitHub PR #6/#7 into +`origin/develop`, which this branch pulled in (`0a0db8a`) — "다른 +워크트리 확인" items were re-checked against that merged code and either +closed out or reworded below. README prose is still another worktree's. ### High -- [ ] *(다른 워크트리 확인)* **공급망 보안(체크섬/서명)** — - `.github/workflows/release.yml`에 SHA-256/Sigstore 서명 단계. 자동 - 버전 동기화 작업이 같은 파일을 건드릴 수 있어 그쪽에 붙이는 것을 권장. - (감사 보고서 §2.2 2.2) -- [ ] *(다른 워크트리 확인)* **8.4 자동 버전 산정 방향 재검토** — 감사 - 보고서 원 권고는 "semantic-release류 자동 버전 산정 강제 도입 - 비권장"이었음. 진행 중인 자동 버전 동기화 작업 방향과 배치되지 않는지 - 확인. (감사 보고서 §2.8 8.4) +- [ ] **공급망 보안(체크섬/서명)** — `.github/workflows/release.yml` + 확인 결과 여전히 테스트/버전 체크/릴리스 생성만 있고 SHA-256/Sigstore + 서명 단계는 없음. 더 이상 다른 워크트리가 이 파일을 동시에 만지고 + 있지 않으므로(그 작업은 이미 병합됨) 이제 이 워크트리에서 진행 가능 — + 다만 릴리스 파이프라인(운영 인프라)을 건드리는 작업이라 시작 전 + 오너 확인 필요. (감사 보고서 §2.2 2.2) ### Medium @@ -44,10 +43,10 @@ also touch. 중 3건 완료(`0307a1c`, `9a44342`). 남은 건 1건뿐:** - [ ] *(전제조건: Antigravity CLI가 공개 패키지 레지스트리 지원)* - **harness-parity CI에 Antigravity 편입** — 현재 Codex/Gemini만 CI에서 - 실제 설치까지 검증하고 Antigravity(`agy`)는 README에 "수동 검증"으로 - 명시됨. `agy` 작업 자체는 다른 브랜치 소관이라 이 항목도 그쪽과 함께 - 검토. (감사 보고서 §2.1 1.2) + **harness-parity CI에 Antigravity 편입** — agy 관련 작업(PR #6)은 + 이미 병합됐지만 `adapters/antigravity/README.md`를 재확인한 결과 여전히 + "Manually verified"뿐, agy 자체가 아직 공개 패키지 레지스트리를 + 지원하지 않음 — 전제조건 그대로 미충족. (감사 보고서 §2.1 1.2) **출처: `docs/enterprise-adoption.md` §4/§6-9** — 코드/아키텍처 감사와는 별개의, 조직 도입·거버넌스 성숙도를 다루는 문서. 아래 항목 대부분은 @@ -123,6 +122,17 @@ the gap where `rules/conflict.py`'s runtime SIGALRM timeout guard is POSIX-only and Windows had zero ReDoS protection. Heuristic, not a full detector -- alternation-based ReDoS shapes remain uncaught. +**8.4 자동 버전 산정 방향 재검토 — reviewed, closed, no code change.** +The Antigravity/version-sync worktree's work merged in via PR #6/#7 +(`origin/develop`, pulled into this branch at `0a0db8a`). Re-checked +`scripts/sync_version.py` and `.github/workflows/release.yml` against the +audit's original recommendation ("manual version bump, don't force +semantic-release-style automation") -- confirmed no conflict: versioning +is still manual (`VERSION` file + `sync_version.py --check`), no +auto-bump tooling was introduced. Removed from Open; the supply-chain +signing item stays open (verified not implemented) but no longer flagged +"다른 워크트리 확인" since that worktree's work is already merged. + Test suite: 395 → 465 passing (this branch's own work), zero regressions; 469 after merging `origin/develop`; 479 after the Low-priority follow-up work; 518 as of this note (includes a parallel Codex session's own From 7fb373e2e0cdbe2321bd3bd606baf111072db0f5 Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 15:01:12 +0900 Subject: [PATCH 48/58] fix: complete adoption metrics source contracts --- .../2026-09-01-adoption-metrics-design.md | 13 +- scripts/adoption_metrics.py | 127 +++++++++++------- tests/unit/test_adoption_metrics.py | 125 +++++++++++++++++ 3 files changed, 215 insertions(+), 50 deletions(-) diff --git a/docs/superpowers/specs/2026-09-01-adoption-metrics-design.md b/docs/superpowers/specs/2026-09-01-adoption-metrics-design.md index 30cee1a..0629c3e 100644 --- a/docs/superpowers/specs/2026-09-01-adoption-metrics-design.md +++ b/docs/superpowers/specs/2026-09-01-adoption-metrics-design.md @@ -124,8 +124,8 @@ Supported event names are: | --- | --- | --- | | `adr_created` | `adr_id`, `status` | Establish first known lifecycle state | | `adr_status_changed` | `adr_id`, `from`, `to` | Establish decision and supersession transitions | -| `review_requested` | `adr_id`, `reviewer` | Start review latency | -| `review_submitted` | `adr_id`, `reviewer`, `qualified` | End review latency at first qualified review | +| `review_requested` | `adr_id`, `reviewer`, `review_cycle` | Start review latency | +| `review_submitted` | `adr_id`, `reviewer`, `review_cycle`, `qualified` | End review latency at first qualified review | | `violation_observed` | `fingerprint`, `adr_id`, `rule_id` | Open or continue a violation | | `violation_resolved` | `fingerprint`, `adr_id`, `rule_id` | Close a previously observed violation | @@ -135,6 +135,11 @@ GitHub are deduplicated. Evidence precedence is explicit JSONL, then local Git, then GitHub. A higher-precedence event wins when two sources disagree, and the conflict is emitted as a warning. +`review_cycle` is a provider-neutral, opaque string shared by every request and +submission belonging to one review cycle. Provider adapters must not expose a +pull-request number directly; the GitHub collector hashes its provider node ID +into a stable opaque cycle key. + ## Git And GitHub Collection The Git collector follows each `docs/decisions/[0-9]*.md` path through history, @@ -194,7 +199,9 @@ closed after a later `violation_resolved` event. At `until`, report the open count and each open violation's whole-day age from first uninterrupted observation. `--check-results` is a current snapshot: count is available, but age is unavailable unless matching historical observations are supplied with -`--events`. Active exceptions remain visible and do not close violations. +`--events`. An empty `--check-results` file is an authoritative snapshot with +zero open violations, not missing evidence. Active exceptions remain visible +and do not close violations. ### Exception Age diff --git a/scripts/adoption_metrics.py b/scripts/adoption_metrics.py index f3e2d5f..23a0a6c 100644 --- a/scripts/adoption_metrics.py +++ b/scripts/adoption_metrics.py @@ -2,6 +2,7 @@ """Collect provider-neutral ADR adoption metrics as deterministic JSON.""" import argparse +import hashlib import json import re import statistics @@ -35,11 +36,12 @@ } EXCEPTION_ID_RE = re.compile(r"^EXC-\d{4}$") ADR_ID_RE = re.compile(r"^ADR-\d{4}$") +DATE_ONLY_RE = re.compile(r"^\d{4}-\d{2}-\d{2}$") EVENT_REQUIRED_FIELDS = { "adr_created": {"adr_id", "status"}, "adr_status_changed": {"adr_id", "from", "to"}, - "review_requested": {"adr_id", "reviewer"}, - "review_submitted": {"adr_id", "reviewer", "qualified"}, + "review_requested": {"adr_id", "reviewer", "review_cycle"}, + "review_submitted": {"adr_id", "reviewer", "review_cycle", "qualified"}, "violation_observed": {"fingerprint", "adr_id", "rule_id"}, "violation_resolved": {"fingerprint", "adr_id", "rule_id"}, } @@ -48,6 +50,7 @@ repository(owner: $owner, name: $name) { pullRequests(first: 100, after: $cursor, orderBy: {field: UPDATED_AT, direction: DESC}) { nodes { + id number author { login } files(first: 100) { nodes { path } pageInfo { hasNextPage } } @@ -74,7 +77,7 @@ pageInfo { hasNextPage } } } - pageInfo { hasNextPage } + pageInfo { hasNextPage endCursor } } } } @@ -160,7 +163,11 @@ def read_exceptions(adr_dir: Path) -> Tuple[List[Dict[str, Any]], List[Dict[str, raise ValueError("{} must not be empty".format(field)) if not data["scope"]: raise ValueError("scope must contain at least one path pattern") + if not all(isinstance(item, str) for item in data["scope"]): + raise ValueError("scope items must be strings") for field in ("created", "expiry"): + if not DATE_ONLY_RE.fullmatch(data[field]): + raise ValueError("{} must be YYYY-MM-DD".format(field)) try: parse_timestamp(str(data[field])) except ValueError as exc: @@ -257,7 +264,9 @@ def _event_entity(event: Dict[str, Any]) -> str: if "fingerprint" in event: return str(event["fingerprint"]) if event.get("event") in {"review_requested", "review_submitted"}: - return "{}:{}".format(event.get("adr_id"), event.get("reviewer")) + return "{}:{}:{}".format( + event.get("adr_id"), event.get("review_cycle"), event.get("reviewer") + ) return str(event.get("adr_id")) @@ -599,7 +608,10 @@ def normalize_github_reviews( if not adr_ids: continue author = _reviewer_login(pull_request.get("author")) - review_cycle = "github-pr-{}".format(pull_request.get("number")) + provider_cycle = str(pull_request.get("id") or pull_request.get("number")) + review_cycle = hashlib.sha256( + "github:{}".format(provider_cycle).encode("utf-8") + ).hexdigest()[:16] requested = set() timeline_nodes = sorted( timeline.get("nodes", []), @@ -698,9 +710,21 @@ def _decision_lead_time( if proposed_at <= _event_time(event) <= _event_time(outcome) ) - event_adr_ids = set(by_adr) + terminal_event_adr_ids = { + adr_id + for adr_id, adr_events in by_adr.items() + if any( + ( + event.get("status") + if event.get("event") == "adr_created" + else event.get("to") + ) + in {"accepted", "rejected"} + for event in adr_events + ) + } for adr in adrs: - if str(adr.get("id")) in event_adr_ids: + if str(adr.get("id")) in terminal_event_adr_ids: continue if adr.get("status") not in {"accepted", "rejected", "superseded"}: continue @@ -817,7 +841,9 @@ def _supersession_rate( def _unresolved_violations( - events: List[Dict[str, Any]], until: datetime + events: List[Dict[str, Any]], + until: datetime, + current_snapshot: Optional[set] = None, ) -> Dict[str, Any]: violation_events = [ event @@ -825,7 +851,7 @@ def _unresolved_violations( if event.get("event") in {"violation_observed", "violation_resolved"} and _event_time(event) <= until ] - if not violation_events: + if not violation_events and current_snapshot is None: return { "available": False, "open_count": None, @@ -836,31 +862,9 @@ def _unresolved_violations( "reason": "No CHECK violation observations were available.", } - current_events = [ - event for event in violation_events if event.get("observation_mode") == "current" - ] - historical_events = [ - event for event in violation_events if event.get("observation_mode") != "current" - ] - current_fingerprints = { - str(event.get("fingerprint")) - for event in current_events - if event.get("event") == "violation_observed" - } - if current_events and not historical_events: - return { - "available": True, - "open_count": len(current_fingerprints), - "age_available": False, - "median_age_days": None, - "max_age_days": None, - "sources": sorted({str(event.get("source")) for event in current_events}), - "reason": "Current CHECK results have no historical first-observed evidence.", - } - open_since: Dict[str, datetime] = {} sources = set() - for event in sorted(historical_events, key=_event_time): + for event in sorted(violation_events, key=_event_time): fingerprint = str(event.get("fingerprint")) sources.add(str(event.get("source"))) if event["event"] == "violation_resolved": @@ -868,12 +872,14 @@ def _unresolved_violations( elif fingerprint not in open_since: open_since[fingerprint] = _event_time(event) - if current_events: - sources.update(str(event.get("source")) for event in current_events) - if not current_fingerprints.issubset(open_since): + if current_snapshot is not None: + sources.add("check_results") + if not current_snapshot: + open_since = {} + elif not current_snapshot.issubset(open_since): return { "available": True, - "open_count": len(current_fingerprints), + "open_count": len(current_snapshot), "age_available": False, "median_age_days": None, "max_age_days": None, @@ -881,7 +887,7 @@ def _unresolved_violations( "reason": "Some current violations lack historical first-observed evidence.", } open_since = { - fingerprint: open_since[fingerprint] for fingerprint in current_fingerprints + fingerprint: open_since[fingerprint] for fingerprint in current_snapshot } ages = [(until.date() - opened.date()).days for opened in open_since.values()] @@ -925,12 +931,15 @@ def calculate_metrics( events: List[Dict[str, Any]], since: datetime, until: datetime, + current_violation_fingerprints: Optional[set] = None, ) -> Dict[str, Any]: return { "decision_lead_time": _decision_lead_time(adrs, events, since, until), "review_latency": _review_latency(events, since, until), "supersession_rate": _supersession_rate(events, since, until), - "unresolved_violations": _unresolved_violations(events, until), + "unresolved_violations": _unresolved_violations( + events, until, current_violation_fingerprints + ), "exception_age": _exception_age(exceptions, until), } @@ -973,6 +982,23 @@ def _default_since( return min(candidates) if candidates else until +def github_adr_paths( + root: Path, adr_dir: Path, adrs: List[Dict[str, Any]] +) -> Dict[str, str]: + top_result = _run_git(root, ["rev-parse", "--show-toplevel"]) + repository_root = ( + Path(top_result.stdout.strip()).resolve() + if top_result.returncode == 0 and top_result.stdout.strip() + else root + ) + return { + (adr_dir / str(adr["file"])).resolve().relative_to(repository_root).as_posix(): str( + adr["id"] + ) + for adr in adrs + } + + def build_report( root: Path, adr_dir: Path, @@ -989,9 +1015,15 @@ def build_report( exceptions, exception_warnings = read_exceptions(adr_dir) explicit_events, explicit_warnings = read_events(event_paths) check_events, check_warnings = read_events(check_paths) - for event in check_events: - event["source"] = "check_results" - event["observation_mode"] = "current" + current_violation_fingerprints = ( + { + str(event["fingerprint"]) + for event in check_events + if event.get("event") == "violation_observed" + } + if check_paths + else None + ) git_events, git_warnings = collect_git_events(root, adr_dir) github_events: List[Dict[str, Any]] = [] @@ -999,10 +1031,7 @@ def build_report( if use_github: payload, github_warnings = collect_github_payload(root) if payload is not None: - adr_paths = { - (adr_dir / str(adr["file"])).relative_to(root).as_posix(): str(adr["id"]) - for adr in adrs - } + adr_paths = github_adr_paths(root, adr_dir, adrs) github_events, normalization_warnings = normalize_github_reviews( payload, adr_paths ) @@ -1011,7 +1040,6 @@ def build_report( events, merge_warnings = merge_events( [ ("events", explicit_events), - ("check_results", check_events), ("git", git_events), ("github", github_events), ] @@ -1034,7 +1062,12 @@ def build_report( "until": until.date().isoformat(), }, "metrics": calculate_metrics( - adrs, exceptions, events, effective_since, until + adrs, + exceptions, + events, + effective_since, + until, + current_violation_fingerprints, ), "warnings": ( adr_warnings diff --git a/tests/unit/test_adoption_metrics.py b/tests/unit/test_adoption_metrics.py index eaa6fef..37e61b4 100644 --- a/tests/unit/test_adoption_metrics.py +++ b/tests/unit/test_adoption_metrics.py @@ -164,6 +164,27 @@ def test_read_exceptions_rejects_wrong_types_and_invalid_ids(tmp_path): assert warnings[0]["code"] == "BAD_EXCEPTION" +def test_read_exceptions_rejects_non_string_scope_and_timestamp_dates(tmp_path): + exceptions_dir = tmp_path / "exceptions" + exceptions_dir.mkdir() + invalid = { + "id": "EXC-0001", + "adr_id": "ADR-0001", + "rule_id": "r1", + "owner": "team", + "reason": "migration", + "scope": [123], + "created": "2026-01-01T12:00:00Z", + "expiry": "2026-02-01", + } + (exceptions_dir / "0001.json").write_text(json.dumps(invalid), encoding="utf-8") + + records, warnings = adoption_metrics.read_exceptions(tmp_path) + + assert records == [] + assert warnings[0]["code"] == "BAD_EXCEPTION" + + def test_exception_created_after_report_date_is_not_counted_as_negative_age(): exceptions = [ {"id": "EXC-0001", "created": "2026-02-01", "expiry": "2026-03-01"} @@ -286,6 +307,24 @@ def test_decision_lead_time_does_not_replace_pre_period_first_outcome(): assert result["coverage"] == {"eligible": 0, "measured": 0, "ratio": None} +def test_decision_lead_time_uses_snapshot_fallback_after_only_proposed_event(): + adrs = [{"id": "ADR-0001", "status": "accepted", "date": "2026-01-02"}] + events = [ + _event( + "adr_created", + "2026-01-01T00:00:00Z", + adr_id="ADR-0001", + status="proposed", + ) + ] + + result = adoption_metrics.calculate_metrics(adrs, [], events, SINCE, UNTIL)[ + "decision_lead_time" + ] + + assert result["coverage"] == {"eligible": 1, "measured": 0, "ratio": 0.0} + + def test_review_latency_uses_first_qualified_review_after_request(): events = [ _event( @@ -716,6 +755,27 @@ def test_read_events_rejects_wrong_payload_types(tmp_path): assert warnings[0]["code"] == "BAD_EVENT_SCHEMA" +def test_read_events_requires_review_cycle_for_review_events(tmp_path): + events_path = tmp_path / "events.jsonl" + events_path.write_text( + json.dumps( + _event( + "review_requested", + "2026-01-01T00:00:00Z", + adr_id="ADR-0001", + reviewer="alice", + ) + ) + + "\n", + encoding="utf-8", + ) + + events, warnings = adoption_metrics.read_events([events_path]) + + assert events == [] + assert warnings[0]["code"] == "BAD_EVENT_SCHEMA" + + def test_merge_events_treats_equivalent_utc_timestamp_spellings_as_duplicates(): explicit = _event( "adr_created", @@ -851,6 +911,22 @@ def test_collect_git_events_follows_rename_inside_a_nested_project_root(tmp_path ] +def test_github_adr_paths_are_relative_to_git_top_level_for_nested_root(tmp_path): + repository = tmp_path / "repository" + root = repository / "nested-project" + adr_dir = root / "docs" / "decisions" + adr_dir.mkdir(parents=True) + _git(repository, "init") + + paths = adoption_metrics.github_adr_paths( + root, adr_dir, [{"id": "ADR-0001", "file": "0001-test.md"}] + ) + + assert paths == { + "nested-project/docs/decisions/0001-test.md": "ADR-0001" + } + + def test_collect_git_events_degrades_cleanly_outside_a_git_repository(tmp_path): adr_dir = tmp_path / "docs" / "decisions" adr_dir.mkdir(parents=True) @@ -889,6 +965,7 @@ def test_normalize_github_reviews_qualifies_requested_non_author_reviewer(): "nodes": [ { "number": 7, + "id": "PR_node_opaque_7", "author": {"login": "owner"}, "files": { "nodes": [ @@ -943,6 +1020,7 @@ def test_normalize_github_reviews_qualifies_requested_non_author_reviewer(): ("alice", True), ] assert all(event["adr_id"] == "ADR-0001" for event in events) + assert all(event["review_cycle"] != "github-pr-7" for event in events) def test_normalize_github_reviews_warns_when_provider_result_is_truncated(): @@ -1032,6 +1110,8 @@ def fake_gh(root, arguments): elif any(value == "cursor=CURSOR-1" for value in arguments): stdout = json.dumps(second_page) else: + query_argument = next(value for value in arguments if value.startswith("query=")) + assert "pageInfo { hasNextPage endCursor }" in query_argument stdout = json.dumps(first_page) return subprocess.CompletedProcess(["gh", *arguments], 0, stdout, "") @@ -1172,6 +1252,51 @@ def test_cli_current_check_results_report_count_without_inventing_age(tmp_path, assert violations["sources"] == ["check_results"] +def test_cli_empty_current_check_snapshot_reports_zero_and_closes_stale_history( + tmp_path, capsys +): + adr_dir = tmp_path / "docs" / "decisions" + adr_dir.mkdir(parents=True) + _write_adr(adr_dir / "0001-test.md", "ADR-0001", "accepted") + history_path = tmp_path / "events.jsonl" + history_path.write_text( + json.dumps( + _event( + "violation_observed", + "2026-01-01T00:00:00Z", + adr_id="ADR-0001", + rule_id="r1", + fingerprint="f1", + ) + ) + + "\n", + encoding="utf-8", + ) + check_path = tmp_path / "check.jsonl" + check_path.write_text("", encoding="utf-8") + + return_code = adoption_metrics.main( + [ + "--root", + str(tmp_path), + "--dir", + "docs/decisions", + "--until", + "2026-01-31", + "--events", + "events.jsonl", + "--check-results", + "check.jsonl", + "--json", + ] + ) + + violations = json.loads(capsys.readouterr().out)["metrics"]["unresolved_violations"] + assert return_code == 0 + assert violations["available"] is True + assert violations["open_count"] == 0 + + def test_cli_opt_in_github_failure_is_a_warning(tmp_path, capsys, monkeypatch): adr_dir = tmp_path / "docs" / "decisions" adr_dir.mkdir(parents=True) From e00f48cfd36c13a3424b4d7f3d8b68681091e488 Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 15:06:01 +0900 Subject: [PATCH 49/58] fix: validate current check snapshots --- .../2026-09-01-adoption-metrics-design.md | 11 +- scripts/adoption_metrics.py | 44 ++++- tests/unit/test_adoption_metrics.py | 158 ++++++++++++++++++ 3 files changed, 201 insertions(+), 12 deletions(-) diff --git a/docs/superpowers/specs/2026-09-01-adoption-metrics-design.md b/docs/superpowers/specs/2026-09-01-adoption-metrics-design.md index 0629c3e..56c055e 100644 --- a/docs/superpowers/specs/2026-09-01-adoption-metrics-design.md +++ b/docs/superpowers/specs/2026-09-01-adoption-metrics-design.md @@ -49,9 +49,8 @@ non-Git SCMs, and manually exported audit data. The implementation is a single repository tool, `scripts/adoption_metrics.py`, split internally into four boundaries: -1. Local readers parse ADR frontmatter, exception JSON, and optional historical - CHECK observation files without importing the skill-internal `scripts` - package. +1. Local readers parse ADR frontmatter, exception JSON, and optional current + CHECK snapshot files without importing the skill-internal `scripts` package. 2. Collectors normalize explicit JSONL, local Git history, and optional GitHub review data into provider-neutral events. 3. Pure calculation functions consume current records and normalized events. @@ -101,6 +100,12 @@ for interval filtering and whole-day age calculations. Invalid inputs produce a non-zero exit and a JSON error object; an individual malformed ADR, exception, or event becomes a warning while other valid records are still processed. +Each `--check-results` file is a current snapshot, not an event-history input. +Every non-empty line must therefore be a valid `violation_observed` record at +or before `--until`. A successfully read empty snapshot authoritatively means +zero current violations. A missing, malformed, mixed-event, or future-dated +snapshot emits warnings and is not allowed to replace historical evidence. + ## Normalized Events Each line of `--events` and `--check-results` is one JSON object with a versioned diff --git a/scripts/adoption_metrics.py b/scripts/adoption_metrics.py index 23a0a6c..3f43792 100644 --- a/scripts/adoption_metrics.py +++ b/scripts/adoption_metrics.py @@ -260,6 +260,39 @@ def read_events( return events, warnings +def read_check_snapshot( + paths: List[Path], until: datetime +) -> Tuple[Optional[set], List[Dict[str, Any]]]: + if not paths: + return None, [] + + events, warnings = read_events(paths) + complete = not warnings + fingerprints = set() + for event in events: + if event["event"] != "violation_observed": + warnings.append( + { + "code": "BAD_CHECK_SNAPSHOT", + "detail": "CHECK snapshots may contain only violation_observed records.", + } + ) + complete = False + continue + if _event_time(event) > until: + warnings.append( + { + "code": "BAD_CHECK_SNAPSHOT", + "detail": "CHECK snapshot contains an observation after --until.", + } + ) + complete = False + continue + fingerprints.add(str(event["fingerprint"])) + + return (fingerprints if complete else None), warnings + + def _event_entity(event: Dict[str, Any]) -> str: if "fingerprint" in event: return str(event["fingerprint"]) @@ -1014,15 +1047,8 @@ def build_report( adrs, adr_warnings = read_adrs(adr_dir) exceptions, exception_warnings = read_exceptions(adr_dir) explicit_events, explicit_warnings = read_events(event_paths) - check_events, check_warnings = read_events(check_paths) - current_violation_fingerprints = ( - { - str(event["fingerprint"]) - for event in check_events - if event.get("event") == "violation_observed" - } - if check_paths - else None + current_violation_fingerprints, check_warnings = read_check_snapshot( + check_paths, until ) git_events, git_warnings = collect_git_events(root, adr_dir) diff --git a/tests/unit/test_adoption_metrics.py b/tests/unit/test_adoption_metrics.py index 37e61b4..13e7b0a 100644 --- a/tests/unit/test_adoption_metrics.py +++ b/tests/unit/test_adoption_metrics.py @@ -5,6 +5,8 @@ from datetime import datetime, timezone from pathlib import Path +import pytest + _ADOPTION_METRICS_PATH = ( Path(__file__).resolve().parents[2] / "scripts" / "adoption_metrics.py" @@ -1297,6 +1299,162 @@ def test_cli_empty_current_check_snapshot_reports_zero_and_closes_stale_history( assert violations["open_count"] == 0 +@pytest.mark.parametrize("snapshot_contents", ["{not-json}\n", None]) +def test_cli_invalid_check_snapshot_does_not_clear_stale_history( + tmp_path, capsys, snapshot_contents +): + adr_dir = tmp_path / "docs" / "decisions" + adr_dir.mkdir(parents=True) + _write_adr(adr_dir / "0001-test.md", "ADR-0001", "accepted") + history_path = tmp_path / "events.jsonl" + history_path.write_text( + json.dumps( + _event( + "violation_observed", + "2026-01-01T00:00:00Z", + adr_id="ADR-0001", + rule_id="r1", + fingerprint="f1", + ) + ) + + "\n", + encoding="utf-8", + ) + check_path = tmp_path / "check.jsonl" + if snapshot_contents is not None: + check_path.write_text(snapshot_contents, encoding="utf-8") + + return_code = adoption_metrics.main( + [ + "--root", + str(tmp_path), + "--dir", + "docs/decisions", + "--until", + "2026-01-31", + "--events", + "events.jsonl", + "--check-results", + "check.jsonl", + "--json", + ] + ) + + result = json.loads(capsys.readouterr().out) + assert return_code == 0 + assert result["metrics"]["unresolved_violations"]["open_count"] == 1 + + +def test_cli_non_violation_check_record_does_not_clear_stale_history(tmp_path, capsys): + adr_dir = tmp_path / "docs" / "decisions" + adr_dir.mkdir(parents=True) + _write_adr(adr_dir / "0001-test.md", "ADR-0001", "accepted") + history_path = tmp_path / "events.jsonl" + history_path.write_text( + json.dumps( + _event( + "violation_observed", + "2026-01-01T00:00:00Z", + adr_id="ADR-0001", + rule_id="r1", + fingerprint="f1", + ) + ) + + "\n", + encoding="utf-8", + ) + check_path = tmp_path / "check.jsonl" + check_path.write_text( + json.dumps( + _event( + "violation_resolved", + "2026-01-30T00:00:00Z", + adr_id="ADR-0001", + rule_id="r1", + fingerprint="f1", + ) + ) + + "\n", + encoding="utf-8", + ) + + adoption_metrics.main( + [ + "--root", + str(tmp_path), + "--dir", + "docs/decisions", + "--until", + "2026-01-31", + "--events", + "events.jsonl", + "--check-results", + "check.jsonl", + "--json", + ] + ) + + result = json.loads(capsys.readouterr().out) + assert result["metrics"]["unresolved_violations"]["open_count"] == 1 + assert any(warning["code"] == "BAD_CHECK_SNAPSHOT" for warning in result["warnings"]) + + +def test_cli_check_record_after_until_does_not_replace_history(tmp_path, capsys): + adr_dir = tmp_path / "docs" / "decisions" + adr_dir.mkdir(parents=True) + _write_adr(adr_dir / "0001-test.md", "ADR-0001", "accepted") + history_path = tmp_path / "events.jsonl" + history_path.write_text( + json.dumps( + _event( + "violation_observed", + "2026-01-01T00:00:00Z", + adr_id="ADR-0001", + rule_id="r1", + fingerprint="old", + ) + ) + + "\n", + encoding="utf-8", + ) + check_path = tmp_path / "check.jsonl" + check_path.write_text( + json.dumps( + _event( + "violation_observed", + "2026-02-01T00:00:00Z", + adr_id="ADR-0001", + rule_id="r2", + fingerprint="future", + ) + ) + + "\n", + encoding="utf-8", + ) + + adoption_metrics.main( + [ + "--root", + str(tmp_path), + "--dir", + "docs/decisions", + "--until", + "2026-01-31", + "--events", + "events.jsonl", + "--check-results", + "check.jsonl", + "--json", + ] + ) + + result = json.loads(capsys.readouterr().out) + violations = result["metrics"]["unresolved_violations"] + assert violations["open_count"] == 1 + assert violations["median_age_days"] == 30 + assert any(warning["code"] == "BAD_CHECK_SNAPSHOT" for warning in result["warnings"]) + + def test_cli_opt_in_github_failure_is_a_warning(tmp_path, capsys, monkeypatch): adr_dir = tmp_path / "docs" / "decisions" adr_dir.mkdir(parents=True) From 276c6a0c6603864615cffa04d01c1d9b5244d81b Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 15:06:05 +0900 Subject: [PATCH 50/58] docs: close out adoption metrics collector --- changelog.md | 6 ++++++ handoff.md | 42 ++++++++++++++++-------------------------- improvements.md | 1 - 3 files changed, 22 insertions(+), 27 deletions(-) diff --git a/changelog.md b/changelog.md index a5fce6e..0cf7d9f 100644 --- a/changelog.md +++ b/changelog.md @@ -4,6 +4,12 @@ Lightweight human-readable summary of meaningful repository changes. ## Unreleased +- Added `scripts/adoption_metrics.py`, a JSON-only collector for the five + enterprise adoption metrics defined in `docs/enterprise-adoption.md`: + decision lead time, review latency, supersession rate, unresolved CHECK + violations, and exception age. It combines portable ADR/exception data with + optional local Git, explicit JSONL event, CHECK snapshot, and GitHub review + evidence while reporting source coverage and incomplete-evidence warnings. - `core/contracts.py` now covers all 16 commands' output shapes (was 2). - `PathEscapesRootError` (added in the prior session's path-escape fix) is now caught at all 7 call sites and reported as a structured diff --git a/handoff.md b/handoff.md index c256c03..51b40c4 100644 --- a/handoff.md +++ b/handoff.md @@ -94,17 +94,13 @@ correctly blocked (agy still has no public registry, per `adapters/antigravity/README.md`) -- not everything merged from that worktree closes every item tied to it. -**Concurrent work (owner's own coordination, not this session's):** the -owner assigned `improvements.md`'s "도입 지표 수집 스크립트" -(adoption-metrics script, from `docs/enterprise-adoption.md` §7) to a -**Codex session running in this same worktree/branch** in parallel with -this session, specifically because it's a new-file-only task with no -overlap against the files this session was touching. If you see an -uncommitted or newly-committed `scripts/adoption_metrics.py` (or -similarly named) plus a matching test file that you don't recognize -authoring, that's Codex's work landing -- don't revert it, and check -`improvements.md`'s enterprise-adoption.md sub-group for whether it's -already been checked off before restarting it. +**Adoption metrics follow-up completed:** `9a0de45`..`7fb373e` add the +design, JSON-only `scripts/adoption_metrics.py` collector, and focused tests. +The collector calculates all five metrics from `docs/enterprise-adoption.md` +§7 using ADR/exception data plus optional local Git, explicit JSONL event and +CHECK snapshot files, and GitHub review evidence. Incomplete evidence is +reported through coverage, availability, and warning fields rather than being +silently treated as complete data. All 3 plan files are gitignored by convention (`docs/superpowers/plans/`) but still on disk in this worktree. @@ -182,9 +178,8 @@ code again: - README prose (root README.md, `adapters/*/README.md` content) -- still another worktree's; every fix across all passes that touched adapter or generator code was a code fix, not README prose. -- **Do not touch what the parallel Codex session is doing** (the - adoption-metrics collector -- already 4 commits in as of `45b3472`, - see below). Don't revert, refactor, or duplicate its work. +- The adoption-metrics collector is complete; future changes should preserve + its provider-neutral evidence contracts and JSON-only stdout behavior. ## Next step (for a new session picking this up cold) @@ -202,13 +197,9 @@ parallel Codex session's. Concretely: item left (Antigravity in `harness-parity`), re-verified against `adapters/antigravity/README.md` and still blocked on an external fact (agy has no public package registry) -- don't start it. -3. `improvements.md`'s `### Low` → enterprise-adoption.md sub-group: check - whether the Codex session's adoption-metrics work has been checked off - before assuming it's still open (as of this note it's implemented -- - `9a0de45`..`45b3472` -- but not yet reflected in `improvements.md` - since this session was told not to touch that item's bookkeeping). The - other 3 items there remain precondition-gated (repository going - public, 2+ maintainers, 2+ repositories) -- **not pure code tasks**. +3. `improvements.md`'s `### Low` → enterprise-adoption.md sub-group now has + only 3 precondition-gated items (repository going public, 2+ maintainers, + 2+ repositories) -- **not pure code tasks**. 4. If the user says "continue" / "다음 작업 진행해줘" without naming a task: the supply-chain item is the one thing to offer; otherwise ask what's next rather than inventing scope. @@ -233,11 +224,10 @@ on each. ## Verification -`python3 -m pytest tests/unit tests/integration -v` -> 518 passed as of -commit `26021a9` (395 at session start -> 465 before the `origin/develop` -merge -> 469 after merging in develop's own new tests -> 479 after the -Low-priority follow-up work -> 518 current, which also includes the -parallel Codex session's adoption-metrics tests landing in this branch). +`python3 -m pytest tests/unit tests/integration -q` -> 537 passed after the +adoption-metrics collector and review fixes. `mypy --strict` over the three CI +target modules, examples verification, version-sync verification, collector +compilation, and `git diff --check` also pass. CI now also runs `type-check` (`mypy --strict`), `examples-drift` (from develop), and `pr-title-check` (from develop) jobs alongside the existing `pytest` (now coverage-gated at 85%), `version-drift`, and diff --git a/improvements.md b/improvements.md index 8d0d448..a91af43 100644 --- a/improvements.md +++ b/improvements.md @@ -57,7 +57,6 @@ closed out or reworded below. README prose is still another worktree's. - [ ] *(전제조건: 저장소 public 전환)* **Public 전환 게이트 실제 적용** — PR template/`CONTRIBUTING.md`/`SECURITY.md`는 이미 존재함(v0.2.1에 포함, `origin/develop` 병합으로 확인). 남은 건 `master`/`develop`/`v*` 태그에 대한 실제 GitHub ruleset(PR 필수, required CI, conversation resolution, force-push/삭제 차단) 적용과 API로 실제 상태 재조회뿐 — 코드 작업이 아니라 저장소를 public 전환한 뒤 GitHub 설정/API에서 해야 하는 작업. `project_v1_public_release_plan` 메모리 참고(1.0.0 시점 public 전환 계획). (enterprise-adoption.md §4, §9) - [ ] *(전제조건: qualified maintainer 2명 이상)* **CODEOWNERS 독립 승인 활성화** — 현재 1인 운영 상태에서 필수 code-owner review를 켜면 운영을 막거나 형식적 self-review만 만든다고 보고서 자체가 명시적으로 경고함. 인원 조건 충족 전엔 시작하지 않음. (enterprise-adoption.md §4, §9 "지금 구현하지 않을 것") - [ ] *(전제조건: 저장소 2개 이상)* **조직 단위 ruleset/reusable workflow/audit export/taxonomy** — 여러 저장소가 같은 운영 문제를 반복할 때 설계 시작. 지금은 저장소가 1개뿐이라 시작 조건 미충족. (enterprise-adoption.md §6, §8 항목 5) -- [ ] **도입 지표(adoption metrics) 수집 스크립트** — decision lead time, exception age, unresolved violations 같은 지표는 이미 존재하는 ADR frontmatter(`date`, `status`)와 exception JSON(`created`, `expiry`)만으로 계산 가능해 public 전환이나 멀티레포 없이도 지금 시작할 수 있음. 다만 "이 지표를 수집한다는 사실만으로 성숙도가 올라가지 않는다"는 보고서 자체의 경고를 유념 — 지표 정의 버전 관리, 실제 운영 개선 연결까지 되어야 의미가 있음. (enterprise-adoption.md §7) ## Done From f814d64ae0a031f10b21fe1f70e0d6fe7487c1ee Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 15:06:38 +0900 Subject: [PATCH 51/58] docs: record final adoption metrics verification --- handoff.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/handoff.md b/handoff.md index 51b40c4..b835710 100644 --- a/handoff.md +++ b/handoff.md @@ -224,7 +224,7 @@ on each. ## Verification -`python3 -m pytest tests/unit tests/integration -q` -> 537 passed after the +`python3 -m pytest tests/unit tests/integration -q` -> 541 passed after the adoption-metrics collector and review fixes. `mypy --strict` over the three CI target modules, examples verification, version-sync verification, collector compilation, and `git diff --check` also pass. From 9223bfa6f460a497041e62c33552257f987dc611 Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 15:17:56 +0900 Subject: [PATCH 52/58] docs: mark adoption metrics complete --- handoff.md | 8 ++++++-- improvements.md | 6 ++++++ 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/handoff.md b/handoff.md index b835710..66d20cf 100644 --- a/handoff.md +++ b/handoff.md @@ -94,8 +94,12 @@ correctly blocked (agy still has no public registry, per `adapters/antigravity/README.md`) -- not everything merged from that worktree closes every item tied to it. -**Adoption metrics follow-up completed:** `9a0de45`..`7fb373e` add the -design, JSON-only `scripts/adoption_metrics.py` collector, and focused tests. +**Adoption metrics follow-up completed:** + +- [x] `9a0de45`..`f814d64` add the design, JSON-only + `scripts/adoption_metrics.py` collector, focused tests, review fixes, and + final verification records. + The collector calculates all five metrics from `docs/enterprise-adoption.md` §7 using ADR/exception data plus optional local Git, explicit JSONL event and CHECK snapshot files, and GitHub review evidence. Incomplete evidence is diff --git a/improvements.md b/improvements.md index a91af43..950d5c5 100644 --- a/improvements.md +++ b/improvements.md @@ -113,6 +113,12 @@ instead of only when CHECK later runs against it. The 4th item (Antigravity in harness-parity CI) stays open, blocked on `agy` getting a public package registry. +- [x] **도입 지표(adoption metrics) 수집 스크립트** — + `scripts/adoption_metrics.py`가 `docs/enterprise-adoption.md` §7의 다섯 + 지표를 JSON으로 계산한다. ADR/exception 스냅샷, 로컬 Git, 명시적 JSONL, + CHECK 스냅샷, 선택적 GitHub 리뷰 근거를 지원하며 불완전한 근거는 + coverage/availability/warning으로 노출한다. (`9a0de45`..`f814d64`) + **Windows ReDoS static complexity linter** (promoted from `handoff.md`'s Open Risks, not originally a numbered backlog item) — `core/constraints.py` now statically rejects a nested-quantifier `pattern` value (e.g. `(a+)+`) From 18d46623b476c829cfc7469101117c8fee703724 Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 16:03:14 +0900 Subject: [PATCH 53/58] feat: add build provenance attestation to the release workflow MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit .github/workflows/release.yml now packages skills/adr-toolkit/ into a version-named tarball, checksums it (SHA-256), and generates a Sigstore- backed GitHub Artifact Attestation for it via actions/attest-build- provenance@v2 -- keyless (OIDC-based), no private key to manage or rotate. Both the tarball and its checksum are attached to the GitHub Release. Chose this over signing the git tag itself: every adapter (Claude Code marketplace source:"./", Codex/Gemini CLI plugin installs, generic copy/ symlink) references the repo or skill folder directly rather than downloading a packaged release, and tags in this project are created locally by a human before the push that triggers this workflow -- CI can't retroactively sign a tag that already exists. Attestation instead ties provenance to the exact commit the tag points to, verifiable via `gh attestation verify`, which covers the one real gap: someone grabbing the archive off the GitHub Releases page instead of cloning. SECURITY.md documents the verification commands (sha256sum -c + gh attestation verify) and is explicit that the git-clone/adapter-install paths verify via Git/GitHub history already, not this archive. Verified locally: the tar+sha256sum packaging commands run correctly against the real skills/adr-toolkit/ tree, and the modified YAML parses without syntax errors. The actual OIDC/attestation exchange can only be exercised by a real tag push through GitHub Actions, which this session did not do (release pipeline; requires explicit owner action to trigger). docs/adr-toolkit-audit-report.md §2.2 2.2. --- .github/workflows/release.yml | 25 +++++++++++++++++++++++++ SECURITY.md | 29 +++++++++++++++++++++++++++++ 2 files changed, 54 insertions(+) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index a908c94..9d65cf4 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -15,8 +15,17 @@ jobs: runs-on: ubuntu-latest # softprops/action-gh-release needs write access to create the release; # the default GITHUB_TOKEN is read-only for repos created since Feb 2023. + # attest-build-provenance needs id-token (to mint an OIDC token for + # Sigstore's keyless signing) and attestations: write (to publish the + # resulting attestation to the repo) -- no private key to manage or + # rotate; every install/consumption path here references the git repo + # or this packaged skill folder directly (docs/adr-toolkit-audit-report.md + # §2.2 2.2), so provenance tied to the exact commit is what actually + # matters, not a signed build of something nobody downloads separately. permissions: contents: write + id-token: write + attestations: write steps: - uses: actions/checkout@v4 - uses: actions/setup-python@v5 @@ -32,7 +41,23 @@ jobs: run: | test "v$(cat skills/adr-toolkit/VERSION)" = "$GITHUB_REF_NAME" \ || { echo "tag $GITHUB_REF_NAME != v$(cat skills/adr-toolkit/VERSION)"; exit 1; } + - name: Package the distributable skill + id: package + run: | + set -euo pipefail + VERSION="$(cat skills/adr-toolkit/VERSION)" + ARCHIVE="adr-toolkit-skill-v${VERSION}.tar.gz" + tar -czf "$ARCHIVE" -C skills adr-toolkit + sha256sum "$ARCHIVE" > "${ARCHIVE}.sha256" + echo "archive=$ARCHIVE" >> "$GITHUB_OUTPUT" + - name: Generate build provenance attestation + uses: actions/attest-build-provenance@v2 + with: + subject-path: ${{ steps.package.outputs.archive }} - name: Create GitHub Release uses: softprops/action-gh-release@v2 with: generate_release_notes: true + files: | + ${{ steps.package.outputs.archive }} + ${{ steps.package.outputs.archive }}.sha256 diff --git a/SECURITY.md b/SECURITY.md index 84f604e..2332879 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -18,6 +18,35 @@ public issue. Include: Do not include exploit details in a public issue until a fix or mitigation is available. +## Verifying a Release + +Every `v*` release is built by `.github/workflows/release.yml` from a +tagged commit, and the workflow publishes a [GitHub Artifact +Attestation](https://docs.github.com/en/actions/security-guides/using-artifact-attestations-to-establish-provenance-for-builds) +for the packaged skill archive it attaches to the release +(`adr-toolkit-skill-vX.Y.Z.tar.gz`). This is a Sigstore-backed, keyless +signature -- no private key is held or rotated by this project -- proving +the archive was produced by this repository's own CI, from the exact +commit the release tag points to. + +To verify a downloaded archive: + +```bash +# Checksum: confirms the file wasn't corrupted/tampered with in transit +sha256sum -c adr-toolkit-skill-vX.Y.Z.tar.gz.sha256 + +# Provenance: confirms the archive was actually built by this repo's CI, +# not a look-alike release from a compromised account or a different repo +gh attestation verify adr-toolkit-skill-vX.Y.Z.tar.gz -R SHcommit/ADR-toolkit +``` + +Anyone consuming this repository directly (`git clone`, or an adapter's +`marketplace add`/`plugin add` pointing at the repo, which is how every +adapter installs this skill today) is verifying via Git/GitHub's own +commit and tag history rather than this archive -- the attestation is +primarily for the one path where a bare checkout doesn't happen: someone +who downloads the release archive off the GitHub Releases page directly. + ## Scope In scope: From cf6ed6267dd68c6d0bbac263e152ef64eb2eeb8b Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 16:10:20 +0900 Subject: [PATCH 54/58] docs: close out supply-chain attestation and add worklog Marks the release-pipeline attestation work (18d4662) done in improvements.md/handoff.md and records the option analysis in a troubleshooting worklog, without touching the concurrent adoption- metrics entries already present in both files. --- changelog.md | 5 + .../2026-09-01-supply-chain-attestation.md | 102 ++++++++++++++++++ handoff.md | 55 ++++++---- improvements.md | 18 ++-- 4 files changed, 154 insertions(+), 26 deletions(-) create mode 100644 docs/worklogs/2026-09-01-supply-chain-attestation.md diff --git a/changelog.md b/changelog.md index 0cf7d9f..d1ecc73 100644 --- a/changelog.md +++ b/changelog.md @@ -4,6 +4,11 @@ Lightweight human-readable summary of meaningful repository changes. ## Unreleased +- `.github/workflows/release.yml` now packages `skills/adr-toolkit/` into a + version-named tarball, SHA-256 checksums it, and generates a Sigstore-backed + GitHub Artifact Attestation (`actions/attest-build-provenance@v2`, keyless) + for it; the tarball and checksum are attached to the GitHub Release. + `SECURITY.md` documents how to verify a downloaded release archive. - Added `scripts/adoption_metrics.py`, a JSON-only collector for the five enterprise adoption metrics defined in `docs/enterprise-adoption.md`: decision lead time, review latency, supersession rate, unresolved CHECK diff --git a/docs/worklogs/2026-09-01-supply-chain-attestation.md b/docs/worklogs/2026-09-01-supply-chain-attestation.md new file mode 100644 index 0000000..5ab4f95 --- /dev/null +++ b/docs/worklogs/2026-09-01-supply-chain-attestation.md @@ -0,0 +1,102 @@ +# 릴리스 아티팩트 공급망 보안(Build Provenance Attestation) 도입 + +## 날짜 + +2026-09-01 + +## 문제 상황 + +`docs/adr-toolkit-audit-report.md` §2.2 2.2 감사 항목이 "공급망 보안 +체크섬/서명 부재"를 지적했다. `.github/workflows/release.yml`은 테스트 +실행, 매니페스트 버전 동기화 검사(`sync_version.py --check`), 태그== +`VERSION` 일치 검증까지만 하고 `softprops/action-gh-release@v2`로 +릴리스를 생성할 뿐, 릴리스에 첨부되는 어떤 아티팩트도 체크섬이나 +서명이 없었다. + +## 기존 구조나 방식의 한계 + +일반적인 npm/PyPI 프로젝트라면 "빌드 산출물을 체크섬 찍고 Sigstore로 +서명"하는 패턴이 바로 적용된다. 하지만 이 프로젝트는 빌드 산출물 +자체가 없다 — Claude Code 마켓플레이스(`marketplace.json`의 +`source: "./"`), Codex/Gemini CLI 플러그인 설치, 일반 copy/symlink 설치 +전부 git 저장소나 `skills/adr-toolkit/` 폴더를 **직접** 참조한다. +즉 "체크섬/서명할 아티팩트가 무엇인가"부터 정의되지 않은 상태였고, +감사 보고서의 원안(빌드 아티팩트를 체크섬+서명)을 그대로 옮기면 +아무도 실제로 소비하지 않는 파일에 서명하는 형식적 조치가 될 +위험이 있었다. + +## 관련 코드 맥락 + +- `.github/workflows/release.yml` — `v*` 태그 push 시 실행되는 유일한 + 릴리스 파이프라인. 기존에는 `permissions: contents: write`만 있었고 + 패키징 스텝이 전혀 없었다. +- `AGENTS.md`에 문서화된 릴리스 프로세스: 버전 태그는 **사람이 로컬에서 + 직접 생성**한 뒤 push한다. 즉 CI는 이미 push된 태그를 사후에 서명할 + 방법이 없다(태그 자체에 서명하려면 로컬 GPG 서명 절차가 별도로 + 필요하며, 이는 CI 워크플로 범위 밖). +- `SECURITY.md` — 취약점 신고 프로세스만 있고 릴리스 검증 방법에 대한 + 섹션이 없었다. + +## 검토한 선택지 + +1. **빌드 아티팩트 없음 + git 태그 자체에 서명(GPG)** — 태그가 로컬에서 + 사람이 만들기 때문에 CI 워크플로 안에서는 구현 불가. 기각. +2. **`skills/adr-toolkit/`를 tar.gz로 패키징 + 체크섬만** — 전송 중 + 손상은 잡지만 "진짜 이 저장소의 CI가 만들었는가"라는 provenance는 + 증명하지 못함. +3. **tar.gz 패키징 + SHA-256 체크섬 + GitHub Artifact Attestation + (`actions/attest-build-provenance@v2`)** — Sigstore 기반 keyless + 서명이라 개인키 관리/로테이션이 전혀 없고, GitHub Actions OIDC 토큰으로 + "이 커밋의 이 워크플로 실행이 만든 파일"이라는 provenance를 증명한다. +4. **자체 GPG 키를 리포지토리 시크릿으로 관리해 아티팩트 서명** — 개인키 + 보관/로테이션 부담이 있고, 이 프로젝트처럼 유지관리자가 1인인 + 상황에서는 키 손실/유출 리스크만 늘어남. 기각. + +## 판단 기준 + +- 실제 소비 경로(거의 모든 설치가 git repo/스킬 폴더 직접 참조)를 + 기준으로, "아무도 받지 않는 아티팩트에 서명"하는 헛수고를 피한다. +- CI가 사후에 할 수 있는 일만 범위에 넣는다 — 태그 서명처럼 이미 + 일어난 사람의 행동을 CI가 대신할 수 없는 것은 배제. +- 개인키를 만들거나 로테이션하는 운영 부담을 새로 만들지 않는다(1인 + 운영 프로젝트라는 현재 상태를 고려). + +## 최종 결정 + +옵션 3 — tar.gz 패키징 + SHA-256 체크섬 + GitHub Artifact Attestation. +GitHub Releases 페이지에서 직접 아카이브를 내려받는 소수의 경로에 대해 +"이 파일이 이 저장소의 이 커밋에서 만들어졌다"는 provenance를 keyless로 +증명하고, 나머지(git clone/plugin install 경로)는 기존처럼 Git/GitHub +자체의 커밋 이력으로 신뢰성을 확보한다는 점을 `SECURITY.md`에 명시했다. + +## 해결 방식 + +1. `.github/workflows/release.yml`의 `permissions`에 `id-token: write`, + `attestations: write` 추가(OIDC 토큰 발급 + attestation 게시 권한). +2. "Package the distributable skill" 스텝 추가: `VERSION` 파일을 읽어 + `adr-toolkit-skill-v${VERSION}.tar.gz`로 `skills/adr-toolkit`를 + 패키징하고 `sha256sum`으로 체크섬 파일 생성, `$GITHUB_OUTPUT`으로 + 아카이브 경로를 다음 스텝에 전달. +3. "Generate build provenance attestation" 스텝 추가: + `actions/attest-build-provenance@v2`에 `subject-path`로 방금 만든 + 아카이브 경로를 전달. +4. "Create GitHub Release" 스텝의 `files:`에 아카이브와 `.sha256` 파일을 + 함께 첨부. +5. `SECURITY.md`에 "Verifying a Release" 섹션 신설 — `sha256sum -c`와 + `gh attestation verify -R SHcommit/ADR-toolkit` 명령, 그리고 + git clone/adapter 설치 경로는 이 아카이브 검증과 무관하다는 점을 명시. + +## 결과 + +- 커밋 `18d4662` (`feat: add build provenance attestation to the release + workflow`). +- YAML 문법 검증 통과, 전체 테스트 스위트(`pytest tests/unit + tests/integration`) 541 passed로 회귀 없음 확인. +- 실제 OIDC 기반 attestation 발급/검증 플로우 자체는 GitHub Actions + 러너에서 실제 `v*` 태그 push가 일어나야만 최종 확인 가능 — 로컬에서는 + 워크플로 문법과 각 스텝의 셸 로직만 검증했다. 다음 실제 릴리스 + 태그(예: 다음 버전 bump) 때 `gh attestation verify`로 실물 검증 필요. +- 병렬로 진행 중이던 Codex 세션의 도입 지표(adoption metrics) 수집기 + 작업(`9a0de45`..`f814d64`, `scripts/adoption_metrics.py`)은 이 + 세션과 무관하게 이미 완료되어 있었음 — 이 작업으로 인한 충돌은 + 없었다. diff --git a/handoff.md b/handoff.md index 66d20cf..70dbfff 100644 --- a/handoff.md +++ b/handoff.md @@ -94,6 +94,25 @@ correctly blocked (agy still has no public registry, per `adapters/antigravity/README.md`) -- not everything merged from that worktree closes every item tied to it. +**Supply-chain attestation completed** (`18d4662`, the item the +"Backlog reconciliation" note below left open): `.github/workflows/release.yml` +now packages `skills/adr-toolkit/` into a version-named tarball, SHA-256 +checksums it, and generates a Sigstore-backed GitHub Artifact Attestation +(`actions/attest-build-provenance@v2`, keyless/OIDC -- no private key to +manage or rotate) for it; both the tarball and checksum are attached to +the GitHub Release. Chose this over signing the git tag itself because +tags in this project are created locally by a human before the triggering +push (`AGENTS.md`'s documented release process), so CI has no way to +retroactively sign an already-pushed tag -- attestation instead ties +provenance to the exact commit the tag points to. `SECURITY.md` gained a +"Verifying a Release" section (`sha256sum -c` + `gh attestation verify`) +that also clarifies git-clone/adapter-install paths verify via Git/GitHub +history, not this archive. Full option analysis: +`docs/worklogs/2026-09-01-supply-chain-attestation.md`. Verified: YAML +syntax, full test suite green (541 passed, no regressions) -- the actual +OIDC attestation issuance/verification flow itself can only be confirmed +end-to-end on a real `v*` tag push, not locally. + **Adoption metrics follow-up completed:** - [x] `9a0de45`..`f814d64` add the design, JSON-only @@ -174,11 +193,9 @@ code again: Re-verified against the actual merged code (not assumed): confirmed `scripts/sync_version.py`/`release.yml` still do manual-only version bumps (no conflict with the audit's recommendation -- that review item - is now closed, see `improvements.md`'s `## Done`), and confirmed - `.github/workflows/release.yml` still has no supply-chain checksum/ - signing step (that item is now open and startable in this worktree, - not blocked by a concurrent editor anymore -- but touches the release - pipeline, so confirm with the owner before starting). + is now closed, see `improvements.md`'s `## Done`); the supply-chain + checksum/signing gap this note originally flagged is also closed now + (`18d4662`, GitHub Artifact Attestation -- see `## Done` above). - README prose (root README.md, `adapters/*/README.md` content) -- still another worktree's; every fix across all passes that touched adapter or generator code was a code fix, not README prose. @@ -187,32 +204,32 @@ code again: ## Next step (for a new session picking this up cold) -**One real, startable item exists in `improvements.md`'s `## Open`**: -supply-chain checksums/signing for `.github/workflows/release.yml` -(§2.2 2.2) -- verified not implemented, no longer blocked by a separate -worktree, but touches the release pipeline so confirm with the owner -before starting. Everything else is either precondition-gated or the -parallel Codex session's. Concretely: +**`improvements.md`'s `### High` section is now empty.** The last item +(supply-chain attestation) landed in `18d4662`. Everything left in +`## Open` is either precondition-gated on a real-world fact this +worktree can't change, or belongs to the parallel Codex session (already +done). Concretely: -1. `improvements.md`'s `### High` now has exactly 1 item (the - supply-chain one above) -- startable with owner confirmation, since - it modifies `release.yml`. +1. `improvements.md`'s `### Medium` and `### High` are both empty -- + nothing there to pick up. 2. `improvements.md`'s `### Low` → audit-report sub-group has exactly 1 item left (Antigravity in `harness-parity`), re-verified against `adapters/antigravity/README.md` and still blocked on an external fact (agy has no public package registry) -- don't start it. -3. `improvements.md`'s `### Low` → enterprise-adoption.md sub-group now has - only 3 precondition-gated items (repository going public, 2+ maintainers, +3. `improvements.md`'s `### Low` → enterprise-adoption.md sub-group has + 3 precondition-gated items (repository going public, 2+ maintainers, 2+ repositories) -- **not pure code tasks**. 4. If the user says "continue" / "다음 작업 진행해줘" without naming a - task: the supply-chain item is the one thing to offer; otherwise ask - what's next rather than inventing scope. + task: there is no ready-to-start backlog item left -- say so and ask + what's next (a new audit finding, a precondition that's now met, or + finishing the branch) rather than inventing scope. 5. If the user wants to finish this branch (merge to `develop` / open a PR): that decision was deferred every time it came up this session (owner chose "keep as-is" each time) -- ask again fresh, don't assume the answer carried forward. This branch already includes the merged `origin/develop` history, so a future merge/PR back to `develop` - should be a clean fast-forward-friendly merge. + should be a clean fast-forward-friendly merge. With the backlog now + empty of startable items, this is a reasonable point to raise it. 6. If the user references a new audit finding or a fresh problem: that's genuinely new work -- use the same pattern this session established (writing-plans -> executing-plans, TDD, one commit per task, verify diff --git a/improvements.md b/improvements.md index 950d5c5..97d43f0 100644 --- a/improvements.md +++ b/improvements.md @@ -16,12 +16,8 @@ closed out or reworded below. README prose is still another worktree's. ### High -- [ ] **공급망 보안(체크섬/서명)** — `.github/workflows/release.yml` - 확인 결과 여전히 테스트/버전 체크/릴리스 생성만 있고 SHA-256/Sigstore - 서명 단계는 없음. 더 이상 다른 워크트리가 이 파일을 동시에 만지고 - 있지 않으므로(그 작업은 이미 병합됨) 이제 이 워크트리에서 진행 가능 — - 다만 릴리스 파이프라인(운영 인프라)을 건드리는 작업이라 시작 전 - 오너 확인 필요. (감사 보고서 §2.2 2.2) +`### High`에 남은 항목 없음 — 마지막 항목(공급망 보안)은 아래 `## Done` +참고. ### Medium @@ -86,7 +82,15 @@ CI branch-coverage gate at 85% (measured baseline: 93.32%); `mypy --strict` CI gate + `core/contracts.py` (typed result shapes); `adr.py --diagnostic` timing flag; OS-level (fork+SIGKILL) proof that a mid-write crash never tears an ADR file; shared adapter-manifest validator -(`scripts/adapter_sdk.py`) used by all 4 manifest-based harness adapters. +(`scripts/adapter_sdk.py`) used by all 4 manifest-based harness adapters; +supply-chain build provenance attestation on the release pipeline +(`18d4662`) — `.github/workflows/release.yml` now packages +`skills/adr-toolkit/` into a version-named tarball, SHA-256 checksums it, +and generates a Sigstore-backed GitHub Artifact Attestation +(`actions/attest-build-provenance@v2`, keyless/OIDC) for it; `SECURITY.md` +gained a "Verifying a Release" section. This was the last item in +`### High`; see `docs/worklogs/2026-09-01-supply-chain-attestation.md` +for the full option analysis and rationale. **Medium** — common `AdrToolkitError` base class for all 6 domain exceptions (also closed a gap: `PathEscapesRootError` was raised but never From dd00e379519c5739e1f76bc52226990fe62a99c2 Mon Sep 17 00:00:00 2001 From: shcommit Date: Tue, 1 Sep 2026 17:57:11 +0900 Subject: [PATCH 55/58] docs: add worklogs for the Critical/High hardening pass's core decisions Documents atomic writes + directory locking, the two-stage ReDoS guard, typed result contracts + mypy --strict, and structured JSON logging with correlation IDs -- each grounded in the actual commits and test evidence from this session, per the troubleshooting-worklog skill format. --- .../2026-09-01-atomic-writes-and-locking.md | 106 ++++++++++++++++++ docs/worklogs/2026-09-01-redos-guard.md | 104 +++++++++++++++++ .../worklogs/2026-09-01-structured-logging.md | 90 +++++++++++++++ .../2026-09-01-typed-contracts-mypy-strict.md | 101 +++++++++++++++++ 4 files changed, 401 insertions(+) create mode 100644 docs/worklogs/2026-09-01-atomic-writes-and-locking.md create mode 100644 docs/worklogs/2026-09-01-redos-guard.md create mode 100644 docs/worklogs/2026-09-01-structured-logging.md create mode 100644 docs/worklogs/2026-09-01-typed-contracts-mypy-strict.md diff --git a/docs/worklogs/2026-09-01-atomic-writes-and-locking.md b/docs/worklogs/2026-09-01-atomic-writes-and-locking.md new file mode 100644 index 0000000..9f53678 --- /dev/null +++ b/docs/worklogs/2026-09-01-atomic-writes-and-locking.md @@ -0,0 +1,106 @@ +# ADR/exception/supersede 동시성 레이스 컨디션 제거 + +## 날짜 + +2026-09-01 (원 구현은 `2026-09-01` Critical 하드닝 패스) + +## 문제 상황 + +`docs/adr-toolkit-audit-report.md`가 지적한 Critical 항목: `create`, +`exception`, `supersede` 세 커맨드 모두 "다음 순번 계산 → 파일 존재 확인 +→ `Path.write_text()`로 직접 쓰기" 순서로 동작했다. 이 세 단계 사이에 +원자성이 전혀 없어서, 같은 저장소에 대해 두 프로세스가 동시에 +`adr.py create`를 호출하면 둘 다 같은 다음 번호(예: `ADR-0012`)를 +계산해서 서로의 파일을 덮어쓸 수 있었다. + +## 기존 구조나 방식의 한계 + +- ID 할당에 락이나 원자적 연산이 전혀 없어, "다음 번호 읽기"와 "그 + 번호로 쓰기" 사이에 임의의 다른 프로세스가 끼어들 수 있었다. +- `Path.write_text()`는 원자적이지 않다 — 쓰기 도중 프로세스가 + 죽으면(OOM kill, SIGKILL, 정전) 파일이 반쯤 쓰인 상태로 남을 수 있다. +- 실제로 재현해서 확인함: 수정 전 코드로 20개 concurrent `create` 호출을 + 실행하자 고유 ADR ID가 13개만 생성됐다(`49ede49` 커밋 메시지에 기록). + `exception`도 동일한 방식으로 재현: 20개 중 18개만 고유 ID(`68bbd98`). + +## 관련 코드 맥락 + +- `skills/adr-toolkit/scripts/commands/create.py`의 `run()` — 다음 ID + 계산 후 `_build_frontmatter()`로 프런트매터를 만들고 파일에 쓰는 + 부분이 원래 아무 보호 없이 실행됐다. +- `skills/adr-toolkit/scripts/commands/exception.py`의 `run()` — 동일한 + 패턴이지만, `SCHEMA_ERROR`(잘못된 draft) 시에는 애초에 아무 파일도 + 만들면 안 된다는 기존 테스트 제약이 있었다. +- `skills/adr-toolkit/scripts/commands/supersede.py`의 `run()` — 구 + ADR과 신규 ADR 두 파일을 순서대로 갱신하는데, 첫 파일 쓰기 후 둘째 + 파일 쓰기가 실패하면 롤백을 시도하는 기존 로직이 이미 있었다. + +## 검토한 선택지 + +1. **파일 시스템 락 없이 재시도/충돌 감지만 추가** — 쓰기 후 "내가 쓴 + 파일이 맞는지" 재확인하는 낙관적 동시성 제어. 구현이 복잡해지고 + 재시도 로직 자체에 새로운 엣지 케이스가 생김. 기각. +2. **SQLite 등 외부 상태 저장소로 ID 카운터 이전** — 이 프로젝트의 + "third-party 런타임 의존성 0개" 원칙(zero-dependency 아키텍처 + 가치)과 정면충돌. 기각. +3. **크로스 플랫폼 파일 락(`fcntl`/`msvcrt`) + 임시파일→`os.replace` + 원자적 쓰기** — 표준 라이브러리만으로 두 문제(레이스 컨디션, 쓰기 + 도중 크래시)를 동시에 해결. 채택. + +## 판단 기준 + +- 이 프로젝트는 third-party 런타임 의존성이 전혀 없는 것이 의도된 + 아키텍처 가치(감사 세션 내내 여러 번 확인됨) — 표준 라이브러리만으로 + 해결 가능한지가 최우선 기준. +- Windows/macOS/Linux 3.9~3.12 CI 매트릭스를 그대로 지원해야 하므로, + POSIX 전용 API(`fcntl`)만으로는 부족하고 플랫폼 분기가 필요. +- 기존 dry-run 테스트("dry-run은 아무것도 생성하면 안 된다")를 깨지 + 않아야 함. + +## 최종 결정 + +`core/atomic_io.py`에 `atomic_write_text()`(임시파일 + `os.fsync` + +`os.replace`)와 `adr_directory_lock()`(POSIX `fcntl.flock` / Windows +`msvcrt.locking` 컨텍스트 매니저)을 만들고, `create`/`exception`/ +`supersede`의 실제 쓰기 경로를 이 두 프리미티브로 감쌌다. + +## 해결 방식 + +1. `fc46830` — `core/atomic_io.py` 신설. 아직 어떤 커맨드에도 연결하지 + 않은 순수 프리미티브 단계로 먼저 커밋(리뷰 단위를 작게 유지). +2. `49ede49` — `create.py`: ID 할당 + 존재 확인 + 쓰기 전체를 + `adr_directory_lock()`으로 감싸고 `write_text`를 + `atomic_write_text`로 교체. **dry-run 경로는 락 밖에 완전히 남겨둠** + — dry-run이 `adr_dir`나 락 파일을 생성하면 기존 테스트가 깨지고, + dry-run은 아무것도 영속화하지 않으므로 거기서 레이스가 나도 무해함. +3. `68bbd98` — `exception.py`도 같은 패턴이되 한 가지 추가 보정: 스키마 + 검증은 **디스크를 건드리기 전에** preview ID로 미리 수행. 유효성 + 검증 결과가 실제로 어떤 순번을 받는지와 무관하기 때문에, 이렇게 + 해야 "SCHEMA_ERROR 시 exceptions_dir조차 만들면 안 된다"는 기존 + 테스트와 "dry-run은 아무것도 안 만든다"는 테스트를 모두 만족시킴. + 락은 최종 ID 할당 + 원자적 쓰기 구간만 감쌈. +4. `cec7215` — `supersede.py`: 두 파일 갱신 전체를 락으로 감싸고, 두 + 번의 `write_text` 호출과 롤백 쓰기까지 전부 + `atomic_write_text`로 교체. 기존 테스트 2개 + (`test_supersede_rolls_back_old_file_when_new_file_write_fails`, + `test_supersede_double_write_failure_reports_inconsistent_state_not_silent`)가 + `Path.write_text`를 직접 monkeypatch해서 쓰기 실패를 흉내내고 + 있었는데, 그 시드(seam)가 사라져서 + `supersede.atomic_io.atomic_write_text`로 재타게팅 — 테스트의 원래 + 의도와 검증 내용은 그대로 유지. + +## 결과 + +- 재현 테스트로 수정 전 실패(레이스 발생)를 먼저 확인한 뒤 수정 → + TDD 순서를 지킴. +- OS 레벨 검증(fork + SIGKILL)까지 별도로 수행해, 쓰기 도중 프로세스가 + 강제 종료돼도 ADR 파일이 반쯤 쓰인 상태로 남지 않음을 확인 + (High-priority 패스, `9708bb2`). +- 알려진 남은 한계 (`handoff.md`에 기록): `supersede`의 개별 파일 쓰기는 + 각각 원자적이지만, 두 파일 쓰기 "사이"에 프로세스가 죽으면 두 파일 + 쌍 전체의 일관성(진짜 2-phase commit)까지는 보장하지 않음 — 의도적으로 + 범위 밖으로 둔 것. +- 성공적인 `create`/`exception`/`supersede` 호출마다 `.adr-toolkit.lock` + (0바이트 dotfile)이 `docs/decisions/`, `docs/decisions/exceptions/` + 안에 영구히 남는다 — 크로스 프로세스 뮤텍스로서 의도된 동작이며 + `*.md`/`*.json` glob과 충돌하지 않음. diff --git a/docs/worklogs/2026-09-01-redos-guard.md b/docs/worklogs/2026-09-01-redos-guard.md new file mode 100644 index 0000000..1ef7bf4 --- /dev/null +++ b/docs/worklogs/2026-09-01-redos-guard.md @@ -0,0 +1,104 @@ +# CHECK의 사용자 정의 regex에 대한 ReDoS 방어 (런타임 + 정적, 2단계) + +## 날짜 + +2026-09-01 + +## 문제 상황 + +`constraints:` 블록의 `forbidden_import`/`dependency_forbidden` 규칙은 +작성자가 임의의 정규식(`pattern` 필드)을 직접 써서 diff의 추가된 줄에 +매칭시킨다(`rules/conflict.py::_content_pattern`). 이 정규식은 ADR +작성자가 통제하므로, 실수든 악의든 `(a+)+$`류의 catastrophic +backtracking 패턴이 들어가면 `re.search()`가 사실상 무한정 멈추지 +않는다 — CHECK 전체가 그 한 줄에서 행(hang)될 수 있었다. + +## 기존 구조나 방식의 한계 + +- `_content_pattern`이 `re.compile(pattern).search(line)`을 아무 보호 + 장치 없이 호출했다. +- 이 CLI는 CI 파이프라인(`harness-parity`, PR 체크 등)에서 자동 실행되는 + 경우가 많아, 한 번 hang이 나면 사람이 알아채기 전까지 CI 러너가 + 타임아웃될 때까지 계속 잡아먹는다. +- Python 표준 라이브러리에는 정규식 실행 타임아웃 기능이 없다 — + `signal.alarm`/`setitimer`로 직접 인터럽트를 걸어야 하는데, 이는 + **POSIX 전용**(Windows는 `SIGALRM` 자체가 없음)이라 단일 메커니즘으로 + 모든 CI 플랫폼(ubuntu/macos/windows)을 커버할 수 없었다. + +## 관련 코드 맥락 + +- `skills/adr-toolkit/scripts/rules/conflict.py::_content_pattern()` — + diff의 각 추가된 줄에 대해 규칙의 모든 `pattern`을 매칭 시도하는 + 실제 실행 지점. +- `skills/adr-toolkit/scripts/commands/check.py` — 기존에 이미 + `except re.error`로 정규식 컴파일 오류를 잡아 `BAD_CONSTRAINTS` + 경고로 격하시키는 처리 경로가 있었음 — 이 기존 경로를 재사용할 수 + 있는지가 설계의 핵심이었다. +- `skills/adr-toolkit/scripts/core/constraints.py::_parse_rules()` — + `constraints:` YAML 블록을 파싱해서 규칙 리스트를 만드는 곳. 여기서 + `pattern` 값 자체를 파싱 시점에 검사할 수 있다는 게 두 번째 방어선의 + 근거. + +## 검토한 선택지 + +1. **정규식 엔진을 `re2`류 선형 시간 엔진으로 교체** — third-party + 의존성 추가가 필요해 zero-dependency 원칙과 충돌. 기각. +2. **`multiprocessing`으로 정규식 실행을 별도 프로세스에 격리하고 + `terminate()`** — 플랫폼 독립적이지만, 매 diff 라인마다 프로세스를 + 새로 띄우는 오버헤드가 크고, CHECK는 원래 빠른 사전 검증 도구라는 + 설계 의도와 어긋남. 기각. +3. **POSIX `SIGALRM`/`setitimer` 기반 런타임 타임아웃만 적용** — + 구현이 단순하고 기존 `except re.error` 경로에 자연스럽게 편입되지만, + Windows에서는 완전히 무방비 상태로 남는 절반짜리 해법. +4. **런타임 타임아웃(3) + 파싱 시점 정적 휴리스틱(중첩 quantifier + 거부)을 함께 적용** — Windows/POSIX 모두 최소한의 방어선을 갖도록 + 2단계로 방어. 채택. + +## 판단 기준 + +- CI 매트릭스가 ubuntu/macos/windows 3개 플랫폼을 전부 포함하므로, + "POSIX에서만 동작하는 방어"는 감사 관점에서 "Windows 미방어"라는 + 별도의 Open Risk로 남는다 — 완전히 무시할 수 없음. +- 기존 에러 처리 경로(`except re.error`)를 재사용할 수 있으면 새 + 실패 모드를 추가하지 않고 통합할 수 있다 — 최소 침습 우선. +- 정적 검사는 오탐(false positive)이 나면 정상적인 규칙 작성을 막으므로, + 가장 흔하고 확실한 패턴 모양(중첩 quantifier)만 좁게 잡는 휴리스틱이 + 안전하다. + +## 최종 결정 + +옵션 4 — `rules/conflict.py`에 런타임 SIGALRM 타임아웃 가드를 +추가하고(POSIX만 유효, Windows는 가드 없이 그냥 실행), 별도로 +`core/constraints.py`에 파싱 시점 정적 중첩-quantifier 거부 로직을 +추가해 플랫폼 무관하게 최소 방어선을 확보했다. + +## 해결 방식 + +1. `c0ff907` — `rules/conflict.py`에 `RegexTimeout(re.error)` 예외 + 클래스와 `_guarded_search(regex, line)` 추가. `_REGEX_TIMEOUT_SECONDS + = 0.25`로 `signal.alarm`을 걸고, 시간 초과 시 `RegexTimeout`을 + 발생시킴 — `re.error`의 서브클래스이므로 `check.py`의 기존 + `except re.error` 처리가 코드 수정 없이 그대로 이를 `BAD_CONSTRAINTS` + 경고로 격하시킴. `(a+)+$`류 실제 catastrophic backtracking 패턴으로 + 검증: 가드가 없으면 멈추던 것이 1초 이내에 인터럽트됨. +2. `26021a9` (Medium 패스) — `core/constraints.py`에 + `_NESTED_QUANTIFIER_RE = re.compile(r"\([^()]*" + _QUANTIFIER + r"\)" + + _QUANTIFIER)`와 `_reject_if_redos_prone(pattern)`을 추가해 + `_parse_rules()`의 후처리 단계에서 호출. `forbidden_import`/ + `dependency_forbidden`에만 적용 — `required_path`/`forbidden_path`는 + `pattern`을 glob으로 취급(`core/globs.py` 경유)해서 애초에 + catastrophic backtracking이 발생할 수 없는 구조라, 여기에 적용하면 + 오탐이 된다. 실제로 dogfooding 중인 `ADR-0011`의 constraints 블록으로 + 회귀 테스트: 정상 패턴은 여전히 통과, 위험 패턴은 `re.compile()`에 + 도달하기 전에 걸러짐(테스트에서 `re.compile`을 monkeypatch해서 호출 + 자체가 안 됨을 확인). + +## 결과 + +- 두 커밋 모두 실패 재현 → 가드 추가 → 통과 확인 순서로 진행. +- 남은 한계(코드 주석 및 `handoff.md`에 명시): 이 정적 휴리스틱은 가장 + 흔한 "중첩 quantifier" 모양만 잡는 것이지 범용 ReDoS 탐지기가 + 아니다 — `(a|a)*`류 alternation 기반 패턴은 여전히 미탐지 상태로 + 남으며, Windows에서는 이 정적 검사가 유일한 방어선이다(런타임 + SIGALRM 가드가 없으므로). +- 전체 테스트 스위트 회귀 없음 확인 후 커밋. diff --git a/docs/worklogs/2026-09-01-structured-logging.md b/docs/worklogs/2026-09-01-structured-logging.md new file mode 100644 index 0000000..2c5bea1 --- /dev/null +++ b/docs/worklogs/2026-09-01-structured-logging.md @@ -0,0 +1,90 @@ +# 상관관계 ID를 포함한 구조화 JSON 에러 로깅 + +## 날짜 + +2026-09-01 + +## 문제 상황 + +`adr.py`에서 예상치 못한 예외가 발생하면 최상위 `except` 블록이 이를 +잡아 JSON 에러 응답으로 stdout에 내보냈지만, 그 과정에서 스택 트레이스나 +예외 컨텍스트는 어디에도 기록되지 않고 사라졌다. CI나 에이전트 하네스가 +"어떤 실행에서 어떤 에러가 났는지"를 나중에 추적할 방법이 없었다. + +## 기존 구조나 방식의 한계 + +- stdout은 이 프로젝트의 확고한 계약(ADR-0009: "always JSON contract")이라 + 사람이 읽는 진단 로그를 stdout에 섞을 수 없다. +- 그렇다고 `print(traceback, file=sys.stderr)` 같은 비구조화 텍스트를 + 찍으면, 여러 프로세스가 동시에 도는 CI 환경에서 어떤 stderr 줄이 어떤 + stdout JSON 응답과 짝인지 알 방법이 없다. +- 표준 `logging` 모듈을 그냥 쓰면 pytest의 capsys 캡처와 충돌하기 쉽고 + (핸들러가 이전 테스트에서 누적됨), 프로덕션에서도 매 호출마다 새 + 핸들러가 쌓이는 문제가 생긴다. + +## 관련 코드 맥락 + +- `skills/adr-toolkit/scripts/adr.py`의 최상위 `main()` 함수 — + 예외를 잡는 유일한 지점. 여기서 로거를 얻어 기록하고, 동일한 + 상관관계 ID를 stdout JSON 에러 응답에도 넣어야 두 출력을 나중에 + 매칭할 수 있다. +- `skills/adr-toolkit/scripts/core/telemetry.py`(신규) — + `_JsonLogFormatter`와 `get_logger(operation, *, correlation_id=None)`를 + 정의. 매 호출마다 `logger.handlers.clear()`로 핸들러를 비우고 다시 + 구성하는 게 핵심 — 이래야 pytest에서 각 테스트가 독립적으로 stderr를 + 캡처할 수 있고, 프로덕션에서도 핸들러가 무한정 누적되지 않는다. + +## 검토한 선택지 + +1. **외부 로깅 서비스(Sentry, Datadog 등) 연동** — 네트워크 의존성과 + API 키 관리가 필요해 이 CLI의 "설치 즉시 동작"하는 사용 모델과 + 맞지 않고, 제로 의존성 원칙과도 충돌. 기각. +2. **비구조화 stderr 텍스트 로그(`print(..., file=sys.stderr)`)** — + 구현은 가장 간단하지만, CI 로그가 뒤섞이는 환경에서 특정 실패를 + 특정 stdout 응답과 연결할 방법이 없다. 기각. +3. **표준 `logging` 모듈 + JSON Lines 포맷 + 상관관계 ID를 stdout·stderr + 양쪽에 동일하게 포함** — 표준 라이브러리만 사용하고, 매 요청마다 + 고유 ID를 발급해 두 출력 스트림을 연결할 수 있다. 채택. + +## 판단 기준 + +- stdout의 JSON 전용 계약(ADR-0009)을 절대 깨지 않을 것 — 상관관계 + ID는 stdout 쪽 에러 응답에 필드 하나 추가하는 형태로만 들어간다. +- 제로 의존성 원칙 유지. +- 테스트 스위트(pytest)에서 stderr 캡처가 깨지지 않아야 함 — 이는 + 구현 중 실제로 부딪힌 문제였고, 핸들러를 매번 초기화하는 방식으로 + 해결했다. + +## 최종 결정 + +`core/telemetry.py`에 JSON Lines 포맷 로거를 만들고, `adr.py`의 +전역 예외 핸들러가 여기에 예외를 기록하면서 동일한 상관관계 ID를 +stdout JSON 에러 응답에도 포함시킨다. + +## 해결 방식 + +1. `11c8f4b` — `core/telemetry.get_logger(operation)`이 + `LoggerAdapter`를 반환하도록 구현. 각 로그 라인은 JSON Lines + 형식으로 `level`, `operation`, `correlation_id`, `message`, + (예외 시) `exception_type` 필드를 담아 stderr에 출력된다. +2. `adr.py`의 예외 핸들러를 `logger = get_logger(args.operation); + logger.exception(...)` 형태로 바꾸고, 동일한 `correlation_id`를 + stdout으로 나가는 JSON 에러 응답에도 추가 — CI 로그에서 stderr의 + 특정 줄과 stdout의 특정 실패 응답을 상관관계 ID로 매칭할 수 있게 함. +3. 기본 로그 레벨은 `WARNING`(성공 시 조용함), `ADR_TOOLKIT_LOG_LEVEL` + 환경변수로 오버라이드 가능. +4. `mypy --strict` 게이트를 통과시키는 과정에서 발견된 타입 이슈(문서 + `2026-09-01-typed-contracts-mypy-strict.md` 참고)도 이 모듈에서 + 함께 수정됨 — `Iterator[None]` 반환 타입, `exc_info[0]` narrowing, + `LoggerAdapter` 제네릭 파라미터화. + +## 결과 + +- stdout의 순수 JSON 계약은 상관관계 ID 필드 추가 외에는 변경 없음 — + 기존 소비자(에이전트, CI 스크립트)와 호환. +- pytest의 capsys 기반 테스트가 핸들러 누적 없이 안정적으로 stderr를 + 캡처함을 확인. +- 같은 세션에서 이어진 TTY 전용 사람 친화적 요약 줄(`c3ed01d`, + Medium 패스)은 이 구조화 로깅과는 별개 기능 — 그쪽은 + `sys.stderr.isatty()`일 때만 보이는 인간용 한 줄 요약이고, 이 + telemetry 로거는 항상 JSON Lines로 기록되는 기계 판독용 로그다. diff --git a/docs/worklogs/2026-09-01-typed-contracts-mypy-strict.md b/docs/worklogs/2026-09-01-typed-contracts-mypy-strict.md new file mode 100644 index 0000000..fe12401 --- /dev/null +++ b/docs/worklogs/2026-09-01-typed-contracts-mypy-strict.md @@ -0,0 +1,101 @@ +# 타입드 결과 계약(`core/contracts.py`)과 범위 한정 `mypy --strict` 게이트 + +## 날짜 + +2026-09-01 + +## 문제 상황 + +`adr.py`의 모든 커맨드는 "stdout은 항상 순수 JSON"이라는 계약(ADR-0009)을 +따르지만, 그 JSON이 실제로 어떤 키를 가지는지는 각 커맨드의 `run()` +함수 본문을 읽어야만 알 수 있었다 — 타입 시스템 차원에서 보장되는 +스키마가 전혀 없었다. 감사 보고서는 이를 "출력 계약이 코드에만 존재하고 +타입으로 고정되지 않음" 문제로 지적했다. + +## 기존 구조나 방식의 한계 + +- 16개 커맨드 모두 `dict`를 리턴하며, 필드 이름 오타나 필드 누락이 + 런타임에만(혹은 소비하는 쪽 에이전트/스크립트에서만) 드러남. +- `jsonschema` 같은 런타임 스키마 검증 라이브러리를 쓰면 즉시 해결될 + 것처럼 보이지만, 이 프로젝트의 zero-dependency 원칙과 충돌한다. +- `argparse.Namespace`로 넘어오는 커맨드 인자(`args`)는 동적 속성 + 접근이라 `TypedDict`로 감싸기 어렵다 — 인자 쪽까지 완전히 타입화하려면 + `Protocol` 기반의 더 큰 리팩터링이 필요해서, 이번 패스에서는 "출력 + 결과 타입"만 범위로 잡았다. + +## 관련 코드 맥락 + +- `skills/adr-toolkit/scripts/core/atomic_io.py`, + `core/telemetry.py` — 이번 세션에서 새로 만든, 처음부터 완전히 + 타입 주석이 붙은 두 모듈. `mypy --strict` 게이트의 첫 적용 대상. +- `skills/adr-toolkit/scripts/commands/create.py`의 `run()` 리턴문 — + `CreateResult` TypedDict의 필드 목록을 정할 때 실제 리턴 딕셔너리를 + 읽고 역으로 타입을 뽑아냄(추측이 아니라 코드에서 도출). +- `.github/workflows/test.yml`의 `type-check` job — `mypy --strict`를 + 세 모듈(`atomic_io`, `telemetry`, `contracts`)에만 한정해서 실행. + +## 검토한 선택지 + +1. **`jsonschema` 도입 + JSON Schema로 런타임 검증** — 스펙 표준이라는 + 장점은 있지만 제로 의존성 원칙 위반. 기각. +2. **`dataclasses`로 결과 객체를 감싸고 `asdict()`로 직렬화** — 런타임 + 오버헤드와 기존 dict 기반 코드 전체(16개 커맨드, 관련 테스트 전부)를 + 바꿔야 하는 큰 리팩터링이 필요해 이번 감사 대응 범위에 비해 과함. + 기각. +3. **`TypedDict` + `mypy --strict`를 다 타입화된 모듈에만 우선 + 적용** — 런타임 동작을 전혀 바꾸지 않고(TypedDict는 런타임에 아무 + 효과가 없음) 정적 분석만으로 계약을 문서화·검증. 채택. + +## 판단 기준 + +- 런타임 동작을 바꾸지 않으면서 "출력 계약을 코드로 고정"하는 최소 + 침습적 방법이 우선. +- 이미 확인된 제로 의존성 제약을 다시 어기지 않을 것. +- `argparse.Namespace` 타입화라는 더 큰 리팩터링까지 한 패스에 묶으면 + 범위가 과도하게 커지므로, "출력 타입만" 먼저 고정하고 인자 타입화는 + 모듈 docstring에 명시적으로 향후 과제로 남긴다. + +## 최종 결정 + +`core/contracts.py`에 커맨드별 결과 `TypedDict`를 정의하고, +`mypy --strict` CI 게이트를 새로 만들되 이미 완전히 타입 주석이 붙은 +핵심 모듈에만 적용한다. 명령어 인자(`argparse.Namespace`) 타입화는 +범위 밖으로 명시적으로 남긴다. + +## 해결 방식 + +1. `305c836` — `core/contracts.py` 신설: + `CommandError`/`BaseResult`/`ErrorResult`/`CreateResult` 4개 + TypedDict로 시작. `type-check` CI job을 추가해 `atomic_io`, + `telemetry`, `contracts` 세 모듈에 `mypy --strict` 적용. 이 과정에서 + 실제로 발견한 3개의 진짜 mypy 오류를 수정: + - `adr_directory_lock`의 컨텍스트 매니저 제너레이터에 `Iterator[None]` + 반환 타입 누락. + - `record.exc_info[0]`가 `Optional[type[BaseException]]`이라 미리 + narrowing 없이 쓰면 오류 — `and record.exc_info[0] is not None` + 가드 추가. + - `logging.LoggerAdapter`를 파라미터화하지 않은 제네릭으로 써서 + 오류 — `"logging.LoggerAdapter[logging.Logger]"` 문자열 애노테이션으로 + 해결. + - (나중 커밋에서 추가 발견) TypedDict 필드에 맨 `dict`를 쓰면 + `mypy --strict`의 `type-arg` 검사에 걸림 — `Dict[str, Any]`로 + 교체해야 함. +2. `1df6066` (팔로우업) — 나머지 14개 커맨드(`preflight`, `discover`, + `init`, `index`, `related`, `significance`, `validate`, `status`, + `supersede`, `diff`, `exception`, `graph`, `search`)까지 확장해 + 2/16 → 16/16 커버리지. 각 TypedDict 필드는 실제 `run()` 리턴문을 + 읽어서 결정했고(추측 금지), `status`/`supersede`의 에러 경로까지 + 실제로 실행해서 대조 확인. 에러/경고/중첩 페이로드 필드는 공용 + `CommandError` 타입 대신 `Dict[str, Any]`를 쓴 경우가 있는데, 이는 + 실제 에러 딕셔너리가 `file`/`id`/`ids`/`cycle` 등 `CommandError`가 + 선언하지 않은 추가 필드를 갖는 커맨드가 있어서, 사실이 아닌 구조를 + 타입으로 과장하지 않기 위함. + +## 결과 + +- `mypy --strict`가 실제로 CI에 게이트로 걸려 통과 상태 유지 중. +- 남은 과제(코드 주석 및 `handoff.md`에 기록): `mypy --strict`를 16개 + 커맨드 모듈 자체(현재는 결과 타입만 타입화되고 커맨드 구현부는 + 미적용)까지 확장하는 건 `argparse.Namespace` 타입화 리팩터링이 + 선행돼야 해서 여전히 향후 과제로 남아있다. +- 전체 테스트 스위트 회귀 없이 각 커밋 완료. From c59d3dbbc6e830036a9af74f60e4e77e950a67f1 Mon Sep 17 00:00:00 2001 From: shcommit Date: Wed, 2 Sep 2026 09:22:12 +0900 Subject: [PATCH 56/58] docs: record this session's hardening decisions as ADR-0012..0016 Uses the ADR toolkit itself to document the Critical/High-priority hardening work: atomic writes + directory locking, the two-stage ReDoS guard, typed result contracts + mypy --strict, structured JSON logging with correlation IDs, and release artifact attestation. Each ADR passed significance scoring (recommended band) and `related` conflict checks before creation; all 16 ADRs now validate and the index/graph are regenerated. Also removes docs/worklogs/ (superseded by these ADRs and backed up externally) and gitignores the .adr-toolkit.lock runtime mutex file, which was never meant to be committed. --- .gitignore | 4 + ...king-for-create-exception-and-supersede.md | 80 +++++++++++++ ...ck-s-author-supplied-constraint-regexes.md | 87 ++++++++++++++ ...tracts-and-a-scoped-mypy-strict-ci-gate.md | 77 +++++++++++++ ...ith-correlation-ids-for-uncaught-errors.md | 77 +++++++++++++ ...tation-instead-of-a-managed-signing-key.md | 83 ++++++++++++++ docs/decisions/README.md | 107 ++++++++++++++++++ docs/decisions/relationships.mmd | 13 +++ docs/decisions/relationships.svg | 45 ++++++-- .../2026-09-01-atomic-writes-and-locking.md | 106 ----------------- docs/worklogs/2026-09-01-redos-guard.md | 104 ----------------- .../worklogs/2026-09-01-structured-logging.md | 90 --------------- .../2026-09-01-supply-chain-attestation.md | 102 ----------------- .../2026-09-01-typed-contracts-mypy-strict.md | 101 ----------------- 14 files changed, 566 insertions(+), 510 deletions(-) create mode 100644 docs/decisions/0012-atomic-writes-and-cross-platform-directory-locking-for-create-exception-and-supersede.md create mode 100644 docs/decisions/0013-two-stage-redos-defense-for-check-s-author-supplied-constraint-regexes.md create mode 100644 docs/decisions/0014-typed-result-contracts-and-a-scoped-mypy-strict-ci-gate.md create mode 100644 docs/decisions/0015-structured-json-stderr-logging-with-correlation-ids-for-uncaught-errors.md create mode 100644 docs/decisions/0016-sign-release-artifacts-with-github-artifact-attestation-instead-of-a-managed-signing-key.md delete mode 100644 docs/worklogs/2026-09-01-atomic-writes-and-locking.md delete mode 100644 docs/worklogs/2026-09-01-redos-guard.md delete mode 100644 docs/worklogs/2026-09-01-structured-logging.md delete mode 100644 docs/worklogs/2026-09-01-supply-chain-attestation.md delete mode 100644 docs/worklogs/2026-09-01-typed-contracts-mypy-strict.md diff --git a/.gitignore b/.gitignore index 3d12363..e01e0c9 100644 --- a/.gitignore +++ b/.gitignore @@ -35,3 +35,7 @@ adapters/antigravity/skills/adr-toolkit # docs/superpowers/specs/ (the design docs) stay tracked; only plans/ is # excluded. History from before this commit still has them if ever needed. docs/superpowers/plans/ + +# Cross-process mutex file created by core/atomic_io.py's directory lock +# during create/exception/supersede -- a runtime artifact, not source. +.adr-toolkit.lock diff --git a/docs/decisions/0012-atomic-writes-and-cross-platform-directory-locking-for-create-exception-and-supersede.md b/docs/decisions/0012-atomic-writes-and-cross-platform-directory-locking-for-create-exception-and-supersede.md new file mode 100644 index 0000000..cb625cd --- /dev/null +++ b/docs/decisions/0012-atomic-writes-and-cross-platform-directory-locking-for-create-exception-and-supersede.md @@ -0,0 +1,80 @@ +--- +id: ADR-0012 +title: Atomic writes and cross-platform directory locking for CREATE, EXCEPTION, and SUPERSEDE +status: accepted +date: 2026-09-01 +locale: en +decision_makers: + - YangSeungHyun +related: + - ADR-0006 +affected_paths: + - skills/adr-toolkit/scripts/core/atomic_io.py + - skills/adr-toolkit/scripts/commands/create.py + - skills/adr-toolkit/scripts/commands/exception.py + - skills/adr-toolkit/scripts/commands/supersede.py + - tests/unit/test_atomic_io.py +tags: + - concurrency + - reliability + - core + - v0.3.0 +retrospective: false +--- + +# Atomic writes and cross-platform directory locking for CREATE, EXCEPTION, and SUPERSEDE + +## Context and Problem Statement + +CREATE, EXCEPTION, and SUPERSEDE each compute the next sequential ID, check that no file with that ID exists yet, and write a new file with a direct `Path.write_text()` call. None of these three steps were protected by any lock or atomic operation. Reproducing the problem directly: 20 concurrent `create` invocations against the same repository produced only 13 unique ADR IDs, and 20 concurrent `exception` invocations produced only 18 unique IDs, before any fix. Separately, `write_text()` is not atomic -- a process killed mid-write (OOM kill, SIGKILL, power loss) can leave a torn, half-written file on disk. + +## Decision Drivers + +* This CLI has zero third-party runtime dependencies by design; the fix must use only the standard library. +* CI already runs the full test matrix on ubuntu, macOS, and Windows (Python 3.9-3.12), so any locking primitive must work identically on all three. +* Existing tests assert that a dry run creates nothing on disk (not even the ADR directory or a lock file); the fix must not break that guarantee. + +## Considered Options + +* Optimistic concurrency: write, then re-read to detect a collision, and retry +* Move ID allocation into an external state store (e.g. SQLite) +* Cross-platform advisory file locking (`fcntl` on POSIX, `msvcrt` on Windows) plus atomic temp-file-then-`os.replace()` writes + +## Decision Outcome + +Chosen option: **cross-platform advisory locking plus atomic writes**, because it solves both the ID-collision race and the torn-write problem using only the standard library, with no new runtime dependency and no change to the on-disk file format. + +`core/atomic_io.py` provides `atomic_write_text()` (write to a temp file, `fsync`, then `os.replace()`) and `adr_directory_lock()` (a context manager using `fcntl.flock` on POSIX and `msvcrt.locking` on Windows). CREATE and EXCEPTION wrap ID allocation, the existence check, and the write inside the lock; their dry-run and (for EXCEPTION) schema-validation-failure paths stay outside the lock entirely, since those paths must not create the ADR directory or a lock file. SUPERSEDE wraps its full two-file update in the same lock and routes both writes, plus its rollback write, through `atomic_write_text()`. + +### Consequences + +* Good: concurrent `create`/`exception`/`supersede` invocations can no longer allocate duplicate IDs or corrupt each other's files. +* Good: a process killed mid-write never leaves a torn ADR or exception file -- verified at the OS level with a fork+SIGKILL test. +* Bad: SUPERSEDE's two-file update guarantees each individual file is atomic, but not that the *pair* stays consistent if the process is killed between the two writes; true two-phase commit across both files was explicitly scoped out. +* Bad: every successful CREATE/EXCEPTION/SUPERSEDE call leaves a `.adr-toolkit.lock` file permanently in the ADR/exceptions directory, as the cross-process mutex. + +### Confirmation + +`tests/unit/test_atomic_io.py` covers the primitives directly; the CREATE/EXCEPTION/SUPERSEDE test suites include a concurrency reproduction (20 parallel invocations must all get unique IDs) and assert dry-run/schema-error paths still create nothing on disk. + +## Pros and Cons of the Options + +### Optimistic concurrency (write, detect, retry) + +* Good, because it needs no new locking primitive. +* Bad, because a correct retry loop is harder to get right than a lock, and it adds a new class of edge case (how many retries, what backoff) rather than removing one. + +### External state store (SQLite) + +* Good, because SQLite's own locking would handle the race correctly. +* Bad, because it introduces a new runtime dependency and a second source of truth alongside the Markdown/JSON files, conflicting with this project's zero-dependency, file-is-the-database design. + +### Cross-platform advisory locking + atomic writes + +* Good, because it uses only the standard library and fixes both the race and the torn-write problem in one primitive module. +* Bad, because POSIX and Windows locking semantics differ enough that the module needs two code paths, each tested separately. + +## Revisit Triggers + +* A future command needs true multi-file atomicity (e.g. an operation touching three or more files that must all succeed or all roll back together). +* Real-world usage shows the permanent `.adr-toolkit.lock` file causing confusion or tooling conflicts. diff --git a/docs/decisions/0013-two-stage-redos-defense-for-check-s-author-supplied-constraint-regexes.md b/docs/decisions/0013-two-stage-redos-defense-for-check-s-author-supplied-constraint-regexes.md new file mode 100644 index 0000000..e2a7cd0 --- /dev/null +++ b/docs/decisions/0013-two-stage-redos-defense-for-check-s-author-supplied-constraint-regexes.md @@ -0,0 +1,87 @@ +--- +id: ADR-0013 +title: Two-stage ReDoS defense for CHECK's author-supplied constraint regexes +status: accepted +date: 2026-09-01 +locale: en +decision_makers: + - YangSeungHyun +related: + - ADR-0002 + - ADR-0007 + - ADR-0008 +affected_paths: + - skills/adr-toolkit/scripts/rules/conflict.py + - skills/adr-toolkit/scripts/core/constraints.py + - tests/unit/test_conflict.py + - tests/unit/test_constraints.py +tags: + - security + - check + - core + - v0.3.0 +retrospective: false +--- + +# Two-stage ReDoS defense for CHECK's author-supplied constraint regexes + +## Context and Problem Statement + +`forbidden_import` and `dependency_forbidden` constraint rules let an ADR author write an arbitrary regular expression in the `pattern` field, which `rules/conflict.py` then matches against every added line in a diff. Because the author controls this regex, an accidental or malicious catastrophic-backtracking pattern (e.g. `(a+)+$`) can make `re.search()` hang for an unbounded time on ordinary input, stalling CHECK -- including in CI, where a single hung run occupies a runner until it times out. + +## Decision Drivers + +* CI runs on ubuntu, macOS, and Windows; a defense that only works on one platform family leaves the others fully exposed. +* Python's standard library has no regex execution timeout; the usual approach (`signal.alarm`/`setitimer`) is POSIX-only. +* `commands/check.py` already downgrades a `re.error` at pattern-compile time to a `BAD_CONSTRAINTS` warning; reusing that path avoids adding a new failure mode. +* No third-party regex engine (e.g. `re2`) may be introduced, per this project's zero-dependency constraint. + +## Considered Options + +* Replace Python's `re` with a linear-time third-party engine +* Run each regex match in a separate subprocess and kill it on timeout +* POSIX `SIGALRM`/`setitimer` runtime timeout only +* Runtime timeout (POSIX) plus a platform-independent static rejection of the most common catastrophic-backtracking shape at parse time + +## Decision Outcome + +Chosen option: **runtime timeout plus static rejection**, because a single platform-only defense leaves one whole platform family (Windows) with zero protection, and the static check closes that gap without a new dependency or a per-line subprocess. + +`rules/conflict.py` adds `RegexTimeout(re.error)` and `_guarded_search()`, which wraps `regex.search()` in a 0.25s `SIGALRM` timeout on POSIX; because `RegexTimeout` subclasses `re.error`, `check.py`'s existing `except re.error` handling downgrades a timeout to `BAD_CONSTRAINTS` with no code change there. Separately, `core/constraints.py` adds `_reject_if_redos_prone()`, a static check that rejects any `forbidden_import`/`dependency_forbidden` pattern containing a quantified group whose own body ends in a quantifier (e.g. `(a+)+`, `(a*)*`) at parse time, before the pattern ever reaches `re.compile()`. This check is scoped to only those two rule kinds; `required_path`/`forbidden_path` treat `pattern` as a glob via `core/globs.py`, which cannot produce catastrophic backtracking, so applying the same check there would be a false positive. + +### Consequences + +* Good: a catastrophic-backtracking pattern like `(a+)+$` is now interrupted well under 1 second on POSIX, and rejected outright at parse time on every platform if it matches the nested-quantifier shape. +* Good: no new dependency, and no change needed in `check.py`'s existing error handling. +* Bad: the static check is a heuristic for the single most common ReDoS shape, not a general detector -- alternation-based patterns such as `(a|a)*` are not caught, and on Windows (no `SIGALRM`) the static check is the *only* defense such a pattern would face. + +### Confirmation + +`tests/unit/test_conflict.py` verifies `(a+)+$` is interrupted by the timeout guard; `tests/unit/test_constraints.py` verifies a nested-quantifier pattern is rejected before `re.compile()` is ever called (via a monkeypatched `re.compile` that asserts it is not invoked), and that the real, already-dogfooded `ADR-0011` constraints block still validates cleanly. + +## Pros and Cons of the Options + +### Third-party linear-time regex engine + +* Good, because it would eliminate catastrophic backtracking structurally. +* Bad, because it requires a new runtime dependency, violating this project's zero-dependency architecture. + +### Subprocess-per-match with a kill timeout + +* Good, because it is platform-independent and enforces a true wall-clock timeout. +* Bad, because spawning a process per diff line is heavy overhead for what is meant to be a fast pre-commit/CI check. + +### POSIX-only runtime timeout + +* Good, because it is the simplest fix and slots directly into the existing `re.error` handling. +* Bad, because Windows CI is left with no protection at all -- a known, documented gap rather than a fix. + +### Runtime timeout + static rejection (chosen) + +* Good, because every supported platform gets at least one layer of defense. +* Bad, because the static heuristic only covers the nested-quantifier shape, not all ReDoS-prone patterns. + +## Revisit Triggers + +* A real ADR author needs a legitimate alternation-heavy pattern that the static check would need to distinguish from a ReDoS-prone one. +* Python gains a standard-library, cross-platform regex timeout mechanism, which would let the static heuristic be replaced by a strict runtime guard everywhere. diff --git a/docs/decisions/0014-typed-result-contracts-and-a-scoped-mypy-strict-ci-gate.md b/docs/decisions/0014-typed-result-contracts-and-a-scoped-mypy-strict-ci-gate.md new file mode 100644 index 0000000..db559b8 --- /dev/null +++ b/docs/decisions/0014-typed-result-contracts-and-a-scoped-mypy-strict-ci-gate.md @@ -0,0 +1,77 @@ +--- +id: ADR-0014 +title: Typed result contracts and a scoped mypy --strict CI gate +status: accepted +date: 2026-09-01 +locale: en +decision_makers: + - YangSeungHyun +related: + - ADR-0009 +affected_paths: + - skills/adr-toolkit/scripts/core/contracts.py + - .github/workflows/test.yml + - tests/unit/test_contracts.py +tags: + - typing + - contract + - core + - v0.3.0 +retrospective: false +--- + +# Typed result contracts and a scoped mypy --strict CI gate + +## Context and Problem Statement + +Every one of the 16 commands returns a plain `dict` as its stdout JSON result, per this project's fixed "stdout is always JSON" contract. But that JSON's actual shape existed only in each command's `run()` function body -- there was no type-level definition of what keys a caller (an agent, a CI script) could rely on. A typo'd or dropped field would only surface at runtime, in whichever consumer happened to read it. + +## Decision Drivers + +* No third-party runtime dependency may be introduced (rules out `jsonschema`-based runtime validation). +* The fix should not change runtime behavior -- only make the existing contract checkable statically. +* `argparse.Namespace`-based command arguments resist `TypedDict` typing without a larger `Protocol`-based refactor; that refactor is out of scope for this pass. + +## Considered Options + +* Add `jsonschema` and validate every command's output against a JSON Schema at runtime +* Wrap every result in a `dataclass` and serialize with `asdict()` +* Define `TypedDict` result shapes in a new `core/contracts.py` and gate them with `mypy --strict`, scoped to already-fully-typed modules + +## Decision Outcome + +Chosen option: **`TypedDict` contracts plus a scoped `mypy --strict` gate**, because `TypedDict` has zero runtime cost, requires no new dependency, and lets the type checker -- rather than a runtime validator -- catch a shape mismatch before it ships. + +`core/contracts.py` defines one `TypedDict` per command's result shape, covering all 16 commands. A `type-check` CI job runs `mypy --strict` over the modules that are fully type-annotated (`atomic_io`, `telemetry`, `contracts`); extending strict mode into the 16 command modules themselves is deferred until their `argparse.Namespace` arguments are typed. Each `TypedDict`'s fields were read directly from the corresponding command's actual `return` statements -- including at least one error-path branch for `status` and `supersede` -- rather than inferred from documentation. Where a command's real error or warning payload carries fields the shared `CommandError` type doesn't declare (e.g. `file`, `id`, `ids`, `cycle`), that field is typed `Dict[str, Any]` instead of `CommandError`, so the contract never claims more structure than the code actually guarantees. + +### Consequences + +* Good: the output contract for all 16 commands is now expressed in code and checked by `mypy --strict` in CI, not just implied by convention. +* Good: adding this caught 3 real, pre-existing type errors in `atomic_io.py`/`telemetry.py` (a missing generator return type, an unnarrowed `Optional` access, and an unparameterized generic). +* Bad: command *argument* types (`argparse.Namespace`) remain untyped, so `mypy --strict` does not yet cover a command's full implementation, only its result shape. + +### Confirmation + +`tests/unit/test_contracts.py` asserts that each command's actual JSON output is a valid subset of its declared `TypedDict` keys; `.github/workflows/test.yml`'s `type-check` job runs `mypy --strict` on every push and pull request. + +## Pros and Cons of the Options + +### `jsonschema` runtime validation + +* Good, because JSON Schema is a widely understood, tool-agnostic format. +* Bad, because it adds a runtime dependency to a project that deliberately has none, and validates on every call instead of at development time. + +### `dataclass` + `asdict()` + +* Good, because it gives real runtime objects, not just static types. +* Bad, because it requires rewriting all 16 commands and their tests to build and return dataclass instances instead of dicts -- a much larger change than the contract gap actually calls for. + +### `TypedDict` + scoped `mypy --strict` (chosen) + +* Good, because it adds a type-level contract with no runtime cost and no new dependency. +* Bad, because `TypedDict` provides no runtime enforcement -- a command could still, in principle, return a dict that mypy didn't check if the command module itself stays outside strict mode. + +## Revisit Triggers + +* `argparse.Namespace` gets a `Protocol`-based typed wrapper, at which point `mypy --strict` coverage should extend into the 16 command modules themselves. +* A consumer reports a real production bug caused by a result-shape mismatch that `TypedDict` alone did not prevent, which would argue for adding runtime validation after all. diff --git a/docs/decisions/0015-structured-json-stderr-logging-with-correlation-ids-for-uncaught-errors.md b/docs/decisions/0015-structured-json-stderr-logging-with-correlation-ids-for-uncaught-errors.md new file mode 100644 index 0000000..5a216be --- /dev/null +++ b/docs/decisions/0015-structured-json-stderr-logging-with-correlation-ids-for-uncaught-errors.md @@ -0,0 +1,77 @@ +--- +id: ADR-0015 +title: Structured JSON stderr logging with correlation IDs for uncaught errors +status: accepted +date: 2026-09-01 +locale: en +decision_makers: + - YangSeungHyun +related: + - ADR-0009 +affected_paths: + - skills/adr-toolkit/scripts/core/telemetry.py + - skills/adr-toolkit/scripts/adr.py + - tests/unit/test_telemetry.py +tags: + - observability + - core + - v0.3.0 +retrospective: false +--- + +# Structured JSON stderr logging with correlation IDs for uncaught errors + +## Context and Problem Statement + +When an uncaught exception reached `adr.py`'s top-level handler, it was converted into a JSON error response on stdout, but the exception's context and stack trace were discarded -- nothing was recorded anywhere else. A CI pipeline or agent harness that hit a failure had no way to look up what actually happened, and no way to tie a given stdout failure back to any diagnostic detail. + +## Decision Drivers + +* stdout must remain pure, contract-stable JSON (established by ADR-0009); diagnostic output cannot be mixed into it. +* Multiple `adr.py` invocations can run concurrently in CI; an unstructured stderr line has no way to be matched back to the specific stdout response it belongs to. +* The test suite uses pytest's `capsys` to capture stderr; a logging setup that accumulates handlers across calls breaks that capture between tests. +* No new runtime dependency (e.g. a hosted logging service) may be introduced. + +## Considered Options + +* Send errors to an external logging service (e.g. Sentry) +* Print unstructured tracebacks to stderr +* Standard-library `logging`, emitting JSON Lines to stderr, with a correlation ID shared between the stderr log line and the stdout JSON error response + +## Decision Outcome + +Chosen option: **structured JSON Lines on stderr with a shared correlation ID**, because it needs no new dependency or network call, and the correlation ID lets a CI log or agent harness join a specific stdout failure to its stderr diagnostic detail. + +`core/telemetry.get_logger(operation)` returns a `LoggerAdapter` that emits JSON Lines to stderr with `level`, `operation`, `correlation_id`, `message`, and (on exceptions) `exception_type`. `adr.py`'s global exception handler logs through this adapter and includes the same `correlation_id` in the stdout JSON error response. The default log level is `WARNING` (silent on success); `ADR_TOOLKIT_LOG_LEVEL` overrides it. Critically, `get_logger()` clears and rebuilds its logger's handlers on every call, rather than accumulating them -- this is what keeps pytest's per-test `capsys` capture correct and prevents unbounded handler growth in a long-running process. + +### Consequences + +* Good: a CI failure's stdout JSON response and its stderr diagnostic log line can now be matched by `correlation_id`. +* Good: stdout's JSON-only contract is unchanged except for the additive `correlation_id` field. +* Bad: logs go only to local stderr -- there is no built-in shipping to a centralized log store; that remains the consuming CI/harness's responsibility. + +### Confirmation + +`tests/unit/test_telemetry.py` verifies the JSON Lines shape and that repeated `get_logger()` calls don't accumulate handlers; `tests/unit/test_adr_cli.py` verifies the stdout error response's `correlation_id` matches what was logged. + +## Pros and Cons of the Options + +### External logging service + +* Good, because it would give centralized, queryable log storage out of the box. +* Bad, because it requires network access and credential management, which conflicts with this CLI's install-and-run-immediately model and its zero-dependency principle. + +### Unstructured stderr tracebacks + +* Good, because it is the simplest possible change. +* Bad, because concurrent CI runs interleave their stderr output with no way to tell which traceback belongs to which invocation or which stdout response. + +### Structured JSON Lines + correlation ID (chosen) + +* Good, because it solves the matching problem with only the standard library. +* Bad, because it is still a local-only log -- aggregating logs across many CI runs still requires the harness to collect and store stderr itself. + +## Revisit Triggers + +* A harness or CI setup needs logs shipped somewhere other than local stderr, which would require revisiting the zero-dependency stance for this specific concern. +* `ADR_TOOLKIT_LOG_LEVEL` proves insufficient and per-operation log-level configuration is needed. diff --git a/docs/decisions/0016-sign-release-artifacts-with-github-artifact-attestation-instead-of-a-managed-signing-key.md b/docs/decisions/0016-sign-release-artifacts-with-github-artifact-attestation-instead-of-a-managed-signing-key.md new file mode 100644 index 0000000..327f87b --- /dev/null +++ b/docs/decisions/0016-sign-release-artifacts-with-github-artifact-attestation-instead-of-a-managed-signing-key.md @@ -0,0 +1,83 @@ +--- +id: ADR-0016 +title: Sign release artifacts with GitHub Artifact Attestation instead of a managed signing key +status: accepted +date: 2026-09-01 +locale: en +decision_makers: + - YangSeungHyun +related: + - ADR-0005 + - ADR-0011 +affected_paths: + - .github/workflows/release.yml + - SECURITY.md +tags: + - release + - security + - supply-chain + - v0.3.0 +retrospective: false +--- + +# Sign release artifacts with GitHub Artifact Attestation instead of a managed signing key + +## Context and Problem Statement + +`.github/workflows/release.yml` ran tests, checked manifest version sync, verified the pushed tag matched `VERSION`, and published a GitHub Release -- but the release carried no checksummed or signed artifact of any kind. Unlike a typical npm/PyPI project, this toolkit has no separately-built distributable: every install path (Claude Code's `marketplace.json` with `source: "./"`, Codex/Gemini CLI plugin installs, a plain `git clone`) consumes the git repository or the `skills/adr-toolkit/` folder directly. That meant the usual "checksum and sign the build artifact" pattern had no artifact to attach to in the first place. + +## Decision Drivers + +* This project's own release process (documented in `AGENTS.md`) has a human create and push the version tag locally -- CI only ever sees an already-pushed tag, so it cannot retroactively sign the tag itself. +* No private signing key should need to be generated, stored, or rotated by a single-maintainer project. +* Whatever gets signed should matter to at least one real consumption path, not be signed purely for form's sake. + +## Considered Options + +* Sign the git tag itself with GPG +* Package a tarball and only checksum it (no provenance signature) +* Package a tarball, checksum it, and generate a GitHub Artifact Attestation (Sigstore-backed, keyless) +* Manage a project GPG key as a repository secret and sign a packaged artifact with it + +## Decision Outcome + +Chosen option: **package plus checksum plus GitHub Artifact Attestation**, because it is the only option that provides real provenance (not just transit-integrity) for the one consumption path that doesn't already trust Git/GitHub's own commit history -- someone downloading the release archive directly from the GitHub Releases page -- without requiring any private key management. + +`release.yml` now packages `skills/adr-toolkit/` into `adr-toolkit-skill-v${VERSION}.tar.gz`, computes its SHA-256 checksum, and runs `actions/attest-build-provenance@v2` against it, using the workflow's OIDC token to produce a Sigstore-backed, keyless attestation. Both the archive and its checksum are attached to the GitHub Release. `SECURITY.md` documents `sha256sum -c` and `gh attestation verify` as the two verification steps, and states explicitly that `git clone`/adapter-install paths verify through Git/GitHub's own commit and tag history instead, since they never touch this archive. + +### Consequences + +* Good: anyone who downloads the release archive directly can cryptographically verify it was built by this repository's own CI from the exact tagged commit, with no key for the maintainer to generate or protect. +* Good: adding this required no change to the existing tag-then-push release trigger. +* Bad: the attestation only covers the one archive-download path; every adapter-based install path (the majority of actual installs today) is unaffected by this change and continues to rely on Git/GitHub history alone. + +### Confirmation + +`release.yml`'s YAML was validated for syntax; the packaging and checksum steps' shell logic were reviewed locally. The OIDC-based attestation issuance and `gh attestation verify` round trip can only be fully confirmed against a real `v*` tag push, since GitHub's attestation API is not available to a local dry run. + +## Pros and Cons of the Options + +### GPG-sign the git tag + +* Good, because a signed tag is a widely recognized supply-chain practice. +* Bad, because this project's tags are created locally by a human before the push that triggers CI -- CI never has an opportunity to sign the tag itself, only whatever it produces after the fact. + +### Checksum only, no signature + +* Good, because it is the simplest possible improvement and needs no new CI permissions. +* Bad, because a checksum only proves the file wasn't corrupted in transit; it says nothing about who produced it, so it cannot detect a look-alike release from a compromised account. + +### Package + checksum + GitHub Artifact Attestation (chosen) + +* Good, because it adds real provenance with no private key to manage, using GitHub's own OIDC/Sigstore integration. +* Bad, because it only protects the archive-download path, not the more common adapter-install paths. + +### Repository-secret GPG key + +* Good, because GPG signatures are recognized by tools outside GitHub's own ecosystem. +* Bad, because a single-maintainer project would then be responsible for key generation, secure storage, and rotation -- operational burden with no corresponding increase in trust over the keyless option. + +## Revisit Triggers + +* This project starts publishing to a package registry (npm, PyPI) with its own signing conventions, which would need its own decision. +* A consumer outside the GitHub ecosystem needs to verify a release without `gh` or without trusting Sigstore's transparency log. diff --git a/docs/decisions/README.md b/docs/decisions/README.md index 864d41a..02c424d 100644 --- a/docs/decisions/README.md +++ b/docs/decisions/README.md @@ -13,6 +13,11 @@ - [ADR-0009 — --json is a documented no-op; CLI output is always JSON](0009-json-flag-always-json-contract.md) - [ADR-0010 — Codex skill-creator's quick_validate.py incompatibility is not this project's problem](0010-codex-quick-validate-not-applicable.md) - [ADR-0011 — Expose ADR relationships as Mermaid and SVG navigation artifacts](0011-adr-relationship-graph-public-readiness.md) +- [ADR-0012 — Atomic writes and cross-platform directory locking for CREATE, EXCEPTION, and SUPERSEDE](0012-atomic-writes-and-cross-platform-directory-locking-for-create-exception-and-supersede.md) +- [ADR-0013 — Two-stage ReDoS defense for CHECK's author-supplied constraint regexes](0013-two-stage-redos-defense-for-check-s-author-supplied-constraint-regexes.md) +- [ADR-0014 — Typed result contracts and a scoped mypy --strict CI gate](0014-typed-result-contracts-and-a-scoped-mypy-strict-ci-gate.md) +- [ADR-0015 — Structured JSON stderr logging with correlation IDs for uncaught errors](0015-structured-json-stderr-logging-with-correlation-ids-for-uncaught-errors.md) +- [ADR-0016 — Sign release artifacts with GitHub Artifact Attestation instead of a managed signing key](0016-sign-release-artifacts-with-github-artifact-attestation-instead-of-a-managed-signing-key.md) ### Superseded - [ADR-0003 — Localize only index.py's generated strings, not agent-composed text](0003-localize-only-index-py-s-generated-strings-not-agent-composed-text.md) @@ -31,6 +36,7 @@ - [ADR-0002 — Limit CHECK's conflict detection to structural evidence only](0002-limit-check-s-conflict-detection-to-structural-evidence-only.md) - [ADR-0007 — Promote CHECK's kind-to-confidence mapping to a stable output field](0007-check-confidence-field.md) - [ADR-0008 — Deterministic CHECK policy exceptions: schema-validated, annotate-only, never suppress](0008-check-exceptions-annotate-only.md) +- [ADR-0013 — Two-stage ReDoS defense for CHECK's author-supplied constraint regexes](0013-two-stage-redos-defense-for-check-s-author-supplied-constraint-regexes.md) ### cli - [ADR-0009 — --json is a documented no-op; CLI output is always JSON](0009-json-flag-always-json-contract.md) @@ -38,12 +44,24 @@ ### codex - [ADR-0010 — Codex skill-creator's quick_validate.py incompatibility is not this project's problem](0010-codex-quick-validate-not-applicable.md) +### concurrency +- [ADR-0012 — Atomic writes and cross-platform directory locking for CREATE, EXCEPTION, and SUPERSEDE](0012-atomic-writes-and-cross-platform-directory-locking-for-create-exception-and-supersede.md) + ### confidence - [ADR-0007 — Promote CHECK's kind-to-confidence mapping to a stable output field](0007-check-confidence-field.md) ### configuration - [ADR-0006 — Localize deterministic ADR generation through repository configuration](0006-localized-adr-generation.md) +### contract +- [ADR-0014 — Typed result contracts and a scoped mypy --strict CI gate](0014-typed-result-contracts-and-a-scoped-mypy-strict-ci-gate.md) + +### core +- [ADR-0012 — Atomic writes and cross-platform directory locking for CREATE, EXCEPTION, and SUPERSEDE](0012-atomic-writes-and-cross-platform-directory-locking-for-create-exception-and-supersede.md) +- [ADR-0013 — Two-stage ReDoS defense for CHECK's author-supplied constraint regexes](0013-two-stage-redos-defense-for-check-s-author-supplied-constraint-regexes.md) +- [ADR-0014 — Typed result contracts and a scoped mypy --strict CI gate](0014-typed-result-contracts-and-a-scoped-mypy-strict-ci-gate.md) +- [ADR-0015 — Structured JSON stderr logging with correlation IDs for uncaught errors](0015-structured-json-stderr-logging-with-correlation-ids-for-uncaught-errors.md) + ### cross-harness - [ADR-0010 — Codex skill-creator's quick_validate.py incompatibility is not this project's problem](0010-codex-quick-validate-not-applicable.md) @@ -77,6 +95,9 @@ ### navigation - [ADR-0011 — Expose ADR relationships as Mermaid and SVG navigation artifacts](0011-adr-relationship-graph-public-readiness.md) +### observability +- [ADR-0015 — Structured JSON stderr logging with correlation IDs for uncaught errors](0015-structured-json-stderr-logging-with-correlation-ids-for-uncaught-errors.md) + ### output-contract - [ADR-0009 — --json is a documented no-op; CLI output is always JSON](0009-json-flag-always-json-contract.md) @@ -86,6 +107,20 @@ ### release - [ADR-0005 — Adopt Git Flow with direct-tag release automation](0005-adopt-git-flow-with-direct-tag-release-automation.md) +- [ADR-0016 — Sign release artifacts with GitHub Artifact Attestation instead of a managed signing key](0016-sign-release-artifacts-with-github-artifact-attestation-instead-of-a-managed-signing-key.md) + +### reliability +- [ADR-0012 — Atomic writes and cross-platform directory locking for CREATE, EXCEPTION, and SUPERSEDE](0012-atomic-writes-and-cross-platform-directory-locking-for-create-exception-and-supersede.md) + +### security +- [ADR-0013 — Two-stage ReDoS defense for CHECK's author-supplied constraint regexes](0013-two-stage-redos-defense-for-check-s-author-supplied-constraint-regexes.md) +- [ADR-0016 — Sign release artifacts with GitHub Artifact Attestation instead of a managed signing key](0016-sign-release-artifacts-with-github-artifact-attestation-instead-of-a-managed-signing-key.md) + +### supply-chain +- [ADR-0016 — Sign release artifacts with GitHub Artifact Attestation instead of a managed signing key](0016-sign-release-artifacts-with-github-artifact-attestation-instead-of-a-managed-signing-key.md) + +### typing +- [ADR-0014 — Typed result contracts and a scoped mypy --strict CI gate](0014-typed-result-contracts-and-a-scoped-mypy-strict-ci-gate.md) ### v0.2.0 - [ADR-0006 — Localize deterministic ADR generation through repository configuration](0006-localized-adr-generation.md) @@ -94,6 +129,13 @@ - [ADR-0009 — --json is a documented no-op; CLI output is always JSON](0009-json-flag-always-json-contract.md) - [ADR-0011 — Expose ADR relationships as Mermaid and SVG navigation artifacts](0011-adr-relationship-graph-public-readiness.md) +### v0.3.0 +- [ADR-0012 — Atomic writes and cross-platform directory locking for CREATE, EXCEPTION, and SUPERSEDE](0012-atomic-writes-and-cross-platform-directory-locking-for-create-exception-and-supersede.md) +- [ADR-0013 — Two-stage ReDoS defense for CHECK's author-supplied constraint regexes](0013-two-stage-redos-defense-for-check-s-author-supplied-constraint-regexes.md) +- [ADR-0014 — Typed result contracts and a scoped mypy --strict CI gate](0014-typed-result-contracts-and-a-scoped-mypy-strict-ci-gate.md) +- [ADR-0015 — Structured JSON stderr logging with correlation IDs for uncaught errors](0015-structured-json-stderr-logging-with-correlation-ids-for-uncaught-errors.md) +- [ADR-0016 — Sign release artifacts with GitHub Artifact Attestation instead of a managed signing key](0016-sign-release-artifacts-with-github-artifact-attestation-instead-of-a-managed-signing-key.md) + ## By affected path ### `.adr-toolkit.json` @@ -112,6 +154,12 @@ ### `.github/workflows/` - [ADR-0005 — Adopt Git Flow with direct-tag release automation](0005-adopt-git-flow-with-direct-tag-release-automation.md) +### `.github/workflows/release.yml` +- [ADR-0016 — Sign release artifacts with GitHub Artifact Attestation instead of a managed signing key](0016-sign-release-artifacts-with-github-artifact-attestation-instead-of-a-managed-signing-key.md) + +### `.github/workflows/test.yml` +- [ADR-0014 — Typed result contracts and a scoped mypy --strict CI gate](0014-typed-result-contracts-and-a-scoped-mypy-strict-ci-gate.md) + ### `.gitignore` - [ADR-0004 — Adapter packaging: manifest-only directories, install-time symlinks, verified formats](0004-adapter-packaging-manifest-only-directories-install-time-symlinks-verified-formats.md) @@ -129,6 +177,7 @@ ### `SECURITY.md` - [ADR-0011 — Expose ADR relationships as Mermaid and SVG navigation artifacts](0011-adr-relationship-graph-public-readiness.md) +- [ADR-0016 — Sign release artifacts with GitHub Artifact Attestation instead of a managed signing key](0016-sign-release-artifacts-with-github-artifact-attestation-instead-of-a-managed-signing-key.md) ### `adapters/` - [ADR-0004 — Adapter packaging: manifest-only directories, install-time symlinks, verified formats](0004-adapter-packaging-manifest-only-directories-install-time-symlinks-verified-formats.md) @@ -182,6 +231,7 @@ - [ADR-0008 — Deterministic CHECK policy exceptions: schema-validated, annotate-only, never suppress](0008-check-exceptions-annotate-only.md) - [ADR-0009 — --json is a documented no-op; CLI output is always JSON](0009-json-flag-always-json-contract.md) - [ADR-0011 — Expose ADR relationships as Mermaid and SVG navigation artifacts](0011-adr-relationship-graph-public-readiness.md) +- [ADR-0015 — Structured JSON stderr logging with correlation IDs for uncaught errors](0015-structured-json-stderr-logging-with-correlation-ids-for-uncaught-errors.md) ### `skills/adr-toolkit/scripts/commands/check.py` - [ADR-0002 — Limit CHECK's conflict detection to structural evidence only](0002-limit-check-s-conflict-detection-to-structural-evidence-only.md) @@ -191,12 +241,14 @@ ### `skills/adr-toolkit/scripts/commands/create.py` - [ADR-0003 — Localize only index.py's generated strings, not agent-composed text](0003-localize-only-index-py-s-generated-strings-not-agent-composed-text.md) - [ADR-0006 — Localize deterministic ADR generation through repository configuration](0006-localized-adr-generation.md) +- [ADR-0012 — Atomic writes and cross-platform directory locking for CREATE, EXCEPTION, and SUPERSEDE](0012-atomic-writes-and-cross-platform-directory-locking-for-create-exception-and-supersede.md) ### `skills/adr-toolkit/scripts/commands/diff.py` - [ADR-0002 — Limit CHECK's conflict detection to structural evidence only](0002-limit-check-s-conflict-detection-to-structural-evidence-only.md) ### `skills/adr-toolkit/scripts/commands/exception.py` - [ADR-0008 — Deterministic CHECK policy exceptions: schema-validated, annotate-only, never suppress](0008-check-exceptions-annotate-only.md) +- [ADR-0012 — Atomic writes and cross-platform directory locking for CREATE, EXCEPTION, and SUPERSEDE](0012-atomic-writes-and-cross-platform-directory-locking-for-create-exception-and-supersede.md) ### `skills/adr-toolkit/scripts/commands/graph.py` - [ADR-0011 — Expose ADR relationships as Mermaid and SVG navigation artifacts](0011-adr-relationship-graph-public-readiness.md) @@ -210,15 +262,25 @@ - [ADR-0003 — Localize only index.py's generated strings, not agent-composed text](0003-localize-only-index-py-s-generated-strings-not-agent-composed-text.md) - [ADR-0006 — Localize deterministic ADR generation through repository configuration](0006-localized-adr-generation.md) +### `skills/adr-toolkit/scripts/commands/supersede.py` +- [ADR-0012 — Atomic writes and cross-platform directory locking for CREATE, EXCEPTION, and SUPERSEDE](0012-atomic-writes-and-cross-platform-directory-locking-for-create-exception-and-supersede.md) + ### `skills/adr-toolkit/scripts/commands/validate.py` - [ADR-0006 — Localize deterministic ADR generation through repository configuration](0006-localized-adr-generation.md) +### `skills/adr-toolkit/scripts/core/atomic_io.py` +- [ADR-0012 — Atomic writes and cross-platform directory locking for CREATE, EXCEPTION, and SUPERSEDE](0012-atomic-writes-and-cross-platform-directory-locking-for-create-exception-and-supersede.md) + ### `skills/adr-toolkit/scripts/core/config.py` - [ADR-0003 — Localize only index.py's generated strings, not agent-composed text](0003-localize-only-index-py-s-generated-strings-not-agent-composed-text.md) - [ADR-0006 — Localize deterministic ADR generation through repository configuration](0006-localized-adr-generation.md) ### `skills/adr-toolkit/scripts/core/constraints.py` - [ADR-0002 — Limit CHECK's conflict detection to structural evidence only](0002-limit-check-s-conflict-detection-to-structural-evidence-only.md) +- [ADR-0013 — Two-stage ReDoS defense for CHECK's author-supplied constraint regexes](0013-two-stage-redos-defense-for-check-s-author-supplied-constraint-regexes.md) + +### `skills/adr-toolkit/scripts/core/contracts.py` +- [ADR-0014 — Typed result contracts and a scoped mypy --strict CI gate](0014-typed-result-contracts-and-a-scoped-mypy-strict-ci-gate.md) ### `skills/adr-toolkit/scripts/core/exceptions.py` - [ADR-0008 — Deterministic CHECK policy exceptions: schema-validated, annotate-only, never suppress](0008-check-exceptions-annotate-only.md) @@ -240,12 +302,16 @@ ### `skills/adr-toolkit/scripts/core/schema.py` - [ADR-0006 — Localize deterministic ADR generation through repository configuration](0006-localized-adr-generation.md) +### `skills/adr-toolkit/scripts/core/telemetry.py` +- [ADR-0015 — Structured JSON stderr logging with correlation IDs for uncaught errors](0015-structured-json-stderr-logging-with-correlation-ids-for-uncaught-errors.md) + ### `skills/adr-toolkit/scripts/i18n/` - [ADR-0003 — Localize only index.py's generated strings, not agent-composed text](0003-localize-only-index-py-s-generated-strings-not-agent-composed-text.md) - [ADR-0006 — Localize deterministic ADR generation through repository configuration](0006-localized-adr-generation.md) ### `skills/adr-toolkit/scripts/rules/conflict.py` - [ADR-0002 — Limit CHECK's conflict detection to structural evidence only](0002-limit-check-s-conflict-detection-to-structural-evidence-only.md) +- [ADR-0013 — Two-stage ReDoS defense for CHECK's author-supplied constraint regexes](0013-two-stage-redos-defense-for-check-s-author-supplied-constraint-regexes.md) ### `tests/integration/test_cli.py` - [ADR-0009 — --json is a documented no-op; CLI output is always JSON](0009-json-flag-always-json-contract.md) @@ -253,9 +319,21 @@ ### `tests/unit/test_adr_cli.py` - [ADR-0011 — Expose ADR relationships as Mermaid and SVG navigation artifacts](0011-adr-relationship-graph-public-readiness.md) +### `tests/unit/test_atomic_io.py` +- [ADR-0012 — Atomic writes and cross-platform directory locking for CREATE, EXCEPTION, and SUPERSEDE](0012-atomic-writes-and-cross-platform-directory-locking-for-create-exception-and-supersede.md) + ### `tests/unit/test_check.py` - [ADR-0007 — Promote CHECK's kind-to-confidence mapping to a stable output field](0007-check-confidence-field.md) +### `tests/unit/test_conflict.py` +- [ADR-0013 — Two-stage ReDoS defense for CHECK's author-supplied constraint regexes](0013-two-stage-redos-defense-for-check-s-author-supplied-constraint-regexes.md) + +### `tests/unit/test_constraints.py` +- [ADR-0013 — Two-stage ReDoS defense for CHECK's author-supplied constraint regexes](0013-two-stage-redos-defense-for-check-s-author-supplied-constraint-regexes.md) + +### `tests/unit/test_contracts.py` +- [ADR-0014 — Typed result contracts and a scoped mypy --strict CI gate](0014-typed-result-contracts-and-a-scoped-mypy-strict-ci-gate.md) + ### `tests/unit/test_graph_command.py` - [ADR-0011 — Expose ADR relationships as Mermaid and SVG navigation artifacts](0011-adr-relationship-graph-public-readiness.md) @@ -265,8 +343,16 @@ ### `tests/unit/test_relationships.py` - [ADR-0011 — Expose ADR relationships as Mermaid and SVG navigation artifacts](0011-adr-relationship-graph-public-readiness.md) +### `tests/unit/test_telemetry.py` +- [ADR-0015 — Structured JSON stderr logging with correlation IDs for uncaught errors](0015-structured-json-stderr-logging-with-correlation-ids-for-uncaught-errors.md) + ## Chronological (newest first) +- 2026-09-01 — [ADR-0012 — Atomic writes and cross-platform directory locking for CREATE, EXCEPTION, and SUPERSEDE](0012-atomic-writes-and-cross-platform-directory-locking-for-create-exception-and-supersede.md) +- 2026-09-01 — [ADR-0013 — Two-stage ReDoS defense for CHECK's author-supplied constraint regexes](0013-two-stage-redos-defense-for-check-s-author-supplied-constraint-regexes.md) +- 2026-09-01 — [ADR-0014 — Typed result contracts and a scoped mypy --strict CI gate](0014-typed-result-contracts-and-a-scoped-mypy-strict-ci-gate.md) +- 2026-09-01 — [ADR-0015 — Structured JSON stderr logging with correlation IDs for uncaught errors](0015-structured-json-stderr-logging-with-correlation-ids-for-uncaught-errors.md) +- 2026-09-01 — [ADR-0016 — Sign release artifacts with GitHub Artifact Attestation instead of a managed signing key](0016-sign-release-artifacts-with-github-artifact-attestation-instead-of-a-managed-signing-key.md) - 2026-08-31 — [ADR-0011 — Expose ADR relationships as Mermaid and SVG navigation artifacts](0011-adr-relationship-graph-public-readiness.md) - 2026-08-30 — [ADR-0001 — Record architecture decisions](0001-record-architecture-decisions.md) - 2026-08-30 — [ADR-0002 — Limit CHECK's conflict detection to structural evidence only](0002-limit-check-s-conflict-detection-to-structural-evidence-only.md) @@ -292,6 +378,14 @@ - ADR-0011 "Expose ADR relationships as Mermaid and SVG navigation artifacts" related to: ADR-0006 "Localize deterministic ADR generation through repository configuration" - ADR-0011 "Expose ADR relationships as Mermaid and SVG navigation artifacts" related to: ADR-0008 "Deterministic CHECK policy exceptions: schema-validated, annotate-only, never suppress" - ADR-0011 "Expose ADR relationships as Mermaid and SVG navigation artifacts" related to: ADR-0009 "--json is a documented no-op; CLI output is always JSON" +- ADR-0012 "Atomic writes and cross-platform directory locking for CREATE, EXCEPTION, and SUPERSEDE" related to: ADR-0006 "Localize deterministic ADR generation through repository configuration" +- ADR-0013 "Two-stage ReDoS defense for CHECK's author-supplied constraint regexes" related to: ADR-0002 "Limit CHECK's conflict detection to structural evidence only" +- ADR-0013 "Two-stage ReDoS defense for CHECK's author-supplied constraint regexes" related to: ADR-0007 "Promote CHECK's kind-to-confidence mapping to a stable output field" +- ADR-0013 "Two-stage ReDoS defense for CHECK's author-supplied constraint regexes" related to: ADR-0008 "Deterministic CHECK policy exceptions: schema-validated, annotate-only, never suppress" +- ADR-0014 "Typed result contracts and a scoped mypy --strict CI gate" related to: ADR-0009 "--json is a documented no-op; CLI output is always JSON" +- ADR-0015 "Structured JSON stderr logging with correlation IDs for uncaught errors" related to: ADR-0009 "--json is a documented no-op; CLI output is always JSON" +- ADR-0016 "Sign release artifacts with GitHub Artifact Attestation instead of a managed signing key" related to: ADR-0005 "Adopt Git Flow with direct-tag release automation" +- ADR-0016 "Sign release artifacts with GitHub Artifact Attestation instead of a managed signing key" related to: ADR-0011 "Expose ADR relationships as Mermaid and SVG navigation artifacts" ```mermaid flowchart LR @@ -306,10 +400,23 @@ flowchart LR ADR_0009["ADR-0009
--json is a documented no-op; CLI output is always JSON"] ADR_0010["ADR-0010
Codex skill-creator's quick_validate.py incompatibility is not this project's problem"] ADR_0011["ADR-0011
Expose ADR relationships as Mermaid and SVG navigation artifacts"] + ADR_0012["ADR-0012
Atomic writes and cross-platform directory locking for CREATE, EXCEPTION, and SUPERSEDE"] + ADR_0013["ADR-0013
Two-stage ReDoS defense for CHECK's author-supplied constraint regexes"] + ADR_0014["ADR-0014
Typed result contracts and a scoped mypy --strict CI gate"] + ADR_0015["ADR-0015
Structured JSON stderr logging with correlation IDs for uncaught errors"] + ADR_0016["ADR-0016
Sign release artifacts with GitHub Artifact Attestation instead of a managed signing key"] ADR_0006 -.->|related| ADR_0003 ADR_0006 -->|supersedes| ADR_0003 ADR_0011 -.->|related| ADR_0001 ADR_0011 -.->|related| ADR_0006 ADR_0011 -.->|related| ADR_0008 ADR_0011 -.->|related| ADR_0009 + ADR_0012 -.->|related| ADR_0006 + ADR_0013 -.->|related| ADR_0002 + ADR_0013 -.->|related| ADR_0007 + ADR_0013 -.->|related| ADR_0008 + ADR_0014 -.->|related| ADR_0009 + ADR_0015 -.->|related| ADR_0009 + ADR_0016 -.->|related| ADR_0005 + ADR_0016 -.->|related| ADR_0011 ``` diff --git a/docs/decisions/relationships.mmd b/docs/decisions/relationships.mmd index f36a1e0..91fe037 100644 --- a/docs/decisions/relationships.mmd +++ b/docs/decisions/relationships.mmd @@ -10,9 +10,22 @@ flowchart LR ADR_0009["ADR-0009
--json is a documented no-op; CLI output is always JSON"] ADR_0010["ADR-0010
Codex skill-creator's quick_validate.py incompatibility is not this project's problem"] ADR_0011["ADR-0011
Expose ADR relationships as Mermaid and SVG navigation artifacts"] + ADR_0012["ADR-0012
Atomic writes and cross-platform directory locking for CREATE, EXCEPTION, and SUPERSEDE"] + ADR_0013["ADR-0013
Two-stage ReDoS defense for CHECK's author-supplied constraint regexes"] + ADR_0014["ADR-0014
Typed result contracts and a scoped mypy --strict CI gate"] + ADR_0015["ADR-0015
Structured JSON stderr logging with correlation IDs for uncaught errors"] + ADR_0016["ADR-0016
Sign release artifacts with GitHub Artifact Attestation instead of a managed signing key"] ADR_0006 -.->|related| ADR_0003 ADR_0006 -->|supersedes| ADR_0003 ADR_0011 -.->|related| ADR_0001 ADR_0011 -.->|related| ADR_0006 ADR_0011 -.->|related| ADR_0008 ADR_0011 -.->|related| ADR_0009 + ADR_0012 -.->|related| ADR_0006 + ADR_0013 -.->|related| ADR_0002 + ADR_0013 -.->|related| ADR_0007 + ADR_0013 -.->|related| ADR_0008 + ADR_0014 -.->|related| ADR_0009 + ADR_0015 -.->|related| ADR_0009 + ADR_0016 -.->|related| ADR_0005 + ADR_0016 -.->|related| ADR_0011 diff --git a/docs/decisions/relationships.svg b/docs/decisions/relationships.svg index 7afc55f..b267074 100644 --- a/docs/decisions/relationships.svg +++ b/docs/decisions/relationships.svg @@ -1,4 +1,4 @@ - + ADR relationship graph Vector export of ADR supersession and related-decision links. @@ -37,16 +37,47 @@ ADR-0011 Expose ADR relationships as Merma... + +ADR-0012 +Atomic writes and cross-platform ... + +ADR-0013 +Two-stage ReDoS defense for CHECK... + +ADR-0014 +Typed result contracts and a scop... + +ADR-0015 +Structured JSON stderr logging wi... + +ADR-0016 +Sign release artifacts with GitHu... -ADR-0006 related ADR-0003 +ADR-0006 related ADR-0003 -ADR-0006 supersedes ADR-0003 +ADR-0006 supersedes ADR-0003 -ADR-0011 related ADR-0001 +ADR-0011 related ADR-0001 -ADR-0011 related ADR-0006 +ADR-0011 related ADR-0006 -ADR-0011 related ADR-0008 +ADR-0011 related ADR-0008 -ADR-0011 related ADR-0009 +ADR-0011 related ADR-0009 + +ADR-0012 related ADR-0006 + +ADR-0013 related ADR-0002 + +ADR-0013 related ADR-0007 + +ADR-0013 related ADR-0008 + +ADR-0014 related ADR-0009 + +ADR-0015 related ADR-0009 + +ADR-0016 related ADR-0005 + +ADR-0016 related ADR-0011 diff --git a/docs/worklogs/2026-09-01-atomic-writes-and-locking.md b/docs/worklogs/2026-09-01-atomic-writes-and-locking.md deleted file mode 100644 index 9f53678..0000000 --- a/docs/worklogs/2026-09-01-atomic-writes-and-locking.md +++ /dev/null @@ -1,106 +0,0 @@ -# ADR/exception/supersede 동시성 레이스 컨디션 제거 - -## 날짜 - -2026-09-01 (원 구현은 `2026-09-01` Critical 하드닝 패스) - -## 문제 상황 - -`docs/adr-toolkit-audit-report.md`가 지적한 Critical 항목: `create`, -`exception`, `supersede` 세 커맨드 모두 "다음 순번 계산 → 파일 존재 확인 -→ `Path.write_text()`로 직접 쓰기" 순서로 동작했다. 이 세 단계 사이에 -원자성이 전혀 없어서, 같은 저장소에 대해 두 프로세스가 동시에 -`adr.py create`를 호출하면 둘 다 같은 다음 번호(예: `ADR-0012`)를 -계산해서 서로의 파일을 덮어쓸 수 있었다. - -## 기존 구조나 방식의 한계 - -- ID 할당에 락이나 원자적 연산이 전혀 없어, "다음 번호 읽기"와 "그 - 번호로 쓰기" 사이에 임의의 다른 프로세스가 끼어들 수 있었다. -- `Path.write_text()`는 원자적이지 않다 — 쓰기 도중 프로세스가 - 죽으면(OOM kill, SIGKILL, 정전) 파일이 반쯤 쓰인 상태로 남을 수 있다. -- 실제로 재현해서 확인함: 수정 전 코드로 20개 concurrent `create` 호출을 - 실행하자 고유 ADR ID가 13개만 생성됐다(`49ede49` 커밋 메시지에 기록). - `exception`도 동일한 방식으로 재현: 20개 중 18개만 고유 ID(`68bbd98`). - -## 관련 코드 맥락 - -- `skills/adr-toolkit/scripts/commands/create.py`의 `run()` — 다음 ID - 계산 후 `_build_frontmatter()`로 프런트매터를 만들고 파일에 쓰는 - 부분이 원래 아무 보호 없이 실행됐다. -- `skills/adr-toolkit/scripts/commands/exception.py`의 `run()` — 동일한 - 패턴이지만, `SCHEMA_ERROR`(잘못된 draft) 시에는 애초에 아무 파일도 - 만들면 안 된다는 기존 테스트 제약이 있었다. -- `skills/adr-toolkit/scripts/commands/supersede.py`의 `run()` — 구 - ADR과 신규 ADR 두 파일을 순서대로 갱신하는데, 첫 파일 쓰기 후 둘째 - 파일 쓰기가 실패하면 롤백을 시도하는 기존 로직이 이미 있었다. - -## 검토한 선택지 - -1. **파일 시스템 락 없이 재시도/충돌 감지만 추가** — 쓰기 후 "내가 쓴 - 파일이 맞는지" 재확인하는 낙관적 동시성 제어. 구현이 복잡해지고 - 재시도 로직 자체에 새로운 엣지 케이스가 생김. 기각. -2. **SQLite 등 외부 상태 저장소로 ID 카운터 이전** — 이 프로젝트의 - "third-party 런타임 의존성 0개" 원칙(zero-dependency 아키텍처 - 가치)과 정면충돌. 기각. -3. **크로스 플랫폼 파일 락(`fcntl`/`msvcrt`) + 임시파일→`os.replace` - 원자적 쓰기** — 표준 라이브러리만으로 두 문제(레이스 컨디션, 쓰기 - 도중 크래시)를 동시에 해결. 채택. - -## 판단 기준 - -- 이 프로젝트는 third-party 런타임 의존성이 전혀 없는 것이 의도된 - 아키텍처 가치(감사 세션 내내 여러 번 확인됨) — 표준 라이브러리만으로 - 해결 가능한지가 최우선 기준. -- Windows/macOS/Linux 3.9~3.12 CI 매트릭스를 그대로 지원해야 하므로, - POSIX 전용 API(`fcntl`)만으로는 부족하고 플랫폼 분기가 필요. -- 기존 dry-run 테스트("dry-run은 아무것도 생성하면 안 된다")를 깨지 - 않아야 함. - -## 최종 결정 - -`core/atomic_io.py`에 `atomic_write_text()`(임시파일 + `os.fsync` + -`os.replace`)와 `adr_directory_lock()`(POSIX `fcntl.flock` / Windows -`msvcrt.locking` 컨텍스트 매니저)을 만들고, `create`/`exception`/ -`supersede`의 실제 쓰기 경로를 이 두 프리미티브로 감쌌다. - -## 해결 방식 - -1. `fc46830` — `core/atomic_io.py` 신설. 아직 어떤 커맨드에도 연결하지 - 않은 순수 프리미티브 단계로 먼저 커밋(리뷰 단위를 작게 유지). -2. `49ede49` — `create.py`: ID 할당 + 존재 확인 + 쓰기 전체를 - `adr_directory_lock()`으로 감싸고 `write_text`를 - `atomic_write_text`로 교체. **dry-run 경로는 락 밖에 완전히 남겨둠** - — dry-run이 `adr_dir`나 락 파일을 생성하면 기존 테스트가 깨지고, - dry-run은 아무것도 영속화하지 않으므로 거기서 레이스가 나도 무해함. -3. `68bbd98` — `exception.py`도 같은 패턴이되 한 가지 추가 보정: 스키마 - 검증은 **디스크를 건드리기 전에** preview ID로 미리 수행. 유효성 - 검증 결과가 실제로 어떤 순번을 받는지와 무관하기 때문에, 이렇게 - 해야 "SCHEMA_ERROR 시 exceptions_dir조차 만들면 안 된다"는 기존 - 테스트와 "dry-run은 아무것도 안 만든다"는 테스트를 모두 만족시킴. - 락은 최종 ID 할당 + 원자적 쓰기 구간만 감쌈. -4. `cec7215` — `supersede.py`: 두 파일 갱신 전체를 락으로 감싸고, 두 - 번의 `write_text` 호출과 롤백 쓰기까지 전부 - `atomic_write_text`로 교체. 기존 테스트 2개 - (`test_supersede_rolls_back_old_file_when_new_file_write_fails`, - `test_supersede_double_write_failure_reports_inconsistent_state_not_silent`)가 - `Path.write_text`를 직접 monkeypatch해서 쓰기 실패를 흉내내고 - 있었는데, 그 시드(seam)가 사라져서 - `supersede.atomic_io.atomic_write_text`로 재타게팅 — 테스트의 원래 - 의도와 검증 내용은 그대로 유지. - -## 결과 - -- 재현 테스트로 수정 전 실패(레이스 발생)를 먼저 확인한 뒤 수정 → - TDD 순서를 지킴. -- OS 레벨 검증(fork + SIGKILL)까지 별도로 수행해, 쓰기 도중 프로세스가 - 강제 종료돼도 ADR 파일이 반쯤 쓰인 상태로 남지 않음을 확인 - (High-priority 패스, `9708bb2`). -- 알려진 남은 한계 (`handoff.md`에 기록): `supersede`의 개별 파일 쓰기는 - 각각 원자적이지만, 두 파일 쓰기 "사이"에 프로세스가 죽으면 두 파일 - 쌍 전체의 일관성(진짜 2-phase commit)까지는 보장하지 않음 — 의도적으로 - 범위 밖으로 둔 것. -- 성공적인 `create`/`exception`/`supersede` 호출마다 `.adr-toolkit.lock` - (0바이트 dotfile)이 `docs/decisions/`, `docs/decisions/exceptions/` - 안에 영구히 남는다 — 크로스 프로세스 뮤텍스로서 의도된 동작이며 - `*.md`/`*.json` glob과 충돌하지 않음. diff --git a/docs/worklogs/2026-09-01-redos-guard.md b/docs/worklogs/2026-09-01-redos-guard.md deleted file mode 100644 index 1ef7bf4..0000000 --- a/docs/worklogs/2026-09-01-redos-guard.md +++ /dev/null @@ -1,104 +0,0 @@ -# CHECK의 사용자 정의 regex에 대한 ReDoS 방어 (런타임 + 정적, 2단계) - -## 날짜 - -2026-09-01 - -## 문제 상황 - -`constraints:` 블록의 `forbidden_import`/`dependency_forbidden` 규칙은 -작성자가 임의의 정규식(`pattern` 필드)을 직접 써서 diff의 추가된 줄에 -매칭시킨다(`rules/conflict.py::_content_pattern`). 이 정규식은 ADR -작성자가 통제하므로, 실수든 악의든 `(a+)+$`류의 catastrophic -backtracking 패턴이 들어가면 `re.search()`가 사실상 무한정 멈추지 -않는다 — CHECK 전체가 그 한 줄에서 행(hang)될 수 있었다. - -## 기존 구조나 방식의 한계 - -- `_content_pattern`이 `re.compile(pattern).search(line)`을 아무 보호 - 장치 없이 호출했다. -- 이 CLI는 CI 파이프라인(`harness-parity`, PR 체크 등)에서 자동 실행되는 - 경우가 많아, 한 번 hang이 나면 사람이 알아채기 전까지 CI 러너가 - 타임아웃될 때까지 계속 잡아먹는다. -- Python 표준 라이브러리에는 정규식 실행 타임아웃 기능이 없다 — - `signal.alarm`/`setitimer`로 직접 인터럽트를 걸어야 하는데, 이는 - **POSIX 전용**(Windows는 `SIGALRM` 자체가 없음)이라 단일 메커니즘으로 - 모든 CI 플랫폼(ubuntu/macos/windows)을 커버할 수 없었다. - -## 관련 코드 맥락 - -- `skills/adr-toolkit/scripts/rules/conflict.py::_content_pattern()` — - diff의 각 추가된 줄에 대해 규칙의 모든 `pattern`을 매칭 시도하는 - 실제 실행 지점. -- `skills/adr-toolkit/scripts/commands/check.py` — 기존에 이미 - `except re.error`로 정규식 컴파일 오류를 잡아 `BAD_CONSTRAINTS` - 경고로 격하시키는 처리 경로가 있었음 — 이 기존 경로를 재사용할 수 - 있는지가 설계의 핵심이었다. -- `skills/adr-toolkit/scripts/core/constraints.py::_parse_rules()` — - `constraints:` YAML 블록을 파싱해서 규칙 리스트를 만드는 곳. 여기서 - `pattern` 값 자체를 파싱 시점에 검사할 수 있다는 게 두 번째 방어선의 - 근거. - -## 검토한 선택지 - -1. **정규식 엔진을 `re2`류 선형 시간 엔진으로 교체** — third-party - 의존성 추가가 필요해 zero-dependency 원칙과 충돌. 기각. -2. **`multiprocessing`으로 정규식 실행을 별도 프로세스에 격리하고 - `terminate()`** — 플랫폼 독립적이지만, 매 diff 라인마다 프로세스를 - 새로 띄우는 오버헤드가 크고, CHECK는 원래 빠른 사전 검증 도구라는 - 설계 의도와 어긋남. 기각. -3. **POSIX `SIGALRM`/`setitimer` 기반 런타임 타임아웃만 적용** — - 구현이 단순하고 기존 `except re.error` 경로에 자연스럽게 편입되지만, - Windows에서는 완전히 무방비 상태로 남는 절반짜리 해법. -4. **런타임 타임아웃(3) + 파싱 시점 정적 휴리스틱(중첩 quantifier - 거부)을 함께 적용** — Windows/POSIX 모두 최소한의 방어선을 갖도록 - 2단계로 방어. 채택. - -## 판단 기준 - -- CI 매트릭스가 ubuntu/macos/windows 3개 플랫폼을 전부 포함하므로, - "POSIX에서만 동작하는 방어"는 감사 관점에서 "Windows 미방어"라는 - 별도의 Open Risk로 남는다 — 완전히 무시할 수 없음. -- 기존 에러 처리 경로(`except re.error`)를 재사용할 수 있으면 새 - 실패 모드를 추가하지 않고 통합할 수 있다 — 최소 침습 우선. -- 정적 검사는 오탐(false positive)이 나면 정상적인 규칙 작성을 막으므로, - 가장 흔하고 확실한 패턴 모양(중첩 quantifier)만 좁게 잡는 휴리스틱이 - 안전하다. - -## 최종 결정 - -옵션 4 — `rules/conflict.py`에 런타임 SIGALRM 타임아웃 가드를 -추가하고(POSIX만 유효, Windows는 가드 없이 그냥 실행), 별도로 -`core/constraints.py`에 파싱 시점 정적 중첩-quantifier 거부 로직을 -추가해 플랫폼 무관하게 최소 방어선을 확보했다. - -## 해결 방식 - -1. `c0ff907` — `rules/conflict.py`에 `RegexTimeout(re.error)` 예외 - 클래스와 `_guarded_search(regex, line)` 추가. `_REGEX_TIMEOUT_SECONDS - = 0.25`로 `signal.alarm`을 걸고, 시간 초과 시 `RegexTimeout`을 - 발생시킴 — `re.error`의 서브클래스이므로 `check.py`의 기존 - `except re.error` 처리가 코드 수정 없이 그대로 이를 `BAD_CONSTRAINTS` - 경고로 격하시킴. `(a+)+$`류 실제 catastrophic backtracking 패턴으로 - 검증: 가드가 없으면 멈추던 것이 1초 이내에 인터럽트됨. -2. `26021a9` (Medium 패스) — `core/constraints.py`에 - `_NESTED_QUANTIFIER_RE = re.compile(r"\([^()]*" + _QUANTIFIER + r"\)" - + _QUANTIFIER)`와 `_reject_if_redos_prone(pattern)`을 추가해 - `_parse_rules()`의 후처리 단계에서 호출. `forbidden_import`/ - `dependency_forbidden`에만 적용 — `required_path`/`forbidden_path`는 - `pattern`을 glob으로 취급(`core/globs.py` 경유)해서 애초에 - catastrophic backtracking이 발생할 수 없는 구조라, 여기에 적용하면 - 오탐이 된다. 실제로 dogfooding 중인 `ADR-0011`의 constraints 블록으로 - 회귀 테스트: 정상 패턴은 여전히 통과, 위험 패턴은 `re.compile()`에 - 도달하기 전에 걸러짐(테스트에서 `re.compile`을 monkeypatch해서 호출 - 자체가 안 됨을 확인). - -## 결과 - -- 두 커밋 모두 실패 재현 → 가드 추가 → 통과 확인 순서로 진행. -- 남은 한계(코드 주석 및 `handoff.md`에 명시): 이 정적 휴리스틱은 가장 - 흔한 "중첩 quantifier" 모양만 잡는 것이지 범용 ReDoS 탐지기가 - 아니다 — `(a|a)*`류 alternation 기반 패턴은 여전히 미탐지 상태로 - 남으며, Windows에서는 이 정적 검사가 유일한 방어선이다(런타임 - SIGALRM 가드가 없으므로). -- 전체 테스트 스위트 회귀 없음 확인 후 커밋. diff --git a/docs/worklogs/2026-09-01-structured-logging.md b/docs/worklogs/2026-09-01-structured-logging.md deleted file mode 100644 index 2c5bea1..0000000 --- a/docs/worklogs/2026-09-01-structured-logging.md +++ /dev/null @@ -1,90 +0,0 @@ -# 상관관계 ID를 포함한 구조화 JSON 에러 로깅 - -## 날짜 - -2026-09-01 - -## 문제 상황 - -`adr.py`에서 예상치 못한 예외가 발생하면 최상위 `except` 블록이 이를 -잡아 JSON 에러 응답으로 stdout에 내보냈지만, 그 과정에서 스택 트레이스나 -예외 컨텍스트는 어디에도 기록되지 않고 사라졌다. CI나 에이전트 하네스가 -"어떤 실행에서 어떤 에러가 났는지"를 나중에 추적할 방법이 없었다. - -## 기존 구조나 방식의 한계 - -- stdout은 이 프로젝트의 확고한 계약(ADR-0009: "always JSON contract")이라 - 사람이 읽는 진단 로그를 stdout에 섞을 수 없다. -- 그렇다고 `print(traceback, file=sys.stderr)` 같은 비구조화 텍스트를 - 찍으면, 여러 프로세스가 동시에 도는 CI 환경에서 어떤 stderr 줄이 어떤 - stdout JSON 응답과 짝인지 알 방법이 없다. -- 표준 `logging` 모듈을 그냥 쓰면 pytest의 capsys 캡처와 충돌하기 쉽고 - (핸들러가 이전 테스트에서 누적됨), 프로덕션에서도 매 호출마다 새 - 핸들러가 쌓이는 문제가 생긴다. - -## 관련 코드 맥락 - -- `skills/adr-toolkit/scripts/adr.py`의 최상위 `main()` 함수 — - 예외를 잡는 유일한 지점. 여기서 로거를 얻어 기록하고, 동일한 - 상관관계 ID를 stdout JSON 에러 응답에도 넣어야 두 출력을 나중에 - 매칭할 수 있다. -- `skills/adr-toolkit/scripts/core/telemetry.py`(신규) — - `_JsonLogFormatter`와 `get_logger(operation, *, correlation_id=None)`를 - 정의. 매 호출마다 `logger.handlers.clear()`로 핸들러를 비우고 다시 - 구성하는 게 핵심 — 이래야 pytest에서 각 테스트가 독립적으로 stderr를 - 캡처할 수 있고, 프로덕션에서도 핸들러가 무한정 누적되지 않는다. - -## 검토한 선택지 - -1. **외부 로깅 서비스(Sentry, Datadog 등) 연동** — 네트워크 의존성과 - API 키 관리가 필요해 이 CLI의 "설치 즉시 동작"하는 사용 모델과 - 맞지 않고, 제로 의존성 원칙과도 충돌. 기각. -2. **비구조화 stderr 텍스트 로그(`print(..., file=sys.stderr)`)** — - 구현은 가장 간단하지만, CI 로그가 뒤섞이는 환경에서 특정 실패를 - 특정 stdout 응답과 연결할 방법이 없다. 기각. -3. **표준 `logging` 모듈 + JSON Lines 포맷 + 상관관계 ID를 stdout·stderr - 양쪽에 동일하게 포함** — 표준 라이브러리만 사용하고, 매 요청마다 - 고유 ID를 발급해 두 출력 스트림을 연결할 수 있다. 채택. - -## 판단 기준 - -- stdout의 JSON 전용 계약(ADR-0009)을 절대 깨지 않을 것 — 상관관계 - ID는 stdout 쪽 에러 응답에 필드 하나 추가하는 형태로만 들어간다. -- 제로 의존성 원칙 유지. -- 테스트 스위트(pytest)에서 stderr 캡처가 깨지지 않아야 함 — 이는 - 구현 중 실제로 부딪힌 문제였고, 핸들러를 매번 초기화하는 방식으로 - 해결했다. - -## 최종 결정 - -`core/telemetry.py`에 JSON Lines 포맷 로거를 만들고, `adr.py`의 -전역 예외 핸들러가 여기에 예외를 기록하면서 동일한 상관관계 ID를 -stdout JSON 에러 응답에도 포함시킨다. - -## 해결 방식 - -1. `11c8f4b` — `core/telemetry.get_logger(operation)`이 - `LoggerAdapter`를 반환하도록 구현. 각 로그 라인은 JSON Lines - 형식으로 `level`, `operation`, `correlation_id`, `message`, - (예외 시) `exception_type` 필드를 담아 stderr에 출력된다. -2. `adr.py`의 예외 핸들러를 `logger = get_logger(args.operation); - logger.exception(...)` 형태로 바꾸고, 동일한 `correlation_id`를 - stdout으로 나가는 JSON 에러 응답에도 추가 — CI 로그에서 stderr의 - 특정 줄과 stdout의 특정 실패 응답을 상관관계 ID로 매칭할 수 있게 함. -3. 기본 로그 레벨은 `WARNING`(성공 시 조용함), `ADR_TOOLKIT_LOG_LEVEL` - 환경변수로 오버라이드 가능. -4. `mypy --strict` 게이트를 통과시키는 과정에서 발견된 타입 이슈(문서 - `2026-09-01-typed-contracts-mypy-strict.md` 참고)도 이 모듈에서 - 함께 수정됨 — `Iterator[None]` 반환 타입, `exc_info[0]` narrowing, - `LoggerAdapter` 제네릭 파라미터화. - -## 결과 - -- stdout의 순수 JSON 계약은 상관관계 ID 필드 추가 외에는 변경 없음 — - 기존 소비자(에이전트, CI 스크립트)와 호환. -- pytest의 capsys 기반 테스트가 핸들러 누적 없이 안정적으로 stderr를 - 캡처함을 확인. -- 같은 세션에서 이어진 TTY 전용 사람 친화적 요약 줄(`c3ed01d`, - Medium 패스)은 이 구조화 로깅과는 별개 기능 — 그쪽은 - `sys.stderr.isatty()`일 때만 보이는 인간용 한 줄 요약이고, 이 - telemetry 로거는 항상 JSON Lines로 기록되는 기계 판독용 로그다. diff --git a/docs/worklogs/2026-09-01-supply-chain-attestation.md b/docs/worklogs/2026-09-01-supply-chain-attestation.md deleted file mode 100644 index 5ab4f95..0000000 --- a/docs/worklogs/2026-09-01-supply-chain-attestation.md +++ /dev/null @@ -1,102 +0,0 @@ -# 릴리스 아티팩트 공급망 보안(Build Provenance Attestation) 도입 - -## 날짜 - -2026-09-01 - -## 문제 상황 - -`docs/adr-toolkit-audit-report.md` §2.2 2.2 감사 항목이 "공급망 보안 -체크섬/서명 부재"를 지적했다. `.github/workflows/release.yml`은 테스트 -실행, 매니페스트 버전 동기화 검사(`sync_version.py --check`), 태그== -`VERSION` 일치 검증까지만 하고 `softprops/action-gh-release@v2`로 -릴리스를 생성할 뿐, 릴리스에 첨부되는 어떤 아티팩트도 체크섬이나 -서명이 없었다. - -## 기존 구조나 방식의 한계 - -일반적인 npm/PyPI 프로젝트라면 "빌드 산출물을 체크섬 찍고 Sigstore로 -서명"하는 패턴이 바로 적용된다. 하지만 이 프로젝트는 빌드 산출물 -자체가 없다 — Claude Code 마켓플레이스(`marketplace.json`의 -`source: "./"`), Codex/Gemini CLI 플러그인 설치, 일반 copy/symlink 설치 -전부 git 저장소나 `skills/adr-toolkit/` 폴더를 **직접** 참조한다. -즉 "체크섬/서명할 아티팩트가 무엇인가"부터 정의되지 않은 상태였고, -감사 보고서의 원안(빌드 아티팩트를 체크섬+서명)을 그대로 옮기면 -아무도 실제로 소비하지 않는 파일에 서명하는 형식적 조치가 될 -위험이 있었다. - -## 관련 코드 맥락 - -- `.github/workflows/release.yml` — `v*` 태그 push 시 실행되는 유일한 - 릴리스 파이프라인. 기존에는 `permissions: contents: write`만 있었고 - 패키징 스텝이 전혀 없었다. -- `AGENTS.md`에 문서화된 릴리스 프로세스: 버전 태그는 **사람이 로컬에서 - 직접 생성**한 뒤 push한다. 즉 CI는 이미 push된 태그를 사후에 서명할 - 방법이 없다(태그 자체에 서명하려면 로컬 GPG 서명 절차가 별도로 - 필요하며, 이는 CI 워크플로 범위 밖). -- `SECURITY.md` — 취약점 신고 프로세스만 있고 릴리스 검증 방법에 대한 - 섹션이 없었다. - -## 검토한 선택지 - -1. **빌드 아티팩트 없음 + git 태그 자체에 서명(GPG)** — 태그가 로컬에서 - 사람이 만들기 때문에 CI 워크플로 안에서는 구현 불가. 기각. -2. **`skills/adr-toolkit/`를 tar.gz로 패키징 + 체크섬만** — 전송 중 - 손상은 잡지만 "진짜 이 저장소의 CI가 만들었는가"라는 provenance는 - 증명하지 못함. -3. **tar.gz 패키징 + SHA-256 체크섬 + GitHub Artifact Attestation - (`actions/attest-build-provenance@v2`)** — Sigstore 기반 keyless - 서명이라 개인키 관리/로테이션이 전혀 없고, GitHub Actions OIDC 토큰으로 - "이 커밋의 이 워크플로 실행이 만든 파일"이라는 provenance를 증명한다. -4. **자체 GPG 키를 리포지토리 시크릿으로 관리해 아티팩트 서명** — 개인키 - 보관/로테이션 부담이 있고, 이 프로젝트처럼 유지관리자가 1인인 - 상황에서는 키 손실/유출 리스크만 늘어남. 기각. - -## 판단 기준 - -- 실제 소비 경로(거의 모든 설치가 git repo/스킬 폴더 직접 참조)를 - 기준으로, "아무도 받지 않는 아티팩트에 서명"하는 헛수고를 피한다. -- CI가 사후에 할 수 있는 일만 범위에 넣는다 — 태그 서명처럼 이미 - 일어난 사람의 행동을 CI가 대신할 수 없는 것은 배제. -- 개인키를 만들거나 로테이션하는 운영 부담을 새로 만들지 않는다(1인 - 운영 프로젝트라는 현재 상태를 고려). - -## 최종 결정 - -옵션 3 — tar.gz 패키징 + SHA-256 체크섬 + GitHub Artifact Attestation. -GitHub Releases 페이지에서 직접 아카이브를 내려받는 소수의 경로에 대해 -"이 파일이 이 저장소의 이 커밋에서 만들어졌다"는 provenance를 keyless로 -증명하고, 나머지(git clone/plugin install 경로)는 기존처럼 Git/GitHub -자체의 커밋 이력으로 신뢰성을 확보한다는 점을 `SECURITY.md`에 명시했다. - -## 해결 방식 - -1. `.github/workflows/release.yml`의 `permissions`에 `id-token: write`, - `attestations: write` 추가(OIDC 토큰 발급 + attestation 게시 권한). -2. "Package the distributable skill" 스텝 추가: `VERSION` 파일을 읽어 - `adr-toolkit-skill-v${VERSION}.tar.gz`로 `skills/adr-toolkit`를 - 패키징하고 `sha256sum`으로 체크섬 파일 생성, `$GITHUB_OUTPUT`으로 - 아카이브 경로를 다음 스텝에 전달. -3. "Generate build provenance attestation" 스텝 추가: - `actions/attest-build-provenance@v2`에 `subject-path`로 방금 만든 - 아카이브 경로를 전달. -4. "Create GitHub Release" 스텝의 `files:`에 아카이브와 `.sha256` 파일을 - 함께 첨부. -5. `SECURITY.md`에 "Verifying a Release" 섹션 신설 — `sha256sum -c`와 - `gh attestation verify -R SHcommit/ADR-toolkit` 명령, 그리고 - git clone/adapter 설치 경로는 이 아카이브 검증과 무관하다는 점을 명시. - -## 결과 - -- 커밋 `18d4662` (`feat: add build provenance attestation to the release - workflow`). -- YAML 문법 검증 통과, 전체 테스트 스위트(`pytest tests/unit - tests/integration`) 541 passed로 회귀 없음 확인. -- 실제 OIDC 기반 attestation 발급/검증 플로우 자체는 GitHub Actions - 러너에서 실제 `v*` 태그 push가 일어나야만 최종 확인 가능 — 로컬에서는 - 워크플로 문법과 각 스텝의 셸 로직만 검증했다. 다음 실제 릴리스 - 태그(예: 다음 버전 bump) 때 `gh attestation verify`로 실물 검증 필요. -- 병렬로 진행 중이던 Codex 세션의 도입 지표(adoption metrics) 수집기 - 작업(`9a0de45`..`f814d64`, `scripts/adoption_metrics.py`)은 이 - 세션과 무관하게 이미 완료되어 있었음 — 이 작업으로 인한 충돌은 - 없었다. diff --git a/docs/worklogs/2026-09-01-typed-contracts-mypy-strict.md b/docs/worklogs/2026-09-01-typed-contracts-mypy-strict.md deleted file mode 100644 index fe12401..0000000 --- a/docs/worklogs/2026-09-01-typed-contracts-mypy-strict.md +++ /dev/null @@ -1,101 +0,0 @@ -# 타입드 결과 계약(`core/contracts.py`)과 범위 한정 `mypy --strict` 게이트 - -## 날짜 - -2026-09-01 - -## 문제 상황 - -`adr.py`의 모든 커맨드는 "stdout은 항상 순수 JSON"이라는 계약(ADR-0009)을 -따르지만, 그 JSON이 실제로 어떤 키를 가지는지는 각 커맨드의 `run()` -함수 본문을 읽어야만 알 수 있었다 — 타입 시스템 차원에서 보장되는 -스키마가 전혀 없었다. 감사 보고서는 이를 "출력 계약이 코드에만 존재하고 -타입으로 고정되지 않음" 문제로 지적했다. - -## 기존 구조나 방식의 한계 - -- 16개 커맨드 모두 `dict`를 리턴하며, 필드 이름 오타나 필드 누락이 - 런타임에만(혹은 소비하는 쪽 에이전트/스크립트에서만) 드러남. -- `jsonschema` 같은 런타임 스키마 검증 라이브러리를 쓰면 즉시 해결될 - 것처럼 보이지만, 이 프로젝트의 zero-dependency 원칙과 충돌한다. -- `argparse.Namespace`로 넘어오는 커맨드 인자(`args`)는 동적 속성 - 접근이라 `TypedDict`로 감싸기 어렵다 — 인자 쪽까지 완전히 타입화하려면 - `Protocol` 기반의 더 큰 리팩터링이 필요해서, 이번 패스에서는 "출력 - 결과 타입"만 범위로 잡았다. - -## 관련 코드 맥락 - -- `skills/adr-toolkit/scripts/core/atomic_io.py`, - `core/telemetry.py` — 이번 세션에서 새로 만든, 처음부터 완전히 - 타입 주석이 붙은 두 모듈. `mypy --strict` 게이트의 첫 적용 대상. -- `skills/adr-toolkit/scripts/commands/create.py`의 `run()` 리턴문 — - `CreateResult` TypedDict의 필드 목록을 정할 때 실제 리턴 딕셔너리를 - 읽고 역으로 타입을 뽑아냄(추측이 아니라 코드에서 도출). -- `.github/workflows/test.yml`의 `type-check` job — `mypy --strict`를 - 세 모듈(`atomic_io`, `telemetry`, `contracts`)에만 한정해서 실행. - -## 검토한 선택지 - -1. **`jsonschema` 도입 + JSON Schema로 런타임 검증** — 스펙 표준이라는 - 장점은 있지만 제로 의존성 원칙 위반. 기각. -2. **`dataclasses`로 결과 객체를 감싸고 `asdict()`로 직렬화** — 런타임 - 오버헤드와 기존 dict 기반 코드 전체(16개 커맨드, 관련 테스트 전부)를 - 바꿔야 하는 큰 리팩터링이 필요해 이번 감사 대응 범위에 비해 과함. - 기각. -3. **`TypedDict` + `mypy --strict`를 다 타입화된 모듈에만 우선 - 적용** — 런타임 동작을 전혀 바꾸지 않고(TypedDict는 런타임에 아무 - 효과가 없음) 정적 분석만으로 계약을 문서화·검증. 채택. - -## 판단 기준 - -- 런타임 동작을 바꾸지 않으면서 "출력 계약을 코드로 고정"하는 최소 - 침습적 방법이 우선. -- 이미 확인된 제로 의존성 제약을 다시 어기지 않을 것. -- `argparse.Namespace` 타입화라는 더 큰 리팩터링까지 한 패스에 묶으면 - 범위가 과도하게 커지므로, "출력 타입만" 먼저 고정하고 인자 타입화는 - 모듈 docstring에 명시적으로 향후 과제로 남긴다. - -## 최종 결정 - -`core/contracts.py`에 커맨드별 결과 `TypedDict`를 정의하고, -`mypy --strict` CI 게이트를 새로 만들되 이미 완전히 타입 주석이 붙은 -핵심 모듈에만 적용한다. 명령어 인자(`argparse.Namespace`) 타입화는 -범위 밖으로 명시적으로 남긴다. - -## 해결 방식 - -1. `305c836` — `core/contracts.py` 신설: - `CommandError`/`BaseResult`/`ErrorResult`/`CreateResult` 4개 - TypedDict로 시작. `type-check` CI job을 추가해 `atomic_io`, - `telemetry`, `contracts` 세 모듈에 `mypy --strict` 적용. 이 과정에서 - 실제로 발견한 3개의 진짜 mypy 오류를 수정: - - `adr_directory_lock`의 컨텍스트 매니저 제너레이터에 `Iterator[None]` - 반환 타입 누락. - - `record.exc_info[0]`가 `Optional[type[BaseException]]`이라 미리 - narrowing 없이 쓰면 오류 — `and record.exc_info[0] is not None` - 가드 추가. - - `logging.LoggerAdapter`를 파라미터화하지 않은 제네릭으로 써서 - 오류 — `"logging.LoggerAdapter[logging.Logger]"` 문자열 애노테이션으로 - 해결. - - (나중 커밋에서 추가 발견) TypedDict 필드에 맨 `dict`를 쓰면 - `mypy --strict`의 `type-arg` 검사에 걸림 — `Dict[str, Any]`로 - 교체해야 함. -2. `1df6066` (팔로우업) — 나머지 14개 커맨드(`preflight`, `discover`, - `init`, `index`, `related`, `significance`, `validate`, `status`, - `supersede`, `diff`, `exception`, `graph`, `search`)까지 확장해 - 2/16 → 16/16 커버리지. 각 TypedDict 필드는 실제 `run()` 리턴문을 - 읽어서 결정했고(추측 금지), `status`/`supersede`의 에러 경로까지 - 실제로 실행해서 대조 확인. 에러/경고/중첩 페이로드 필드는 공용 - `CommandError` 타입 대신 `Dict[str, Any]`를 쓴 경우가 있는데, 이는 - 실제 에러 딕셔너리가 `file`/`id`/`ids`/`cycle` 등 `CommandError`가 - 선언하지 않은 추가 필드를 갖는 커맨드가 있어서, 사실이 아닌 구조를 - 타입으로 과장하지 않기 위함. - -## 결과 - -- `mypy --strict`가 실제로 CI에 게이트로 걸려 통과 상태 유지 중. -- 남은 과제(코드 주석 및 `handoff.md`에 기록): `mypy --strict`를 16개 - 커맨드 모듈 자체(현재는 결과 타입만 타입화되고 커맨드 구현부는 - 미적용)까지 확장하는 건 `argparse.Namespace` 타입화 리팩터링이 - 선행돼야 해서 여전히 향후 과제로 남아있다. -- 전체 테스트 스위트 회귀 없이 각 커밋 완료. From 2d393488c92c8e70e0c439220ff561971dd3238d Mon Sep 17 00:00:00 2001 From: shcommit Date: Wed, 2 Sep 2026 09:23:46 +0900 Subject: [PATCH 57/58] chore: bump version to 0.3.0 Rolls changelog.md's Unreleased section into a v0.3.0 entry and propagates VERSION to every manifest via scripts/sync_version.py. README.md needed no changes (no version strings, and this release's hardening work doesn't add new user-facing operations); examples/ verified clean with no drift via scripts/verify_examples.py. --- .claude-plugin/plugin.json | 2 +- adapters/antigravity/plugin.json | 2 +- adapters/gemini-cli/gemini-extension.json | 2 +- changelog.md | 7 +++++++ skills/adr-toolkit/SKILL.md | 2 +- skills/adr-toolkit/VERSION | 2 +- 6 files changed, 12 insertions(+), 5 deletions(-) diff --git a/.claude-plugin/plugin.json b/.claude-plugin/plugin.json index 21d12db..eec16f3 100644 --- a/.claude-plugin/plugin.json +++ b/.claude-plugin/plugin.json @@ -1,5 +1,5 @@ { "name": "adr-toolkit", - "version": "0.2.1", + "version": "0.3.0", "description": "Initialize, record, and check Architecture Decision Records by inspecting the repository and existing decisions before asking questions." } diff --git a/adapters/antigravity/plugin.json b/adapters/antigravity/plugin.json index 9e67648..b9d857f 100644 --- a/adapters/antigravity/plugin.json +++ b/adapters/antigravity/plugin.json @@ -1,6 +1,6 @@ { "$schema": "https://antigravity.google/schemas/v1/plugin.json", "name": "adr-toolkit", - "version": "0.2.1", + "version": "0.3.0", "description": "Initialize, record, and check Architecture Decision Records by inspecting the repository and existing decisions before asking questions." } diff --git a/adapters/gemini-cli/gemini-extension.json b/adapters/gemini-cli/gemini-extension.json index 21d12db..eec16f3 100644 --- a/adapters/gemini-cli/gemini-extension.json +++ b/adapters/gemini-cli/gemini-extension.json @@ -1,5 +1,5 @@ { "name": "adr-toolkit", - "version": "0.2.1", + "version": "0.3.0", "description": "Initialize, record, and check Architecture Decision Records by inspecting the repository and existing decisions before asking questions." } diff --git a/changelog.md b/changelog.md index d1ecc73..3e3a945 100644 --- a/changelog.md +++ b/changelog.md @@ -4,6 +4,13 @@ Lightweight human-readable summary of meaningful repository changes. ## Unreleased +## v0.3.0 (2026-09-02) + +- Recorded this release's architectural decisions as ADR-0012 through + ADR-0016 (atomic writes + directory locking, the two-stage ReDoS guard, + typed result contracts + `mypy --strict`, structured JSON logging with + correlation IDs, and release artifact attestation), using the ADR + toolkit itself. - `.github/workflows/release.yml` now packages `skills/adr-toolkit/` into a version-named tarball, SHA-256 checksums it, and generates a Sigstore-backed GitHub Artifact Attestation (`actions/attest-build-provenance@v2`, keyless) diff --git a/skills/adr-toolkit/SKILL.md b/skills/adr-toolkit/SKILL.md index e5e47b7..3a103db 100644 --- a/skills/adr-toolkit/SKILL.md +++ b/skills/adr-toolkit/SKILL.md @@ -2,7 +2,7 @@ name: adr-toolkit description: Initialize, record, and check Architecture Decision Records by inspecting the repository and existing decisions before asking questions. user-invocable: true -version: 0.2.1 +version: 0.3.0 --- # ADR Toolkit diff --git a/skills/adr-toolkit/VERSION b/skills/adr-toolkit/VERSION index 0c62199..9325c3c 100644 --- a/skills/adr-toolkit/VERSION +++ b/skills/adr-toolkit/VERSION @@ -1 +1 @@ -0.2.1 +0.3.0 \ No newline at end of file From 5e8307e595d5ba0235f6449bc309935f87787901 Mon Sep 17 00:00:00 2001 From: shcommit Date: Wed, 2 Sep 2026 09:31:47 +0900 Subject: [PATCH 58/58] fix: make resolve_from_root's containment check filesystem-independent CI on PR #8 caught a real, Windows/Python-3.9-only failure: `adr.py init --dir docs/decisions` (and every other resolve_from_root call site) rejected a plainly-under-root path with PATH_ESCAPES_ROOT, only on windows-latest with Python 3.9 -- not 3.12, and not on ubuntu/macOS at any version. `docs/decisions` doesn't exist on disk yet when INIT scaffolds it, and Path.resolve() on a non-existent path has inconsistent cross-version behavior on Windows. Fix: check containment by lexically normalizing the joined path (os.path.normpath, no filesystem access) against the already-resolved root, instead of calling .resolve() on a path that may not exist yet. The threat model here is lexical `..` traversal, which normpath handles without ever touching the filesystem. All 13 repository_paths/path-escape tests plus the full suite (541) still pass locally. --- skills/adr-toolkit/scripts/core/repository_paths.py | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/skills/adr-toolkit/scripts/core/repository_paths.py b/skills/adr-toolkit/scripts/core/repository_paths.py index 44a2a50..b8fe1c5 100644 --- a/skills/adr-toolkit/scripts/core/repository_paths.py +++ b/skills/adr-toolkit/scripts/core/repository_paths.py @@ -1,4 +1,5 @@ """Resolve repository-owned paths independently of the caller's CWD.""" +import os from pathlib import Path from scripts.core.errors import AdrToolkitError @@ -19,7 +20,15 @@ def resolve_from_root(root, path) -> Path: joined = Path(root) / candidate root_resolved = Path(root).resolve() - if not joined.resolve().is_relative_to(root_resolved): + # `joined` (e.g. INIT scaffolding a brand new "docs/decisions") commonly + # doesn't exist on disk yet. Path.resolve() on a non-existent path has + # inconsistent cross-version behavior on Windows (observed: Python 3.9 + # rejects a legitimate under-root path that Python 3.12 accepts + # identically), so the containment check is done with pure lexical + # normalization (os.path.normpath, no filesystem access) against the + # already-resolved root instead of resolving the joined path itself. + joined_normalized = Path(os.path.normpath(str(root_resolved / candidate))) + if not joined_normalized.is_relative_to(root_resolved): raise PathEscapesRootError(f"{str(path)!r} escapes root {str(root)!r}") return joined