Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/ROADMAP_V2.md
Original file line number Diff line number Diff line change
Expand Up @@ -341,7 +341,7 @@ exact-only compound restriction is superseded by #439's approved scope.

| Milestone | Contenido y criterio de salida |
|---|---|
| [M1 — Safe, inspectable execution](https://github.com/datacas/code-cycle-toolkit/milestone/1) | #327–#330, #259/#261/#290/#294, presupuestos #262 y overrides #285; #326 coordina. Runtime fixes found while running #330: #385 (P0, manual checks restore and idempotent resume; blocks #330), #382 (host-attested verification evidence) and #383 (malformed stage questions), #388 (P2, manual-check name grammar and resume-history tests; follow-up to #387) and #389 (P1, quota-aware routing and controlled fallback handoff on resume; blocks resolving REV-001 on #384), #394 (P1, resume pending verification when a stored check has no authorized command binding) #395 (P2, quota recovery commands accept repository-registered models), #402 (P2, Orca Task-spec reader binding and stock-CLI documentation; follow-up to #330), #393 (P0, urgent, resume blocked on frozen check identity; fixed in #404) and #406 (P0, urgent, ask the user instead of blocking on any command outside a stage's command table; builds on #289; implemented in #409), plus the stops reported on 2026-11-04, all urgent: #410 (P1, execution plan receipt rejected without naming the field), #411 (P1, remote-ref inspection of the review workspace without retry), #412 (P1, stage report rejected on schema grounds ends BLOCKED after a push), #413 (P1, another rereview of the same head instead of the CI gate), #414 (P1, required verification a review stage cannot run loops on resume), #415 (P1, `cycle_status.py --line` mixes cycles of other repositories) and #416 (P1, `basedpyright` does not fit the verification limit); #383 and #364 are urgent as well. Follow-ups found while running them: #421 (P1, urgent, a base-branch advance by another actor fails running stages as a publication breach or read-only violation), #425 (P1, urgent, an accepted `not_verified` check ends a rereview as `HUMAN_INTERVENTION` instead of `VERIFICATION_INCOMPLETE`) and #422 (P2, `read_operands` misses file operands behind abbreviated grep long options and combined jq short flags; follow-up to #406) and #428 (P0, urgent, interim guidance for the `credential_reads` remedy when `issue_review` cannot read the gh store; docs, skills and test only; the runtime fix is #397), #397 (P0, urgent, `issue_review` reads private GitHub issue data through a runtime snapshot instead of the gh credential store), #398 (P0, urgent, ask for consent to run an authorized Docker Compose `run` verification command instead of refusing it) and #435 (P1, urgent, a `READY` issue review is rejected when the runtime adds `permission_outcomes` to its result; follow-up to #406), #426 (P2, how a cycle reacts to an external base-branch advance: branch update and revalidation; follow-up to #421), #432 (P2, a remote inspection failure after review masks a changed implementer HEAD or working tree; found in #411) and #433 (P2, a review report with an integer `checks[].attempts` is rejected after a completed approval; follow-up to #412) and #377 (P1, per-project and per-check verification timeouts; moved from M2 because the 10-minute limit stopped two consecutive stages on a passing `basedpyright` run that needs 12 min 25 s, see #416), #446 (P1, bounded re-read that confirms a review or rereview publication before retrying; split from #364), #450 (P0, urgent, core principle of agent capability parity: audit and remove the toolkit's own vetoes of what the agent can run) with its child issues for the vetoes no earlier issue tracked, listed in `docs/capability-parity-audit.md`: #455 (P1, consent for compound shell commands instead of a parse-time refusal), #456 (P1, consent for Git, gh and Docker table refusals), #457 (P1, the never-grant boundaries become consent with a risk warning and a record), #458 (P2, run with a disclosed reduced guarantee when the configuration defeats the hook) and #459 (P1, normalize report and receipt representations instead of rejecting them), and #187 (P1, split `CycleRecorder.stage` into typed helpers; moved from M7 because that one 755-line method makes the `basedpyright` check take more than 10 minutes, target 10 s or less, see #416 and #377) and #438 (P0, urgent, one malformed stage receipt in an accumulating comment hides every receipt next to it and blocks a clean review; receipts are read one by one, related to #409 and #412), and #460 (P2, clear a stale declined-check record on a later allow and disclose consented `$TMPDIR` prefixes; follow-up to #398) and #439 (P0, urgent, a grant covers its variants: pattern (regular expression) grants, reads outside the stage workspace and layer attribution of refusals; follow-up to #406) and #464 (P0, urgent, a finished stage ends `BLOCKED` on refused read-only commands and detached runs cannot answer consent; follow-up to #406 and #450)) and #469 (P1, `plan_projection` raises `AttributeError` on a non-object `evidence` item and a finished stage loses its report) and #470 (P1, urgent, the parallel CI gate ends `CI_UNREADABLE` when the plan has no branch-protection API; fall back to declared or SHA check runs; case of #450). Cinco perfiles resolubles, plan inspeccionable, ejecución nativa/Orca coherente, controles obligatorios no reducibles y resume trazable. #147 es independiente y #189 recomendable. Correcciones operativas asignadas siguen condicionadas a reproducción. |
| [M1 — Safe, inspectable execution](https://github.com/datacas/code-cycle-toolkit/milestone/1) | #327–#330, #259/#261/#290/#294, presupuestos #262 y overrides #285; #326 coordina. Runtime fixes found while running #330: #385 (P0, manual checks restore and idempotent resume; blocks #330), #382 (host-attested verification evidence) and #383 (malformed stage questions), #388 (P2, manual-check name grammar and resume-history tests; follow-up to #387) and #389 (P1, quota-aware routing and controlled fallback handoff on resume; blocks resolving REV-001 on #384), #394 (P1, resume pending verification when a stored check has no authorized command binding) #395 (P2, quota recovery commands accept repository-registered models), #402 (P2, Orca Task-spec reader binding and stock-CLI documentation; follow-up to #330), #393 (P0, urgent, resume blocked on frozen check identity; fixed in #404) and #406 (P0, urgent, ask the user instead of blocking on any command outside a stage's command table; builds on #289; implemented in #409), plus the stops reported on 2026-11-04, all urgent: #410 (P1, execution plan receipt rejected without naming the field), #411 (P1, remote-ref inspection of the review workspace without retry), #412 (P1, stage report rejected on schema grounds ends BLOCKED after a push), #413 (P1, another rereview of the same head instead of the CI gate), #414 (P1, required verification a review stage cannot run loops on resume), #415 (P1, `cycle_status.py --line` mixes cycles of other repositories) and #416 (P1, `basedpyright` does not fit the verification limit); #383 and #364 are urgent as well. Follow-ups found while running them: #421 (P1, urgent, a base-branch advance by another actor fails running stages as a publication breach or read-only violation), #425 (P1, urgent, an accepted `not_verified` check ends a rereview as `HUMAN_INTERVENTION` instead of `VERIFICATION_INCOMPLETE`) and #422 (P2, `read_operands` misses file operands behind abbreviated grep long options and combined jq short flags; follow-up to #406) and #428 (P0, urgent, interim guidance for the `credential_reads` remedy when `issue_review` cannot read the gh store; docs, skills and test only; the runtime fix is #397), #397 (P0, urgent, `issue_review` reads private GitHub issue data through a runtime snapshot instead of the gh credential store), #398 (P0, urgent, ask for consent to run an authorized Docker Compose `run` verification command instead of refusing it) and #435 (P1, urgent, a `READY` issue review is rejected when the runtime adds `permission_outcomes` to its result; follow-up to #406), #426 (P2, how a cycle reacts to an external base-branch advance: branch update and revalidation; follow-up to #421), #432 (P2, a remote inspection failure after review masks a changed implementer HEAD or working tree; found in #411) and #433 (P2, a review report with an integer `checks[].attempts` is rejected after a completed approval; follow-up to #412) and #377 (P1, per-project and per-check verification timeouts; moved from M2 because the 10-minute limit stopped two consecutive stages on a passing `basedpyright` run that needs 12 min 25 s, see #416), #446 (P1, bounded re-read that confirms a review or rereview publication before retrying; split from #364), #450 (P0, urgent, core principle of agent capability parity: audit and remove the toolkit's own vetoes of what the agent can run) with its child issues for the vetoes no earlier issue tracked, listed in `docs/capability-parity-audit.md`: #455 (P1, consent for compound shell commands instead of a parse-time refusal), #456 (P1, consent for Git, gh and Docker table refusals), #457 (P1, the never-grant boundaries become consent with a risk warning and a record), #458 (P2, run with a disclosed reduced guarantee when the configuration defeats the hook) and #459 (P1, normalize report and receipt representations instead of rejecting them), and #187 (P1, split `CycleRecorder.stage` into typed helpers; moved from M7 because that one 755-line method makes the `basedpyright` check take more than 10 minutes, target 10 s or less, see #416 and #377) and #438 (P0, urgent, one malformed stage receipt in an accumulating comment hides every receipt next to it and blocks a clean review; receipts are read one by one, related to #409 and #412), and #460 (P2, clear a stale declined-check record on a later allow and disclose consented `$TMPDIR` prefixes; follow-up to #398) and #439 (P0, urgent, a grant covers its variants: pattern (regular expression) grants, reads outside the stage workspace and layer attribution of refusals; follow-up to #406) and #464 (P0, urgent, a finished stage ends `BLOCKED` on refused read-only commands and detached runs cannot answer consent; follow-up to #406 and #450)) and #469 (P1, `plan_projection` raises `AttributeError` on a non-object `evidence` item and a finished stage loses its report) and #470 (P1, urgent, the parallel CI gate ends `CI_UNREADABLE` when the plan has no branch-protection API; fall back to declared or SHA check runs; case of #450) and #477 (P0, urgent, a rereview ends `BLOCKED` on a report missing a frozen check's command and on unidentified checks read as declined; follow-up to #412, #433, #398 and #450). Cinco perfiles resolubles, plan inspeccionable, ejecución nativa/Orca coherente, controles obligatorios no reducibles y resume trazable. #147 es independiente y #189 recomendable. Correcciones operativas asignadas siguen condicionadas a reproducción. |
| [M2 — Evidence-based quality and verification](https://github.com/datacas/code-cycle-toolkit/milestone/2) | #146, #156–#159, #191, #260, #251, #331, #367 (condicionada a #329) y #237 refinada; verificación escalable a suites largas #376–#379 (#376 evalúa primero; #380, límite de concurrencia, condicionada a medición); apoyo de #144/#183, CI #145/#231 y trazabilidad #199. Reviews consumen evidencia comparable, behavioral funciona dentro del ciclo y legacy/desconocidos conservan su estado. #332 es opcional. |
| [M3 — Supported execution surfaces and DevSecOps pilot](https://github.com/datacas/code-cycle-toolkit/milestone/3) | #219, #238-A y #238-B según su secuencia; #333/#334, #291 donde haga falta y #257; #246/#282 condicionadas. Matriz de ejecutores/proveedores declarada y validada, y piloto security sin nuevas vías de privilegio. |
| [M4 — Measured cycle effectiveness](https://github.com/datacas/code-cycle-toolkit/milestone/4) | #185, #192, #180 y #127 (primera entrega: observabilidad descriptiva de revisiones). Tiempos, costes, rework, findings e intervenciones por cohortes con cobertura de datos visible. [#391](https://github.com/datacas/code-cycle-toolkit/issues/391), P2: benchmark offline de reviewers, A/B informativo por caso con etiquetas independientes; entrega experimental independiente, sin bloquear el candidato principal ni depender de #126/#331/#335. |
Expand Down
2 changes: 1 addition & 1 deletion docs/capability-parity-audit.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ given instead of line numbers so the references stay valid.
| PA-31 | When Claude's Bash sandbox cannot start, static-only mode denies verification, Docker, arbitrary-code tool choices and publication, and the checks become `not_verified` | `executors.py` (`static_only=True`); `stage_permissions._check_words` and `_check_git` | No repository code runs unconfined; no host-side push | #345, #348 | toolkit veto | #291 (checks), #457 (publication) |
| PA-32 | A Claude dispatch stops `BLOCKED` when the loaded configuration could defeat the hook (`disableAllHooks`, managed `allowManagedHooksOnly`, a `tool.check` mod) or widens the sandbox; a Codex review stops `BLOCKED` when its sandbox cannot be shown to confine writes | `executors.py` `policy_preflight`; `stage_permissions.sandbox_override`, `tool_check_mods`; Codex review sandbox proof | The command table must stay in force | #142 | toolkit veto | #458 |
| PA-33 | A configured verification command needs a Bash timeout of at least 600 s, and the toolkit sets `BASH_MAX_TIMEOUT_MS` to the same 600 s, so a longer check cannot finish | `stage_permissions.CLAUDE_VERIFICATION_TIMEOUT_MS`; `executors.py` dispatch environment | Avoids a check timing out early | #416: `basedpyright` needs 12 min 25 s | toolkit veto | #377 |
| PA-34 | A stage report that differs only in representation is rejected: plan receipt keys `plan_revision` / `plan_digest` instead of `revision` / `digest`, `evidence` entries in another shape, unknown keys in a closed result contract. `checks[].attempts` as an integer or a string is no longer rejected: #453 normalizes it | `scripts/review_contract.py` (`PLAN_RECEIPT_KEYS`), `scripts/issue_review.py` (`RESULT_KEYS`), `scripts/verification_plan.py` (`normalize_attempts`) | Machine-readable results | #433, #412, #435 | toolkit veto | #433 (converted by #453 for `attempts`), #459 |
| PA-34 | A stage report that differs only in representation is rejected: plan receipt keys `plan_revision` / `plan_digest` instead of `revision` / `digest`, `evidence` entries in another shape, unknown keys in a closed result contract. `checks[].attempts` as an integer or a string is no longer rejected: #453 normalizes it | `scripts/review_contract.py` (`PLAN_RECEIPT_KEYS`), `scripts/issue_review.py` (`RESULT_KEYS`), `scripts/verification_plan.py` (`normalize_attempts`) | Machine-readable results | #433, #412, #435 | toolkit veto | #433 (converted by #453 for `attempts`), #459; a `tests` block the runtime cannot read no longer stops the cycle `BLOCKED` and a reported frozen check without `command` takes it from the frozen check (#450) |
| PA-35 | A grant for a configured verification command does not cover its variants (an environment prefix, `rtk`, a redirection into `$TMPDIR`), so the stage asks again | `grants.py` exact matching; `stage_permissions.operation` | Exact consent | #439 | toolkit veto | #439 |
| PA-36 | A publishing stage that reported `BLOCKED` after a verified publication stopped the cycle as `stage_not_completed`. Since #464 a stage that reports `BLOCKED` without `completed_status` is judged on its structured result (`inferred_completion`), and a stop that remains names the real cause apart from the incidental refusals | `scripts/command_permissions.py` `reconcile_result`, `inferred_completion`; `scripts/cycle_dispatch.py` `blocked_cause` | Completion evidence | #364, #446, #397 | toolkit veto | #364 (converted by #449 and #464) |
| PA-37 | A review or rereview publication the runtime did not observe was treated as missing and retried without confirming the remote state. Since #446 it is re-read three times with backoff before it is called absent, and a persistent read failure stops the cycle as `BLOCKED` with `publication_unverified`, which is a real limit of the provider and never triggers another publication | `scripts/cycle_publication.py` (`settle_publication`), `scripts/cycle_history.py` (`REREAD_BACKOFF`) | Completion evidence | #446 | toolkit veto | #446 (converted by #446) |
Expand Down
14 changes: 13 additions & 1 deletion docs/verification.md
Original file line number Diff line number Diff line change
Expand Up @@ -254,6 +254,15 @@ number that is not an integer, a boolean, an entry that is neither an object nor
a string) is rejected with the field, the received type and the expected shape.
`evidence` is not normalized this way.

A report that names a frozen check (`configured-test`, `verify:<command>`) and
leaves out part of its identity (`command`, `executor`, `profile`, `context`,
`required`, `level`, `criteria`) is the same check: the runtime takes the missing
fields from the frozen one. A field the report does give still has to match.
A tests block the runtime cannot read at all never ends the cycle `BLOCKED`:
the stage's report is accepted, a note and a warning name the field, its checks
stay `not_verified`, and the verification barrier asks about them at the end like
any other unverified check.

Freeze each check before execution. A result names the same check ID, command,
executor/profile and context. The runtime supplies frozen check identities to
later code stages. Copy that identity, including its owning executor/profile,
Expand Down Expand Up @@ -555,7 +564,10 @@ that obeys the denial and never tries the check again. The cycle ends
the pending check; any other final state names it too. To authorize it, revoke
the denial with `consent_control.py revoke --repo owner/repo --executor
claude|codex --grant-id command_<sha256>`, resume, and allow it when asked, or
pass `--grant command_<sha256>=persistent`. A stage that reports `BLOCKED` over
pass `--grant command_<sha256>=persistent`. A configured check whose
identity the runtime cannot read (`unidentified`) was declined by nobody: it is
not recorded as declined, carries no grant to answer, and the agent's own
evidence, or the plan's `unverified`, decides it. A stage that reports `BLOCKED` over
the declined check without a structured completion claim stays `BLOCKED`: the
runtime never invents a verdict, and the stop names the check. A denial that
would break repository integrity, publication, credentials or security
Expand Down
Loading
Loading