Craft Agents Multi-Agent Orchestration Protocol v3.4.41 — Complete Standalone Guide

Snapshot: 2026-08-16 18:35 Europe/Warsaw Audience: operators and contributors building safer coordinator/worker/auditor control planes with Craft Agents. Purpose: deliver owner-requested product outcomes through autonomous project coordinators while preserving work, preventing split-brain, detecting stalls deterministically, gating irreversible actions, and using evidence as bounded acceptance rather than work for its own sake.

This repository is self-contained: it includes the guide, executable scripts, launchd configuration, canonical skills/prompts, labels configuration, regression tests, and SHA-256 manifest. It contains no API keys, credentials, project repositories, private runtime receipts, or session transcripts.

Craft Protocol is open source under the Apache License 2.0. Contributions are welcome. Start with CONTRIBUTING.md, discuss broad designs in GitHub Discussions, and follow GOVERNANCE.md and CODE_OF_CONDUCT.md. See SUPPORT.md for help channels and report vulnerabilities privately according to SECURITY.md.

Quick start

git clone https://github.com/razumv/craft-protocol.git
cd craft-protocol
./install.sh          # dry-run; no files changed

Install/start a separately reviewed capability-v2 Craft runtime first. The v3.3.0 production-tested identity is 0.11.4-admission.87951ae / 87951ae640df64d00534a54dce9b5e8b5922d27c (including queue-only busy coordinator and recovery-controller delivery), then review the Protocol plan and use ./install.sh --apply. The Protocol installer restores the kill switch before its first payload copy. Merge config/labels.config.json manually rather than replacing an existing label configuration. Self-healing scheduler automations ship permanently disabled. Protocol v3.2.2 requires authenticated admission capability v2 with exact runtime version/commit pinning, durable delivery inspection, and one guarded recovery CAS. Outstanding messages are coalesced until runtime-proven consumption; plain queued is never success. Routine stale/current-handoff/terminal-wait events go directly to the exact authoritative coordinator generation, while complex or destructive recovery remains controller-bound/fail-closed. Keep the kill switch present until the explicit workspace ID, expected runtime version/commit, and owner-only server token are configured; review the v3.2.2 admission guide. Protocol v3.3.0 adds, on top of that unchanged admission lane, a durable coordinator inbox, a per-project product-status snapshot with an owner-facing aggregate report, and observer-bound commitments so report storms cannot extend a coordinator turn and every future wait has a durable observer; review the v3.3.0 coordinator inbox guide. Protocol v3.4.0 reuses those primitives and changes the delivery unit to a Product Increment: a bounded story DAG, one integrated candidate, batch CI/deploy, risk-boundary acceptance, classified failures, a real-workflow demonstration, and customer-first owner reporting; review Product Increments v3.4. No runtime server upgrade is required beyond the production-tested v3.3 capability-v2 runtime.

1. Mental model

graph LR
    O[Human owner] --> G[Owner gates]
    G --> C[Authoritative coordinator]
    C --> W1[Worker attempt]
    C --> W2[Worker attempt]
    C --> A[Independent auditor]
    W1 --> L[Leases and job receipts]
    W2 --> L
    A --> L
    L --> D[Deterministic watchdog]
    D --> I[Idempotent recovery incidents]
    I --> P[Consumption-aware admission]
    P -->|routine exact generation| C
    P -->|complex recovery| H[One persistent recovery controller]
    H --> C
    C --> R[Recovery ledger]
    A --> X[Completion certificate]
    X --> C

Roles

Owner: decides product priority and substantive or irreversible questions. Relayed approval is invalid.
Owner-facing infrastructure session: relays exact owner instructions and queries status only on owner request. It does not supervise routine project work, acknowledge coordinator updates, or become a second coordinator.
Coordinator: autonomous persistent project/scope controller. Exactly one authoritative coordinator per scope; it drives product delivery without routine central approval.
Worker: disposable implementation attempt in a unique worktree.
Auditor: disposable skeptical, read-only attempt in its own unique worktree.
Watchdog: deterministic, non-LLM reconciliation. It detects drift, emits idempotent incidents, and performs only preservation-safe cleanup.
Recovery controller: one persistent, bounded infrastructure session awakened only by an admitted notifier. It wakes/reconciles coordinators and may clean only fully preservation-proven terminal lanes; it never decides owner gates or supervises product work.

Sources of truth

GitHub milestone/issues/dependencies/Project fields: what work exists.
Coordinator registry: who owns the project scope.
Owner-gate registry: what the owner has or has not authorized.
Leases/jobs/recovery ledger: what is actually executing.
Completion certificates: what is proved complete.

Chat claims, silence, status names, and repeated CI polling are not authoritative evidence.

2. Load-bearing invariants

Never kill or restart the Craft Agents application process.
Preserve dirty/unpushed work before archive, replacement, or cleanup.
Every worker, replacement, and auditor attempt gets a fresh unique worktree.
Exactly one authoritative coordinator per project/repository scope.
A successor adopts live attempts; it never duplicates them without evidence of terminal failure.
Workers and auditors use permissionMode: allow-all; read-only auditing is a behavioral mandate, not Explore mode.
Routine work does not call SubmitPlan; it is used only when the owner explicitly requests plan review in that exact session.
Default delivery WIP: one primary Product Increment—normally 3–8 coherent stories, but one story is valid when complete—with at most 2 disjoint lightweight story lanes and one integrated candidate.
Only one global heavy job runs at once by default; bounded resource-aware exceptions require explicit authority.
Run scoped checks per story, then one batch CI/deploy and risk-tiered acceptance at the integrated increment boundary; Low risk has no auditor by default.
No audit-of-audit. One product-acceptance failure permits one exact correction and one final re-acceptance; a second or repeated same-root failure escalates instead of spawning attempt N+1.
Admission/environment failure preserves evidence and does not spend product correction budget.
Tests, reports, gates, and certificates verify a candidate; they are not independent indefinite product work.
No irreversible product action without direct owner authority or an exact standing-authority match.
A project HOLD blocks spawn, implementation, merge, and closure until exact direct-owner RESUME.
UI completion requires evidence from the real desktop/mobile/user workflow; unit/DOM checks alone are insufficient.
Session archive comes before guarded harness reaping.
A shared cwd or unknown PID is a hard cleanup refusal.
Coordinators do not send routine updates to the owner-facing infrastructure session, and that session does not acknowledge or poll them without owner request.
When the owner asks for status, lead with customer outcome, demonstrable workflow, remaining work, ETA/confidence and one blocker; PR/SHA/CI/session/audit details are secondary evidence only.

v3.4.41 admission-scale and operational-stability controls

Admission invokes craft-cli --timeout 110000 under a validated 120-second supervisor deadline (configurable as cliTimeoutSeconds / supervisorTimeoutSeconds, with at least five seconds of ordering grace). This permanently replaces the temporary handwritten wrapper and covers measured 28–110 second RPC latency without weakening delivery-unknown handling.
Periodic admission no longer enumerates every workspace before each target cycle. verify-runtime retains the explicit workspace-ID/root activation proof; ticks use exact manifest, target-generation, scope, receipt, and content fences.
A stale exact-identity content-revision receipt is reconciled at most once, under the same immutable message ID/scope, only after exact inspection proves an idle target, empty queue, and a later unrelated legitimate final. That final is never called consumption; active/queued work, fence mismatch, ambiguous outcome, and duplicate mutation fail closed with an auditable reason.
Strict lane confirmation permits a worker/auditor subprocess to run from its own exact reserved canonical worktree. Reserve still rejects the authoritative parent worktree and invoking repository root, while confirm retains exact identity equality, canonicalization, parent truth, runtime identity, token binding, and all admission/manifest/lease/work-unit collision checks.
Rotation pressure is measured against the current coordinator session only. A bounded post-claim/transfer grace window prevents inherited counter noise; lower clear thresholds provide hysteresis after a true pressure signal. Request-buffer/context errors bypass both protections and remain concrete recovery evidence.
Complete v3.4.41 Product Increments consume an immutable completion-certificate file binding and exact candidate, verdict:PASS, acceptance event, merge authority, merge SHA, and merged-main readback on one work unit. Missing, cross-work-unit, stale, or byte-mutated certificate evidence refuses completion. A predecessor left live after an otherwise healthy transfer is classified healthy-with-maintenance-debt, never silently ignored and never conflated with a context failure.
owner-plan-receipt.py requires an authenticated configured owner session, exact plan bytes/fingerprint, increment/work-unit scope, explicit safe effects, every dangerous exclusion, bounded expiry, and durable revocation. It cannot authorize credentials, spending, deployment, protected merges, releases, remote access, or irreversible changes.
release-closure.py verify is read-only and refuses closure without a mode-restricted explicit token file and TLS-authenticated requests to fixed api.github.com. It derives only canonical GitHub origin, reads remote main plus annotated-tag peel directly (never cached origin/main), accepts only ASCII-whitespace-wrapped strict base64 Contents payloads, and requires exact Release/Latest/freshness/assets, a remote manifest with the exact locally audited path/digest set (no omissions, extras, duplicates, or unsafe paths), an authenticated final Release-body fleet-adoption receipt, and no-mutation installer proof. It ignores caller-selected CLI/API environment overrides and never tags, pushes, installs, or prints credentials.
Transfer acceptance revalidates both discovered and pre-existing activeChildren: only live, predecessor/project-bound worker/auditor lanes with unique work-unit/attempt bindings and non-colliding canonical worktrees are adoptable. Archived, foreign, stale, duplicate, or shared-cwd records refuse the handoff.
Pull-only reporting retains project-local violation evidence. Outbound interception remains unavailable and is never claimed; completed transcript sends to the owner-facing session require a one-use, expiring marker-bound permit for one exact prior owner request. An unresolved violation blocks claim/renew/healthy authority and drives preservation-safe successor rotation, while silence and the absence of an owner query never pause autonomous project work.

v3.4.36 product-pressure controls

Every Product Increment contains at least one product story. Enabling stories declare deliverableClass as contract, acceptance, or housekeeping; omitted legacy values mean product. A ready product story cannot be starved by no-code work, and a contract/acceptance PASS dispatches ready product code in the same cycle.
Contract-only correction has one bounded no-code budget. Exact unchanged candidate/test-environment identity permits evidence reuse; report freshness never requires a redundant acceptance run.
Resolved protected-merge authority immediately owes both merge issuance and a readback observer. HOLD still permits truthful publication/reconciliation but blocks spawning, implementation, and merge.
Trusted coordinator CWD syntax is an absolute path or ~/…, stored canonically. Other relative syntax remains refused. v3.4.35 strict lane admission stays valid during the v3.4.36 rollout.
coordinator-reconcile.py apply uses an explicitly configured authenticated Craft RPC rename and validates its receipt; it never rewrites a live session file. Rotation pressure from a direct owner request or complexity threshold outranks routine no-rotation ticks; housekeeping remains quota-bounded.

v3.4.35 admission and owner-memory controls

Worker and auditor dispatch is strict two-phase: reserve immutable lane identity before spawning, then confirm the returned live manifest before lease creation. Legacy lease records remain readable; new dispatches use lane-admission.py.
Coordinator transfer accepts only the exact live successor identity reserved by begin-transfer; session-ID-only handoffs fail closed.
Owner gates canonicalize an exact decision/preference and reuse a matching durable resolved or open record rather than opening duplicate gates.
Owner-facing reporting is configured as pull-only through reporting-policy.py. Runtime outbound-message interception is unavailable, so transcript checking is explicitly best-effort and never proves silence.

3. Current default settings

Models and permissions

Coordinator: chatgpt-plus / pi/gpt-5.6-sol / medium / allow-all.
Worker and auditor: chatgpt-plus / pi/gpt-5.6-terra / medium / allow-all.
Claude/non-Codex: time-bounded fallback only, with reason and default 60-minute TTL.
A live session connection cannot be changed; rotate to a new session instead.

Timing

Worker evidence healthy: up to 15 minutes.
Worker suspect: 15–30 minutes.
Worker stalled: over 30 minutes without session/PID/log/CPU progress.
Active worker heartbeat: every 10–15 minutes and after every meaningful phase.
Coordinator lease default: 3,600 seconds.
Fallback provider TTL: 3,600 seconds.
Watchdog interval: 300 seconds.
Observable-job threshold: commands expected to exceed 10 minutes.

Rotation recommendations

Rotate sequentially when any meaningful threshold is reached:

request-buffer/context error;
repeated provider/SIGTERM failure;
about 200k tokens;
about 500 messages;
3 active lanes;
8 unresolved gates;
repeated recovery complexity.

These are recommendations with cooldown/hysteresis, not authority to discard work.

4. Package layout

README.md
LICENSE
NOTICE
SECURITY.md
SUPPORT.md
CONTRIBUTING.md
CODE_OF_CONDUCT.md
GOVERNANCE.md
ROADMAP.md
CHANGELOG.md
.editorconfig
install.sh
manifest.sha256
scripts/
  orchestration-common.py
  coordinator-registry.py
  coordinator-reconcile.py
  owner-gate.py
  recovery-ledger.py
  completion-certificate.py
  recovery-incident.py
  worker-lease.py
  observable-job.py
  worker-watchdog.py
  post-archive-reaper.py
  worktree-gc.py
  controller-harness.py
  recovery-admission.py
  recovery-admission-cron.sh
  scan-reapable-workers.py
  watchdog-cron.sh
  coordinator-kickoff.md
skills/
  coordinator-lifecycle-protocol/SKILL.md
  worker-completion-protocol/SKILL.md
  self-healing-controller/SKILL.md
config/
  labels.config.json
  self-healing.automations.template.json
  launchd.watchdog.template.plist
  launchd.admission.template.plist
tests/
  test_worker_reliability.py
  test_orchestration_v320.py
  test_self_healing_v311.py
  test_delivery_mode_v320.py
  test_controller_harness_v321.py
  test_recovery_admission_v322.py
  test_coordinator_v330.py
docs/
  PROTOCOL-v3.1.md
  PROTOCOL-v3.3.md
  SELF-HEALING-v3.1.1.md
  SELF-HEALING-v3.2.1.md
  SELF-HEALING-v3.2.2.md
  DELIVERY-MODE-v3.2.0.md
  CURRENT-DEFAULTS.md
tools/
  generate-manifest.sh
.github/
  CODEOWNERS
  dependabot.yml
  pull_request_template.md
  ISSUE_TEMPLATE/
    bug.yml
    feature.yml
    protocol-change.yml
    config.yml
  workflows/
    test.yml

manifest.sha256 authenticates the distributable protocol payload (README, license/notice, installer, scripts, config, skills, tests, docs, and tools). Repository-only community/CI metadata is validated separately so Dependabot can update workflows without rewriting package hashes.

5. Requirements

macOS with Craft Agents installed.
Python 3 with fcntl support. Current machine uses /opt/homebrew/bin/python3.
git, lsof, ps, zsh, launchctl.
A configured Craft workspace, normally ~/.craft-agent/workspaces/general.
A model connection exposing coordinator/worker models. Connection slugs may differ on another machine.
Git remotes/auth configured separately. This package contains no GitHub token.

The scripts are macOS-oriented because they use launchd, lsof, fcntl, and process command inspection.

6. Safe installation

Dry run first

./install.sh

The installer defaults to dry-run. It prints planned destinations and performs no mutation.

Apply

./install.sh --apply

The installer:

creates/restores mode-0600 ~/.craft-agent/runtime/self-healing.disabled before copying any v3.4.41 payload;
backs up overwritten protocol files into a timestamped directory;
installs scripts under ~/.craft-agent/scripts;
installs canonical skills under the selected workspace;
creates remaining runtime/log directories with owner-only permissions;
renders a user-specific launchd plist from the portable template;
does not overwrite labels automatically;
runs syntax checks and tests;
prints, but does not silently run, exact runtime verification and final launchctl commands.

Before admission activation, keep the kill switch present and run:

CRAFT_SERVER_URL=<trusted-url> CRAFT_RPC_CLI=<absolute-cli> \
  ~/.craft-agent/scripts/recovery-admission.py verify-runtime \
  --workspace-id <workspace-id> \
  --expected-runtime-version 0.11.4-admission.87951ae \
  --expected-runtime-commit 87951ae640df64d00534a54dce9b5e8b5922d27c

Require verified: true and the exact reviewed runtime identity. The installed LaunchAgent defaults CRAFT_ADMISSION_CLI_TIMEOUT_SECONDS=110 and CRAFT_ADMISSION_SUPERVISOR_TIMEOUT_SECONDS=120; persistent-controller.json may set integer cliTimeoutSeconds / supervisorTimeoutSeconds overrides within 20..300 / 30..330 seconds, with supervisor at least five seconds larger. Protocol-first activation, launchd admission activation before verification, and kill-switch removal before canary approval are prohibited.

Labels

config/labels.config.json is a minimal orchestration-only label set, not a blind merge patch. Back up the target labels configuration and merge only missing IDs/value types. Validate through the Craft configuration validator before relying on it.

Launchd

After reviewing the rendered plist:

mkdir -p ~/Library/LaunchAgents
cp ~/.craft-agent/scripts/com.craft-protocol.worker-watchdog.plist \
  ~/Library/LaunchAgents/com.craft-protocol.worker-watchdog.plist

launchctl bootout gui/$(id -u) \
  ~/Library/LaunchAgents/com.craft-protocol.worker-watchdog.plist 2>/dev/null || true
launchctl bootstrap gui/$(id -u) \
  ~/Library/LaunchAgents/com.craft-protocol.worker-watchdog.plist
launchctl kickstart -k gui/$(id -u)/com.craft-protocol.worker-watchdog
launchctl print gui/$(id -u)/com.craft-protocol.worker-watchdog

Expected: interval 300 seconds and last exit code 0 after a completed run. state = not running between interval executions is normal.

7. Runtime layout

~/.craft-agent/runtime/
  coordinators/<project>.json
  coordinators.lock
  owner-gates/<project>/<gate>.json
  owner-gates.lock
  recovery-ledger/<project>.json
  recovery-ledger.lock
  completion-certificates/<project>/*.json
  worker-leases/<session>.json
  worker-leases.lock
  worker-jobs/<session>.json
  heavy-job.lock
  heavy-job-owner.json

Runtime files are execution state, not a substitute for Git. Mutations are atomic and lock-protected. Do not hand-edit them while the tools are active.

Environment overrides used by tests/portable deployments:

CRAFT_WORKSPACE
CRAFT_SESSIONS
CRAFT_RUNTIME
CRAFT_PID_DIR
CRAFT_LEASE_HEALTHY_SECONDS
CRAFT_LEASE_STALLED_SECONDS
CRAFT_COORDINATOR_CONNECTION
CRAFT_COORDINATOR_MODEL
CRAFT_COORDINATOR_TTL_SECONDS
CRAFT_FALLBACK_TTL_SECONDS

8. Canonical labels

Coordinator

coordinators
agent-role::coordinator
project::<project-slug>
protocol-version::3
parent-session::<predecessor-id> or equivalent predecessor metadata

Canonical new name:

[<project>] Coordinator v<protocol-version>

Worker/auditor

agent-role::worker | agent-role::auditor
parent-session::<coordinator-id>
work-unit::<issue-or-unit>
attempt::<integer>
github-issue::<full-url>
protocol-version::3

attempt must be numeric. github-issue and github-project are link-valued labels. Historical unknown labels may exist; do not assume they can be written unless configured.

9. Coordinator lifecycle

Initialization

Read the coordinator skill.
Call session info and verify native project binding/model/permissions.
Claim ownership.
Reconstruct recovery state.
Read GitHub milestone/issues/dependencies/Project fields.
Reconcile leases and existing children.
Check gates before dispatch.

~/.craft-agent/scripts/coordinator-registry.py claim \
  --project <slug> --session <coordinator-id> --project-id <native-project-id>

~/.craft-agent/scripts/recovery-ledger.py reconstruct --project <slug>
~/.craft-agent/scripts/worker-lease.py reconcile --apply
~/.craft-agent/scripts/worker-lease.py report
~/.craft-agent/scripts/owner-gate.py check --project <slug> --action spawn

A claim fails with split-brain-refused if another live authoritative owner exists.

Dense waves

Select the largest dependency-valid wave, not simply the maximum number of sessions. If only one issue is unblocked and the second worker would duplicate it, run one worker. Auditors wait for immutable candidate handoff.

Ownership renewal

~/.craft-agent/scripts/coordinator-registry.py renew \
  --project <slug> --session <coordinator-id>

Renew on active turns and before/after major transitions.

10. Unique worktree attempt

Pattern:

<repo>/.worktrees/<work-unit>-a<attempt>-<unique-nonce>

Sequence:

verify no existing session/harness references the path;
create a fresh branch/worktree;
spawn worker/auditor into that exact cwd;
receive real session ID;
create lease immediately;
send the frozen task package;
require the worker skill and startup heartbeat.

~/.craft-agent/scripts/worker-lease.py create \
  --session <child-id> \
  --parent <coordinator-id> \
  --work-unit <unit> \
  --attempt <N> \
  --worktree <absolute-unique-path> \
  --phase task-assigned

Never reuse a predecessor’s cwd, even when the prior session failed before changing code.

11. Leases and heartbeats

Worker startup/progress

~/.craft-agent/scripts/worker-lease.py heartbeat \
  --session <id> --state running --phase task-started \
  --evidence "task package acknowledged"

~/.craft-agent/scripts/worker-lease.py heartbeat \
  --session <id> --phase tests-green \
  --evidence "SHA abc123; 42/42 tests passed"

Valid evidence:

new SHA;
exact test result;
artifact checksum/path;
log size/mtime progression;
child PID with increasing CPU/output;
completed phase.

“Still working” is not evidence.

Terminal lease

~/.craft-agent/scripts/worker-lease.py finish \
  --session <id> --preservation pushed

Then set the session status to needs-review and stop permanently.

Reconciliation

~/.craft-agent/scripts/worker-lease.py reconcile --apply
~/.craft-agent/scripts/worker-lease.py report

Reconciliation creates missing leases for live worker/auditor manifests, classifies progression, removes runtime state for archived/absent sessions, and reports cwd collisions.

12. Observable jobs and global heavy lane

Use for commands expected to exceed 10 minutes:

~/.craft-agent/scripts/observable-job.py start \
  --session <id> \
  --cwd <worktree> \
  --log <absolute-log> \
  -- <command> <args...>

Heavy build:

~/.craft-agent/scripts/observable-job.py start \
  --session <id> --cwd <worktree> --log <log> --heavy -- <command>

Status/acknowledgement:

~/.craft-agent/scripts/observable-job.py status --session <id>
~/.craft-agent/scripts/observable-job.py ack --session <id>

Exit 75 means the global heavy lane is busy. Do not bypass it. A disappeared PID without a successful receipt is failure, not success.

13. Independent audit

Coordinator independently inspects candidate diff/tests.
Create a fresh detached audit worktree at the exact candidate SHA.
Spawn Terra/medium/allow-all auditor.
Auditor is read-only by mandate.
Auditor attempts to falsify the candidate with adversarial tests.
PASS must bind exact SHA and evidence.
FAIL creates root-cause feedback and a fresh implementation attempt.
After two FAILs, freeze a root-cause/spec review before attempt 3.

The auditor never edits the implementation branch and never “confirms the favored approach.”

14. Structured worker handoff

Required report:

STATUS: done | needs-rework | blocked
WORK-UNIT: <id>
ATTEMPT: <N>
BRANCH: <branch>   PR: <url or none>
DONE:
- verified facts
NOT DONE / OPEN:
- remaining work and reason
FILES: key files
VERIFY: commands and exact results
PRESERVATION: clean + pushed at <SHA/ref>
LEASE/JOB: last phase; observable job exit if any

Before reporting:

git status --porcelain
git branch --show-current
git push -u origin HEAD

No terminal handoff until work is clean and remotely preserved.

15. Owner gates and HOLD

Create a decision gate

~/.craft-agent/scripts/owner-gate.py create \
  --project <slug> \
  --gate <id> \
  --work-unit <unit> \
  --question "Decision question" \
  --choices A,B,C \
  --owner-only-category human-product-judgment-action \
  --scope implement \
  --safe-default HOLD

Scopes: project, work-unit, spawn, implement, merge, close.

Every new non-HOLD gate requires a machine-validated owner-only category. Technical PASS-to-next-stage transitions, first CI/audit failure, reversible correction, merge/readback, and ordinary already-authorized deploy are not valid gate categories and must continue autonomously. Existing stored gates remain readable for compatibility.

An unscoped non-project decision does not block unrelated explicit work units. Project-wide blocking must use scope=project or the project HOLD gate.

Check before action

~/.craft-agent/scripts/owner-gate.py check \
  --project <slug> --work-unit <unit> \
  --action <spawn|implement|merge|close>

Resolve

~/.craft-agent/scripts/owner-gate.py resolve \
  --project <slug> --gate <id> --choice <allowed-choice> \
  --authority direct-owner --evidence "exact direct owner message"

Only direct-owner authority is accepted. If the selected outcome is HOLD, keep the machine gate open/blocking or represent HOLD as a dedicated state; do not accidentally “resolve” it into permission.

Project HOLD/RESUME

~/.craft-agent/scripts/owner-gate.py hold \
  --project <slug> --reason "owner HOLD" --evidence "direct message"

~/.craft-agent/scripts/coordinator-registry.py hold \
  --project <slug> --session <coordinator-id> --reason "owner HOLD"

Resume requires exact RESUME in both layers:

~/.craft-agent/scripts/owner-gate.py resolve \
  --project <slug> --gate project-hold --choice RESUME \
  --authority direct-owner --evidence "direct RESUME"

~/.craft-agent/scripts/coordinator-registry.py resume \
  --project <slug> --authorization RESUME

Owner inbox

~/.craft-agent/scripts/owner-gate.py inbox

This intentionally excludes routine progress.

16. Recovery ledger and cold takeover

~/.craft-agent/scripts/recovery-ledger.py snapshot --project <slug>
~/.craft-agent/scripts/recovery-ledger.py reconstruct --project <slug>
~/.craft-agent/scripts/recovery-ledger.py diff --project <slug>

The ledger synthesizes:

authoritative coordinator/generation;
active child sessions, roles, work-units, attempts;
leases, phases, preservation, worktrees;
observable jobs;
open/resolved gates;
completion certificates;
unknowns.

Scope matching uses parent ID, explicit project::<slug>, or repository/worktree containment. Native Craft projectId alone is insufficient because distinct scopes may share it.

A successor must reconcile handoff child IDs against observed manifests/leases/jobs/worktrees. Unknown stays unknown.

17. Two-phase coordinator rotation

Start transfer

~/.craft-agent/scripts/coordinator-registry.py begin-transfer \
  --project <slug> \
  --session <old-id> \
  --successor <new-id> \
  --reason "context/provider/complexity rotation"

Old coordinator stops spawning new work and preserves a recovery snapshot.

Accept

~/.craft-agent/scripts/coordinator-registry.py accept-transfer \
  --project <slug> \
  --session <new-id> \
  --expected-generation <N>

The new owner receives generation N+1. A second pending successor is refused.

Archive predecessor

Only after:

registry confirms new authority;
repo/worktrees are clean and pushed;
child inventory is reconciled;
no irreversible operation is in flight.

Then archive the predecessor through the Craft session tool. Do not edit live session JSONL.

Provider fallback

Record fallback reason/TTL. When Codex is healthy, perform one preservation-first transfer back to Codex. Avoid automatic respawn loops; repeated provider failure escalates to the owner-facing infrastructure session.

18. Completion certificates

Create only after merge/readback evidence exists:

~/.craft-agent/scripts/completion-certificate.py create \
  --project <slug> --work-unit <unit> \
  --candidate-sha <sha> --audited-sha <same-sha> \
  --auditor <auditor-session> --verdict PASS \
  --ci-run <immutable-run-id> --ci-success \
  --merge-sha <merge-sha> --head-unchanged \
  --readback-run <distinct-main-run-id> --readback-success

Validate:

~/.craft-agent/scripts/completion-certificate.py validate --file <certificate.json>
~/.craft-agent/scripts/completion-certificate.py scan

Rejected evidence includes:

candidate/audited SHA mismatch;
non-PASS verdict;
missing/duplicate CI IDs;
reusing PR CI as merged-main readback;
symbolic values such as green instead of immutable IDs;
failed/unknown CI;
unresolved gates;
unproved unchanged head;
missing required closure evidence.

19. Archive-first safe cleanup

Pre-archive inspection

~/.craft-agent/scripts/scan-reapable-workers.py --parent <coordinator-id>

Verify:

terminal structured report;
needs-review;
clean worktree;
exact HEAD on remote;
no unreported job;
unique cwd.

Archive session first

Use Craft’s archive_session tool. If it refuses because the session is still processing, do not guess/kill a PID. Preserve the exact branch/HEAD and retry after the turn exits. Escalate to infrastructure if needed.

Reap second, GC third

~/.craft-agent/scripts/post-archive-reaper.py \
  --session <id> --apply --max-groups 5

# Dry-run is mandatory first; this writes runtime/worktree-gc/latest.json.
~/.craft-agent/scripts/worktree-gc.py
# Review the durable JSON, then apply a bounded batch only:
~/.craft-agent/scripts/worktree-gc.py --apply --max-remove 5

~/.craft-agent/scripts/worker-lease.py reconcile --apply
~/.craft-agent/scripts/worker-lease.py report

The reaper refuses:

non-archived sessions;
coordinator roles;
dirty worktrees;
worker HEAD absent from remote branch/default;
cwd shared with a live session;
non-harness PID;
Craft Agents app process.

For archived auditors, a clean lane is sufficient because they are read-only. The reaper removes only exact harness PIDs mapped by cwd and only after all guards pass. Never guess or automatically kill an unproved PID.

worktree-gc.py runs only after archival and guarded process reaping. It protects the root checkout, every non-archived session CWD, authoritative activeChildren, paths shared by sessions, every observed process CWD, dirty worktrees, and detached HEADs unreachable from a retained ref. Apply mode revalidates each candidate and uses only git worktree remove; it never deletes branches or session history. worker-lease.py report exposes six debt classes: preservation-proven-archivable, terminal-unknown, orphaned-dead, stale-coordinator, archived-active-child, and stale-clean-worktree. archivableBacklog: 0 is compatible output, not proof of cleanliness; require lifecycleDebt.clean: true.

20. Deterministic watchdog

Manual dry run:

~/.craft-agent/scripts/worker-watchdog.py

Apply safe reconciliation:

~/.craft-agent/scripts/worker-watchdog.py --apply

Reports:

archive reaper decisions;
lease state/cwd collisions;
authoritative ownership and lease health;
provider/fallback TTL drift;
metadata/project-binding drift;
owner decision inbox;
certificate health.

The watchdog does not call an LLM, create sessions, decide owner questions, rewrite live JSONL, or kill unsafe processes.

Logs:

~/.craft-agent/logs/worker-watchdog.log
~/.craft-agent/logs/worker-watchdog.stdout.log
~/.craft-agent/logs/worker-watchdog.stderr.log

21. CI and merged-main evidence

Bind checks to the exact candidate SHA.
Use immutable workflow run/job IDs.
Deduplicate repeated polling observations.
If several distinct workflows are required, every named unique run must reach terminal success.
A green PR head is not merged-main verification.
After merge, require distinct main run IDs.
Verify that the exact audited head was merged unchanged.
Do not close issue/Project fields from a relayed coordinator claim.

22. Status semantics

Available status IDs on the current workspace:

backlog
todo
needs-review
done
cancelled

Agents must not close their own tasks into done/cancelled; final closure is the owner’s board decision. Coordinators/workers hand work to needs-review. Runtime activity, ownership state, and workflow status are separate concepts.

Ownership states:

authoritative
rotating
hold
superseded
needs-owner

todo does not prove authority. The coordinator registry does.

23. Troubleshooting

`Pi subprocess exited unexpectedly (signal SIGTERM)`

Preserve/reconstruct state.
One retry for a bounded turn.
After repetition, spawn one fresh project-bound coordinator on an active Codex connection.
Perform two-phase ownership transfer.
Archive failed coordinator only after clean/pushed proof.

`Connection Error: Could not reach the AI service`

Retry once.
Preserve + handoff.
Fresh session; do not change connection in-place.

`Session ... is currently processing a turn`

Do not infer a background job.
Inspect background-task registry, lease, session status, git preservation, and unique cwd.
Wait/retry archive.
Never guess PID by process tree.

`global heavyweight lane busy` / exit 75

Inspect heavy-job-owner.json and the owning job receipt/log.
Queue work; never bypass the global lock.

Coordinator registry says split brain

Inspect the current live owner and generation.
Do not overwrite the record.
Begin a formal transfer or prove the old owner absent/archived with preservation evidence.

Recovery ledger mixes scopes

Ensure explicit project::<slug> labels and distinct repository cwd.
Never use shared native Craft projectId alone.

Worker stuck on unsolicited plan approval

Send one explicit execute instruction.
If it cannot resume, preserve, archive/reap, and replace in a fresh worktree with DO NOT SubmitPlan.

Old coordinator display name

Name drift is advisory unless a safe supported rename API exists.
Do not rewrite live session.jsonl.
Enforce canonical names for newly spawned sessions.

24. Security and privacy

Never put credentials, MFA, passcodes, provider tokens, customer data, or secrets in prompts/logs/gates.
Owner enters credentials only in secure browser/auth UI.
Use least-privilege, disposable test accounts where possible.
Production/provider sends require explicit gate scope and exact evidence.
Historical unknown outcomes are immutable; never retry them blindly.
Generated/private evidence may itself require owner authorization.
Preserve retained evidence append-only; no rm, mv, or overwrite recovery unless explicitly authorized and contractually safe.
Runtime JSON files use mode 0600.

25. Validation and tests

cd <bundle>/tests
python3 -m unittest -v \
  test_worker_reliability.py \
  test_orchestration_v320.py \
  test_self_healing_v311.py \
  test_delivery_mode_v320.py \
  test_controller_harness_v321.py

Current packaged suite covers:

missing lease backfill and archive removal;
coordinator exclusion from worker leases;
terminal/error/stall classification;
observable job receipts;
heavy-job serialization;
app/non-harness guard;
split-brain refusal;
two-phase transfer/interruption;
fallback TTL;
archived owner detection;
HOLD/RESUME authority;
narrow gate targeting;
exact certificate validation;
changed head/reused CI/unresolved-gate rejection;
cold takeover and live-lane adoption;
metadata/complexity warnings;
shared-native-project scope isolation.

Validate packaged hashes:

./tools/generate-manifest.sh --check

26. Operational checklists

Before spawn

GitHub-sourced work-unit and dependencies.
Owner gate check.
No matching live attempt.
Unique worktree/nonced path.
Concurrency slot available.
Heavy lane checked if needed.
Correct repo/issue URL label.

Before worker acceptance

Structured report.
Exact candidate SHA.
Clean/pushed proof.
Coordinator verification.
Fresh independent audit.
Exact required tests.
No unresolved substantive gate.

Before merge/close

Unchanged audited SHA.
Independent PASS.
Required immutable PR CI IDs green.
Exact authority rule.
Merge SHA.
Distinct merged-main readback IDs.
Closure evidence.
Valid completion certificate.

Before release closure

Push the exact main commit and annotated tag; the tag must peel to that exact remote main SHA.
Create the GitHub Release as a draft and upload manifest.sha256 and install.sh with their required SHA-256 asset digests.
Install and adopt the fleet.
Write exactly one json-fenced receipt in the authenticated Release body, then publish it as the non-draft Latest Release:
```
{"schemaVersion":1,"version":"3.4.41","tag":"v3.4.41","commit":"<peeled-tag-commit-sha>","state":"adopted","ownerFacingOrchestratorSessionId":"<owner-facing-session-id>","adoptedAt":"<nonempty-timestamp>","adoptions":[{"project":"<project>","coordinatorSessionId":"<coordinator-session-id>"}]}
```
adoptions is a nonempty canonical roster: each entry has only project and coordinatorSessionId, projects and coordinators are unique, and entries are ascending by (project, coordinatorSessionId). The receipt is deliberately Release metadata, not a tracked file at the tag: committing a file that names its own resulting commit would change that commit.
Run release-closure.py verify. The verifier requires the receipt only after the Release is published, non-draft, and Latest; it rejects missing, malformed, duplicate, stale, or noncanonical receipts.

Before archive/reap

Work preserved.
No live/unknown job.
Archive succeeds first.
Unique cwd.
Reaper app/harness guard passes.
Lease/job/PID runtime state removed.

Before coordinator rotation

Recovery snapshot.
Active child inventory and exact SHAs.
Open gates/decisions.
Observable jobs/heavy owner.
Begin transfer.
Successor accepts expected generation.
Verify provider/project binding.
Archive predecessor after preservation.

27. What remains intentionally manual

Human owner decisions.
Starting/closing tasks on the board.
Secure credential/MFA entry.
Native project binding when the spawn API cannot inherit it from the correct parent.
Session rename when no safe external rename endpoint exists.
Determining legal/compliance retention policy.
Physical-device capture.
Any cleanup whose preservation proof is incomplete.

Automation is strongest when it refuses ambiguous actions.

28. Recommended adoption sequence for another workspace

Read this guide completely.
Verify package hashes.
Run installer dry-run.
Back up workspace scripts/skills/labels.
Install scripts and skills.
Merge/validate labels.
Run syntax/tests.
Start watchdog manually in dry-run.
Migrate one low-risk project into the coordinator registry.
Test worker lease + unique worktree + terminal handoff.
Test independent audit.
Test HOLD/RESUME on a nonproduction fixture.
Test two-phase coordinator rotation.
Enable launchd.
Canary completion certificate on one already-proved merge.
Migrate remaining projects sequentially.

Do not migrate every active project at once.

29. Quick command card

# Ownership
coordinator-registry.py claim --project P --session C --project-id PID
coordinator-registry.py renew --project P --session C
coordinator-registry.py validate

# Recovery
recovery-ledger.py reconstruct --project P
recovery-ledger.py snapshot --project P

# Gates
owner-gate.py inbox
owner-gate.py check --project P --work-unit U --action implement

# Worker lease
worker-lease.py create --session W --parent C --work-unit U --attempt 1 --worktree /abs/path
worker-lease.py heartbeat --session W --phase tests --evidence "42/42"
worker-lease.py finish --session W --preservation pushed

# Long job
observable-job.py start --session W --cwd /abs/path --log /abs/log --heavy -- command args
observable-job.py status --session W

# Rotation
coordinator-registry.py begin-transfer --project P --session OLD --successor NEW --reason rotation
coordinator-registry.py accept-transfer --project P --session NEW --expected-generation N

# Certificate
completion-certificate.py validate --file cert.json
completion-certificate.py scan

# Cleanup/watchdog
post-archive-reaper.py --session W --apply --max-groups 5
worktree-gc.py                         # mandatory dry-run
worktree-gc.py --apply --max-remove 5 # only after JSON review
worker-lease.py reconcile --apply
worker-lease.py report
worker-watchdog.py --apply

30. Final principle

The protocol is not designed to maximize the number of agents. It is designed to maximize verified progress without losing work or authority boundaries:

source task
→ unique attempt
→ observable evidence
→ preserved handoff
→ independent audit
→ exact CI
→ owner gate
→ completion certificate
→ safe archive/reap

When evidence is incomplete, stop and preserve. When authority is unclear, ask the owner. When a session fails, rotate ownership—not the truth.

Name		Name	Last commit message	Last commit date
.github		.github
config		config
docs		docs
scripts		scripts
skills		skills
tests		tests
tools		tools
v4		v4
.editorconfig		.editorconfig
.gitignore		.gitignore
CHANGELOG.md		CHANGELOG.md
CODE_OF_CONDUCT.md		CODE_OF_CONDUCT.md
CONTRIBUTING.md		CONTRIBUTING.md
GOVERNANCE.md		GOVERNANCE.md
LICENSE		LICENSE
NOTICE		NOTICE
README.md		README.md
ROADMAP.md		ROADMAP.md
SECURITY.md		SECURITY.md
SUPPORT.md		SUPPORT.md
WORKFLOW.md		WORKFLOW.md
install.sh		install.sh
manifest.sha256		manifest.sha256

Latest commit

History

Folders and files

Repository files navigation