Pause migrate for a release approval and let rollback run without one (S-008-13, S-008-14)

README.md

@@ -224,14 +224,19 @@ and drift audit to the live dashboard, is in
 ```console
 uv run cloudfall import render render.yaml --application acme --server h1
 uv run cloudfall migrate --build web=main          # show the plan
-uv run cloudfall migrate --build web=main --yes    # run it
+uv run cloudfall migrate --build web=main --yes    # run it; pauses at deploy:web
+uv run cloudfall release approve web --release <release-id>   # a person, at a terminal
+uv run cloudfall migrate --yes                     # resume: deploy, routes, TLS
 ```
 
 The importer never guesses silently: `IMPORT-REPORT.md` records what was not
 imported, every assumption and every step required before cutover. The
 migration runs baseline, services, builds, health-gated deploys, routes, a
 DNS-verification pause, TLS and a final evidence pass, and resumes exactly
-where it stopped. Databases move with `cloudfall data copy`, which verifies
+where it stopped. An imported application declares `approval: manual`, so
+the run pauses before its deploy until a person approves the built release
+with `cloudfall release approve`; the resumed run deploys that release
+without building again. Databases move with `cloudfall data copy`, which verifies
 per-table row counts and writes a receipt. See the
 [Render migration guide](docs/render-migration-guide.md) and the
 [Neon migration guide](docs/neon-migration-guide.md).

changelog.d/165-migrate-approval-pause.md

@@ -0,0 +1,3 @@
+### Changed
+
+- `migrate --yes` checks a `deploy:<component>` step's approval as `deploy --yes` does: for an application that declares `approval: manual` and no approval that counts, it pauses at that step with `PARTIAL_FAILURE`, `context.code: deployment_approval_missing`, the built `release` and the `cloudfall release approve` `command`, and once a person approved it, `migrate --yes` resumes there with the release the completed `build:<component>` step recorded, without building again and without needing `--build` again. The deploy step's detail gains `approval` (the record it went on), `migrate` gains `--approvals` (default `decisions/releases`, bound by `cloudfall mcp serve`) and the `RECORD_INVALID` exit for an approvals file that isn't a valid `ReleaseApproval`. `rollback` needs no approval: its plan gains `approval`, `{"policy": "manual", "required": false, "reason": ...}` for a `manual` application and `{"policy": "automatic"}` otherwise (#165)

docs/render-migration-guide.md

@@ -127,14 +127,28 @@ and validation rejects them if they do.
 ```console
 uv run cloudfall migrate --build acme-api=main
 uv run cloudfall migrate --build acme-api=main --yes
+uv run cloudfall release approve acme-api --release <release-id>
+uv run cloudfall migrate --yes
 ```
 
 Without `--yes` the command shows the plan. With it, the plan executes step
 by step: server baseline and firewall, infrastructure services, artifact
 builds, health-gated deployments, HTTP routes, a DNS-verification pause at
 the cutover moment, TLS issuance, and a final evidence pass requiring a
 compliant audit and healthy routes. Progress persists after every step, so
-a failure or the DNS pause resumes exactly where it stopped.
+a failure or a pause resumes exactly where it stopped.
+
+The importer writes `approval: manual` for every application, so the run
+pauses at `deploy:acme-api` (exit 3, `PARTIAL_FAILURE`, with
+`context.code: deployment_approval_missing`) once the release is built. The
+answer names the built release and the exact `cloudfall release approve`
+command. A person runs it at a terminal: it shows the git commit, the archive
+SHA-256 and the servers, and records the approval under
+`decisions/releases/` once the release id is typed back. `migrate --yes` then
+resumes at the deploy step with the release the build step recorded,
+without building again. When another gate, such as GitHub's required
+reviewers, is the real one, switch the application to `approval: automatic`
+and the run deploys without stopping.
 
 ## Step 6: migrate the data
 

docs/specs/008-enforce-manual-approval.md

@@ -1,6 +1,6 @@
 # S-008: Enforce manual approval
 
-status: approved
+status: built
 serves: G-2
 
 ## Problem
@@ -186,3 +186,22 @@ Touches `sdk/src/cloudfall/app.py` (`deploy`, `rollback`, `migrate`, new `releas
 - cloudfall-dev/cloudfall#165: S-008-13, S-008-14
 
 ## Verification
+
+Every criterion ran through the CLI (or `cloudfall mcp serve`'s tool call) against a copy of the example project, with an engine whose playbooks stand in for the real ones: they run `ansible-playbook` on localhost and leave a marker and a release receipt, so no host was contacted. The S-008-14 artifact was built for real by `cloudfall-engine artifact build` from a local git repository. The terminal attestation ran with a stream that reports itself a terminal; no person typed at a real one, and no deploy reached a real server.
+
+- S-008-1: `test_s008_1_a_manual_application_without_approval_is_refused`, exit 4 `PRECONDITION` with `deployment_approval_missing`, the message named `crm`, `crm-backend`, the release and the command, `context.command` was the exact `release approve` command, and no marker, inventory or receipt was written
+- S-008-2: `test_s008_2_a_person_approves_a_release_by_typing_its_id` and `test_s008_2_the_record_id_is_an_approval_time_and_16_hex`, exit 0 and one file under `decisions/releases/` that validated against `release-approval.schema.json` with every named field, `expiresAt` 24 hours after `approvedAt` and `via: terminal`
+- S-008-3: `test_s008_3_an_approved_release_deploys_and_names_its_approval`, the deploy playbook ran and the result's `approval` carried the record's `path`, `approver` and `approved_at`
+- S-008-4: `test_s008_4_off_a_terminal_nothing_is_approved` and `test_s008_4_a_mistyped_release_id_approves_nothing`, exit 4 `PERSON_REQUIRED` and `ATTESTATION_MISMATCH` with and without `--yes`, and the approvals folder stayed empty
+- S-008-5: `test_s008_5_the_deploy_tool_refuses_an_unapproved_manual_release` and `test_s008_5_the_project_instructions_name_release_approve`, no `release_approve` tool was listed and the `deploy` tool with `yes: true` returned `PRECONDITION` with `deployment_approval_missing`
+- S-008-6: `test_s008_6_a_record_for_another_archive_does_not_count`, `test_s008_6_a_record_for_other_servers_does_not_count`, `test_s008_6_an_expired_record_does_not_count` and `test_s008_6_the_servers_are_compared_as_a_set`, each refusal listed the record with `digest_mismatch`, `servers_changed` or `expired`, and servers in another order still counted
+- S-008-7: `test_s008_7_a_record_for_another_release_or_component_does_not_count`, refused with `deployment_approval_missing` and neither record listed
+- S-008-8: `test_s008_8_the_plan_shows_the_approval_before_and_after_it_is_given` and `test_s008_8_the_latest_counting_approval_is_shown`, exit 0 with no playbook run, `status: missing` with `command` before the approval and `status: approved` with `path`, `approver` and `expires_at` after it, the latest of two counting records shown
+- S-008-9: `test_s008_9_an_automatic_application_needs_no_record`, plan and `--yes` both exit 0 with `approval: {"policy": "automatic"}` beside an approvals file that isn't JSON, and the deploy ran
+- S-008-10: `test_s008_10_a_broken_record_stops_a_manual_deploy` and `test_s008_10_deploy_declares_record_invalid`, exit 86 `RECORD_INVALID` naming the file for invalid JSON, a schema mismatch and a non-object, plan and `--yes` alike, and `deploy --schema` declares the exit
+- S-008-11: `test_s008_11_an_automatic_application_needs_no_approval`, `test_s008_11_a_missing_artifact_is_refused_before_asking` and `test_s008_11_a_tampered_artifact_is_refused_before_asking`, `PRECONDITION` with `release_approval_not_required`, `lifecycle_artifact_missing` and `lifecycle_artifact_digest_mismatch`, no prompt and no file
+- S-008-12: `test_s008_12_a_validity_out_of_range_is_a_usage_error` and `test_s008_12_a_validity_of_two_hours_expires_two_hours_later`, `--valid-for 0` and `169` were usage errors naming `--valid-for`, and `2` wrote `expiresAt` two hours after `approvedAt`
+- S-008-13: `test_s008_13_rollback_needs_no_approval`, for a `manual` application the plan showed `approval` with `policy: manual`, `required: false` and a reason, and `rollback --yes` ran the rollback playbook with no approvals folder at all; an `automatic` application's plan showed `{"policy": "automatic"}`
+- S-008-14: `test_s008_14_migrate_pauses_for_an_approval_and_resumes_without_a_rebuild`, `migrate --build crm-backend=main --yes` exited 3 `PARTIAL_FAILURE` at `deploy:crm-backend` with `deployment_approval_missing`, the release the completed build step recorded and the `release approve` command, no deploy playbook ran; after `release approve`, with the source repository deleted so a second build could not succeed, `migrate --yes` completed `deploy:crm-backend` with that release and the record as its `approval`, and the artifacts folder held the same files. `test_s008_14_a_broken_approval_record_stops_the_deploy_step` (exit 86 `RECORD_INVALID`) and `test_s008_14_an_automatic_application_migrates_without_a_record` cover the step's other answers
+- S-008-15: `test_s008_15_a_failed_health_gate_keeps_the_approval_for_a_retry`, the failed deploy exited `ENGINE_STEP_FAILED`, the record's bytes were unchanged and the retry deployed on the same record
+- S-008-16: `test_s008_16_the_changelog_and_ci_guide_say_deploy_enforces_manual`, the fragment's `### Breaking` entry names `deploy --yes`, `migrate --yes`, `approval: manual`, `cloudfall release approve` and `approval: automatic`, and the CI guide names `cloudfall release approve` and no longer says no command acts on `approval`

sdk/README.md

@@ -51,6 +51,7 @@ engine's playbook contract for execution — never Ansible internals. Each
 operation is split into a pure, testable execution plan and a thin executor:
 
 ```console
+uv run cloudfall release approve --project config/examples crm-backend --release <release-id>
 uv run cloudfall deploy --project config/examples crm-backend --release <release-id> --yes
 uv run cloudfall rollback --project config/examples crm-backend --release <release-id> --yes
 uv run cloudfall restart --project config/examples crm-backend --yes
@@ -63,6 +64,16 @@ servers only with `--yes`. Without it they run every controller-side check
 DNS for a certificate) and print a `plan` naming the target servers, exit `0`,
 so a request can be reviewed before it runs.
 
+The example application `crm` declares `approval: manual`, so `deploy --yes`
+first needs a person's approval of that release: `release approve` asks them
+to type the release id at a terminal and records it under
+`decisions/releases/`, naming the archive SHA-256 and the servers. Without a
+record that counts, `deploy --yes` exits `PRECONDITION` with
+`deployment_approval_missing` and `migrate --yes` pauses at the deploy step;
+`deploy`'s plan shows `approval` with the command to run. `rollback` and
+`restart` ship no new code and need no approval. An application that
+declares `approval: automatic` deploys without a record.
+
 `domains apply <domain>` runs `domains.yml` on the Domain's proxy server
 alone and renders only that Domain, so no baseline, firewall or service task
 runs; it is safe on a server whose baseline another project owns. With

sdk/src/cloudfall/agent_tools.py

@@ -15,6 +15,7 @@
     StepRunner,
     subprocess_step,
 )
+from cloudfall.release_approval import APPROVALS_DIRECTORY
 from cloudfall.secrets import SopsSecretProvider
 
 if TYPE_CHECKING:
@@ -40,6 +41,8 @@ class AgentConfig:
     environment_directory: Path = Path("tmp/env")
     environment_receipts_directory: Path = Path("tmp/env-receipts")
     proposals_directory: Path = Path("tmp/operator/proposals")
+    approvals_directory: Path = Path(APPROVALS_DIRECTORY)
+    """Release approvals a ``manual`` application's deploy step reads."""
     gateway_ca_path: Path | None = None
     gateway_certificate_path: Path | None = None
     gateway_key_path: Path | None = None

sdk/src/cloudfall/app.py

@@ -249,6 +249,7 @@
 from cloudfall.release_approval import (
     APPROVALS_DIRECTORY,
     AUTOMATIC_PLAN,
+    ERROR_DEPLOYMENT_APPROVAL_MISSING,
     VALID_FOR_DEFAULT,
     VALID_FOR_MAXIMUM,
     VALID_FOR_MINIMUM,
@@ -257,9 +258,9 @@
     ReleaseApprovalError,
     ValidityHours,
     approve_release,
-    check_approval,
-    deployment_candidate,
+    deployment_approval,
     release_candidate,
+    rollback_approval,
     write_approval,
 )
 from cloudfall.release_approval import (
@@ -3118,6 +3119,14 @@ class MigrateArgs(EngineArgs):
         default=Path("tmp/migrate/plan.json"),
         description="Persisted migration plan (default: tmp/migrate/plan.json)",
     )
+    approvals: Path = Flag(
+        default=Path(APPROVALS_DIRECTORY),
+        description=(
+            "Directory of release approval records a deploy step reads for an "
+            f"application that declares approval: manual (default: "
+            f"{APPROVALS_DIRECTORY})"
+        ),
+    )
     build: tuple[str, ...] = Flag(
         default=(),
         description="Build a component from a git ref, as COMPONENT=REF (repeatable)",
@@ -3176,6 +3185,7 @@ def __post_init__(self) -> None:
             "releases",
             "artifacts",
             "plan_file",
+            "approvals",
             "secrets_dir",
         ):
             _inside_project(getattr(self, flag), flag.replace("_", "-"))
@@ -3233,6 +3243,7 @@ def config(self, fleet: Fleet, ctx: Ctx) -> AgentConfig:
             releases_directory=fleet.path(self.releases),
             artifacts_directory=fleet.path(self.artifacts),
             secrets_directory=fleet.path(self.secrets_dir),
+            approvals_directory=fleet.path(self.approvals),
             run=_step_runner(ctx, fleet.directory),
             decrypt=_sops_decrypt(ctx),
         )
@@ -3286,6 +3297,7 @@ class Migration:
         "PRECONDITION",
         "PARTIAL_FAILURE",
         "ENGINE_STEP_FAILED",
+        "RECORD_INVALID",
     ],
     timeout=None,
     supports_raw_payload=True,
@@ -3332,7 +3344,21 @@ def _migration_failure(
     cause = {"step": migration.step, **error}
     if migration.status == "paused":
         message = f"the migration paused at {migration.step}; resume it with --yes"
+        if cause.get("code") == ERROR_DEPLOYMENT_APPROVAL_MISSING:
+            # A person approves the built release first (S-008-14).
+            message = (
+                f"the migration paused at {migration.step}: release "
+                f"{cause['release']} needs a person's approval; run "
+                f"`{cause['command']}` at a terminal, then resume it with --yes"
+            )
         return Exit.PARTIAL_FAILURE(message, context=cause, data=migration)
+    if (
+        migration.status == "error"
+        and cause.get("code") == ERROR_APPROVAL_RECORD_INVALID
+    ):
+        # A broken approval record stops the deploy step as it stops deploy.
+        message = f"the migration stopped at {migration.step}: {cause['message']}"
+        return Exit.RECORD_INVALID(message, context=cause, data=migration)
     if (
         migration.status == "error"
         and cause.get("code") == ERROR_DATA_COPY_PREFLIGHT_REFUSED
@@ -4010,17 +4036,14 @@ def _deploy_approval(
 
     An ``automatic`` application's approvals folder is never read.
     """
-    candidate = deployment_candidate(
-        fleet.inventory, args.component, args.release, artifacts, args.schemas
-    )
-    if candidate is None:
-        return None
     try:
-        return check_approval(
-            candidate,
+        return deployment_approval(
+            fleet.inventory,
+            args.component,
+            args.release,
+            artifacts,
             fleet.path(args.approvals),
-            SchemaCatalog(args.schemas),
-            datetime.now(UTC),
+            args.schemas,
         )
     except ReleaseApprovalError as error:
         if error.code == ERROR_APPROVAL_RECORD_INVALID:
@@ -4125,8 +4148,13 @@ def rollback_command(args: RollbackArgs, ctx: Ctx, fleet: Fleet) -> RollbackPayl
     context = args.context(fleet, ctx)
     try:
         if not args.yes:
+            # A rollback never waits for an approval, even for a manual
+            # application; the plan says so and why (S-008-13).
             return RollbackPayload(
-                _plan(preview_rollback(context, args.component, args.release))
+                {
+                    **_plan(preview_rollback(context, args.component, args.release)),
+                    "approval": rollback_approval(fleet.inventory, args.component),
+                }
             )
         result = rollback(context, args.component, args.release)
     except LifecycleError as error:

sdk/src/cloudfall/commands.py

@@ -511,9 +511,12 @@ def _plan(subject: str, *extra: str) -> OutputShape:
         "rollback",
         CommandEffect.SERVERS,
         "switch one component back to an existing release",
-        gate=_YES,
+        gate=(
+            f"{_YES}; it needs no approval, and for an application that declares "
+            "`approval: manual` the plan's `approval` says so and why"
+        ),
         output=(
-            _plan("component", "release", "releaseCommandSkipped?"),
+            _plan("component", "release", "releaseCommandSkipped?", "approval"),
             _shape(*_EXECUTED, "release", when="with `--yes`"),
         ),
     ),
@@ -599,7 +602,11 @@ def _plan(subject: str, *extra: str) -> OutputShape:
         "migrate",
         CommandEffect.SERVERS,
         "run the resumable end-to-end migration plan",
-        gate=_YES,
+        gate=(
+            f"{_YES}; a `deploy:<component>` step of an application that declares "
+            "`approval: manual` pauses until a person runs `cloudfall release "
+            "approve` for the built release"
+        ),
         output=(
             _shape(
                 "status",