Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions changelog.d/106-enforce-manual-approval.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
### Breaking

- `deploy --yes` and `migrate --yes` on an application that declares `approval: manual` now need a person's approval of the release: without a `cloudfall release approve` record that counts (the same archive SHA-256, exactly the component's declared servers, not expired), `deploy --yes` exits 4 `PRECONDITION` with `deployment_approval_missing` before any playbook runs, naming the `release approve` command, and lists each record that doesn't count with `digest_mismatch`, `servers_changed` or `expired`. This hits every project that declares `manual` and deploys today, the example `crm.yaml` and every Render import included. Two ways out: keep `manual` and approve each release first (build it, run `cloudfall release approve` at a terminal, commit the record under `decisions/releases/` when CI deploys, then deploy), or switch the application to `approval: automatic` when another gate, such as GitHub's required reviewers, is the real one. The `deploy` plan and result gain `approval` (`policy`, and for `manual` the `status`, the counting record's `path`, `approver` and `expires_at` or `approved_at`, the records that don't count and the `command` to run), `deploy` gains `--approvals` (default `decisions/releases`) and the `RECORD_INVALID` exit for an approvals file that isn't a valid `ReleaseApproval`, and `cloudfall mcp serve` binds the approvals folder (its own `--approvals`) so a `deploy` call can't name its own; an `automatic` application's deploy never reads the folder (#163)
2 changes: 1 addition & 1 deletion changelog.d/160-release-approve.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
### Added

- `cloudfall release approve <component> --release <release>` records a person's approval of one built release as a `ReleaseApproval` (new `release-approval.schema.json`) under `decisions/releases/`: the application, component, release, archive SHA-256, git commit, servers, approver, `approvedAt`, `expiresAt` (`--valid-for` hours, 1 to 168, default 24) and `via: terminal`. It verifies the artifact first, refuses a component of an `automatic` application with `release_approval_not_required`, and writes the record only once a person typed the release id at a terminal; `--yes` never answers it, and it is no MCP tool. It deploys nothing, and `deploy` does not read the record yet (#160)
- `cloudfall release approve <component> --release <release>` records a person's approval of one built release as a `ReleaseApproval` (new `release-approval.schema.json`) under `decisions/releases/`: the application, component, release, archive SHA-256, git commit, servers, approver, `approvedAt`, `expiresAt` (`--valid-for` hours, 1 to 168, default 24) and `via: terminal`. It verifies the artifact first, refuses a component of an `automatic` application with `release_approval_not_required`, and writes the record only once a person typed the release id at a terminal; `--yes` never answers it, and it is no MCP tool. It deploys nothing (#160)
105 changes: 86 additions & 19 deletions docs/ci-deploy-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -319,25 +319,92 @@ naming `production` still waits for its reviewers.

## Approval: who presses the button

Every `Application` declares `spec.approval: manual` or `automatic`.
Cloudfall validates the field and writes it into the rendered inventory, but
no command acts on it today: `cloudfall deploy --yes` deploys an application
marked `manual` exactly as it deploys one marked `automatic`. The only gate
inside `deploy` is `--yes`, and a workflow that passes `--yes` has approved
its own deploy.

So in CI, the approval comes from GitHub, not from Cloudfall:

- Give every environment that holds an application marked `manual` required
reviewers, and turn on "Prevent self-review", so the person who pushed the
change cannot approve its deploy
- The job waits before its first step until a reviewer approves, so it never
sees the SSH key or the age key of that environment without approval
- Leave `staging` without reviewers if its applications are `automatic`

Read `approval: manual` as your declaration that such a gate must exist; the
environment's reviewers are what enforce it. The approval itself lands in
GitHub's deployment history, not in Cloudfall's record.
Every `Application` declares `spec.approval: manual` or `automatic`, and
`cloudfall deploy` enforces it:

- `automatic`: `deploy --yes` deploys, as the workflow above does. Keep
staging's applications `automatic` and its one job stays as it is
- `manual`: `deploy --yes` deploys a release only when a person approved
that release with `cloudfall release approve`. Without an approval that
counts it exits 4 `PRECONDITION` with `deployment_approval_missing`, before
any playbook runs, and names the command to run. `deploy` without `--yes`
shows the approval in its plan: `approval.status` is `approved` or
`missing`

An approval is a record under `decisions/releases/` in the project. It names
the archive's SHA-256, the component's servers and when it expires (24 hours
by default, `--valid-for` up to a week). It counts only while the archive
being deployed has that digest, the component is declared on exactly those
servers, and it has not expired. A failed health gate does not use it up, so
the same release can be retried until it expires.

A person approves at a terminal by typing the release id back; no flag, and
no CI job, answers that prompt. So a production deploy of a `manual`
application is two jobs with a person between them, and approval needs the
exact archive the deploy ships:

1. A build job builds the release and uploads `tmp/artifacts/<component>/`
as a workflow artifact
2. The person downloads that artifact into `tmp/artifacts/<component>/` of
their checkout and approves it:

```sh
gh run download <run-id> --name cloudfall-artifact-crm-backend --dir tmp/artifacts/crm-backend
uv run cloudfall release approve crm-backend --release 20260101T000000Z-abcdef0
git add decisions/releases
git commit -m "Approve crm-backend 20260101T000000Z-abcdef0"
git push
```

3. They dispatch the deploy job with the build's run id and the release. It
checks out the commit holding the record, downloads the same artifact and
deploys it:

```yaml
build:
runs-on: ubuntu-24.04
steps:
# checkout, uv, uv sync --locked, as in the workflow above
- name: Build the artifact
id: build
run: |
uv run cloudfall-engine artifact build "$COMPONENT" --ref "$REF" --format json > "$RUNNER_TEMP/build.json"
echo "release=$(jq -r '.data.release' "$RUNNER_TEMP/build.json")" >> "$GITHUB_OUTPUT"
- uses: actions/upload-artifact@v7
with:
name: cloudfall-artifact-${{ env.COMPONENT }}
path: tmp/artifacts/${{ env.COMPONENT }}/
```

```yaml
deploy:
runs-on: ubuntu-24.04
environment: production
steps:
# checkout, uv, sops, the SSH and age keys, secrets render, as above
- uses: actions/download-artifact@v7
with:
name: cloudfall-artifact-${{ env.COMPONENT }}
path: tmp/artifacts/${{ env.COMPONENT }}/
run-id: ${{ inputs.build_run_id }}
github-token: ${{ github.token }}
- name: Deploy the approved release
run: uv run cloudfall deploy "$COMPONENT" --release "$RELEASE" --env-file "tmp/env/$COMPONENT.env" --yes
```

The deploy job reads the record from its checkout, so dispatch it on the
commit that holds the approval. A rebuilt release has a new archive, and
its old approval no longer counts.

GitHub's required reviewers remain a useful outer gate: give the
`production` environment reviewers and turn on "Prevent self-review", and
the deploy job never sees that environment's SSH key or age key until a
reviewer lets it start. Their approval lands in GitHub's deployment
history; the `release approve` record is the one in Cloudfall's.

When the reviewers are the real gate and nobody will run `release approve`,
switch the application to `approval: automatic`, so the project no longer
claims a gate Cloudfall does not hold.

## Keeping the receipts

Expand Down
7 changes: 5 additions & 2 deletions docs/operator-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -192,8 +192,11 @@ proposals it executed and which it withheld, with reasons.
Approval is not a tool. A person approves a proposal by running
`cloudfall operator approve` from a shell, as in
[Reviewing and approving](#reviewing-and-approving); an agent that could
call it would run and verify the change it proposed itself. The watch
loop is not a tool either: it runs until stopped, as the service above.
call it would run and verify the change it proposed itself. The same holds
for `cloudfall release approve`: the `deploy` tool deploys a release of an
application that declares `approval: manual` only once a person approved
it at a terminal, and no tool writes that approval. The watch loop is not a
tool either: it runs until stopped, as the service above.

The agent is the brain here and the operator is the record. The agent can
read the proposals, relay the diagnosis and recommend one; a person
Expand Down
16 changes: 16 additions & 0 deletions sdk/src/cloudfall/agent_server.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@
is_project,
resolve_project_directory,
)
from cloudfall.release_approval import APPROVALS_DIRECTORY
from cloudfall.render_api import RENDER_API_URL
from cloudfall.resources import default_engine_directory, default_schema_directory

Expand Down Expand Up @@ -145,6 +146,14 @@ class McpServeArgs:
f"Decision records in the repository (default: {DECISION_DIRECTORY})"
),
)
approvals: Path = Flag(
default=Path(APPROVALS_DIRECTORY),
description=(
"Release approval records deploy reads in the project (default: "
f"{APPROVALS_DIRECTORY})"
),
)

def __post_init__(self) -> None:
"""Refuse two fleets: a server serves a project or a repository."""
if self.project is not None and self.repository is not None:
Expand Down Expand Up @@ -381,6 +390,9 @@ def bound_arguments(args: McpServeArgs) -> dict[str, object]:
"env_dir": _IMPORT_ENV,
"plan_file": _MIGRATE_PLAN,
"proposals": str(args.proposals),
# deploy reads the approvals a person recorded; a call can't point it
# at a folder of its own choosing.
"approvals": str(args.approvals),
"secrets_dir": str(args.secrets_dir),
# A chosen URL would send any file the call names as the key to it.
"api_url": RENDER_API_URL,
Expand Down Expand Up @@ -429,6 +441,10 @@ def bound_arguments(args: McpServeArgs) -> dict[str, object]:
operator_list and operator_show read the proposals that loop records.
Approving one is `cloudfall operator approve`, a command a person runs
from a shell; no tool here can do it, whatever reason is given for asking.
Deploying a release of an application that declares approval: manual
needs `cloudfall release approve` first, also a command a person runs at a
terminal; deploy's plan says whether an approval counts, and without one
deploy with yes refuses with deployment_approval_missing.
Every result is an envelope: read ok, then data, else error.
"""

Expand Down
77 changes: 72 additions & 5 deletions sdk/src/cloudfall/app.py
Original file line number Diff line number Diff line change
Expand Up @@ -248,16 +248,23 @@
from cloudfall.recording import Recording, RecordingError
from cloudfall.release_approval import (
APPROVALS_DIRECTORY,
AUTOMATIC_PLAN,
VALID_FOR_DEFAULT,
VALID_FOR_MAXIMUM,
VALID_FOR_MINIMUM,
ApprovalCheck,
Approver,
ReleaseApprovalError,
ValidityHours,
approve_release,
check_approval,
deployment_candidate,
release_candidate,
write_approval,
)
from cloudfall.release_approval import (
ERROR_RECORD_INVALID as ERROR_APPROVAL_RECORD_INVALID,
)
from cloudfall.render_api import (
ERROR_API_UNREACHABLE,
RENDER_API_URL,
Expand Down Expand Up @@ -397,7 +404,10 @@ def _serve_tools(args: McpServeArgs, ctx: Ctx) -> list[McpTool]:
app.exit_code(
"RECORD_INVALID",
86,
description="A decision record or proposal receipt could not be read",
description=(
"A decision record, proposal receipt or release approval record could "
"not be read"
),
retryable=False,
side_effects="none",
)
Expand Down Expand Up @@ -3872,12 +3882,20 @@ class DeployArgs(ComponentArgs):
default=Path("tmp/releases"),
description="Release receipt directory (default: tmp/releases)",
)
approvals: Path = Flag(
default=Path(APPROVALS_DIRECTORY),
description=(
"Directory of release approval records, read for an application "
f"that declares approval: manual (default: {APPROVALS_DIRECTORY})"
),
)

def __post_init__(self) -> None:
"""Keep relative paths inside the project."""
ComponentArgs.__post_init__(self)
_inside_project(self.artifacts, "artifacts")
_inside_project(self.releases, "releases")
_inside_project(self.approvals, "approvals")


@dataclass(frozen=True, slots=True, kw_only=True)
Expand Down Expand Up @@ -3985,14 +4003,43 @@ def _plan(preview: LifecyclePreview) -> dict[str, object]:
]


def _deploy_approval(
args: DeployArgs, fleet: Fleet, artifacts: Path
) -> ApprovalCheck | None:
"""Check the release's approval; ``None`` for an ``automatic`` application.

An ``automatic`` application's approvals folder is never read.
"""
candidate = deployment_candidate(
fleet.inventory, args.component, args.release, artifacts, args.schemas
)
if candidate is None:
return None
try:
return check_approval(
candidate,
fleet.path(args.approvals),
SchemaCatalog(args.schemas),
datetime.now(UTC),
)
except ReleaseApprovalError as error:
if error.code == ERROR_APPROVAL_RECORD_INVALID:
raise Exit.RECORD_INVALID(
error.detail, context={"code": error.code}
) from error
raise Exit.PRECONDITION(error.detail, context={"code": error.code}) from error


@app.command(
"deploy",
description=(
"Deploy one built component release behind its health gate; without "
"--yes it shows the plan"
"--yes it shows the plan. A release of an application that declares "
"approval: manual deploys only once a person approved it with "
"cloudfall release approve"
),
danger_level="mutating",
exit_codes=_LIFECYCLE_EXIT_CODES,
exit_codes=[*_LIFECYCLE_EXIT_CODES, "RECORD_INVALID"],
timeout=None,
subprocess=Subprocess("ansible-playbook"),
required_tools={"ansible-playbook": "2.21.0"},
Expand All @@ -4013,10 +4060,21 @@ def deploy_command(args: DeployArgs, ctx: Ctx, fleet: Fleet) -> DeployPayload:
context = args.context(fleet, ctx)
artifacts = fleet.path(args.artifacts)
try:
approval = _deploy_approval(args, fleet, artifacts)
if not args.yes:
preview = preview_deploy(context, args.component, args.release, artifacts)
return DeployPayload(
_plan(preview_deploy(context, args.component, args.release, artifacts))
{
**_plan(preview),
"approval": (
dict(AUTOMATIC_PLAN) if approval is None else approval.plan()
),
}
)
# Refused before the inventory render or any playbook: nothing on a
# server changes and no receipt is written.
if approval is not None and approval.counting is None:
raise approval.refusal()
result = deploy(
context,
args.component,
Expand All @@ -4027,11 +4085,20 @@ def deploy_command(args: DeployArgs, ctx: Ctx, fleet: Fleet) -> DeployPayload:
fleet.path(args.env_file) if args.env_file is not None else None
),
receipt_directory=fleet.path(args.releases),
approved_sha256=(
None if approval is None else approval.candidate.artifact_sha256
),
),
)
except LifecycleError as error:
raise _lifecycle_failed(error) from error
return DeployPayload({**result.as_dict(), "effect": "updated"})
return DeployPayload(
{
**result.as_dict(),
"effect": "updated",
"approval": dict(AUTOMATIC_PLAN) if approval is None else approval.used(),
}
)


@app.command(
Expand Down
22 changes: 18 additions & 4 deletions sdk/src/cloudfall/commands.py
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ class OutputKey:
),
**dict.fromkeys(
(
"ansible", "byKind", "byRisk", "certbotTimer", "certificate",
"ansible", "approval", "byKind", "byRisk", "certbotTimer", "certificate",
"dashboard", "decision", "error", "inventory", "operation",
"proposal", "query", "releaseCommand", "summary", "vhost",
),
Expand Down Expand Up @@ -486,10 +486,24 @@ def _plan(subject: str, *extra: str) -> OutputShape:
"deploy",
CommandEffect.SERVERS,
"activate one built release behind its health check",
gate=_YES,
gate=(
f"{_YES}; a release of an application that declares "
"`approval: manual` deploys only once a person approved it with "
"`cloudfall release approve`, and the plan's `approval` says whether "
"one counts"
),
output=(
_plan("component", "release", "releaseCommand"),
_shape(*_EXECUTED, "release", "receipt", when="with `--yes`"),
_plan("component", "release", "releaseCommand", "approval"),
_shape(
*_EXECUTED,
"release",
"receipt",
"approval",
when=(
"with `--yes`; `approval` names the record the deploy went on "
"when the application declares `approval: manual`"
),
),
),
),
CommandContract(
Expand Down
9 changes: 9 additions & 0 deletions sdk/src/cloudfall/lifecycle.py
Original file line number Diff line number Diff line change
Expand Up @@ -178,6 +178,8 @@ class DeployOptions:

environment_file: Path | None = None
receipt_directory: Path | None = None
approved_sha256: Sha256Digest | None = None
"""The archive digest an approval names; the deploy refuses any other."""


@dataclass(frozen=True, slots=True)
Expand Down Expand Up @@ -1201,6 +1203,13 @@ def deploy(
artifact = verify_artifact(
artifact_directory, component_id, release, context.schema_directory
)
approved = resolved_options.approved_sha256
if approved is not None and artifact.sha256 != approved.value:
detail = (
f"artifact digest {artifact.sha256} is not the approved digest "
f"{approved.value}: the archive changed after the approval was checked"
)
raise LifecycleError(_ERROR_ARTIFACT_DIGEST, detail)
plan = plan_deploy(
context,
component_id,
Expand Down
Loading
Loading