Repository navigation
Converge PostgreSQL databases owned by a declared role (S-006-5, 6, 7, 12, 13) - #139
Conversation
services.yml now creates the owner login role, sets its password from passwordRef as SCRAM-SHA-256 on every converge, and gives it the database. Every path that runs services.yml (the converge_services MCP tool, cloudfall migrate, operator approve and run, and cloudfall-engine playbook run services.yml) reads the password from --secrets-dir and stages it in a 0600 file in a private temp dir outside the project, passing only the path to Ansible, as data copy does with its source URL file. The role copies it to the host as 0600, psql reads it there, and every task that sees it is no_log. Answers name the reference, the key, and the value's SHA-256. A missing file or key fails with PRECONDITION naming the path and key before any playbook runs. Backups, the restore drill, and data copy already select by name and restore as the owner; tests now prove it for an owned database. The example project renders byte-identical to goldens captured from main. Fixes #122 Part of #81 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Spec 006 lists the operator's converge-services operation among the paths that run services.yml, and its answer must name the reference, the key, and the value's SHA-256. operator approve answers with the proposal receipt, so its outcome gains an optional ownedRoles list; operator run adds it to each executed proposal's entry. The schema change is one optional property, so every existing receipt validates. Part of #122 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cloudfall-engine playbook run services.yml refused an inventory that wasn't a rendered Cloudfall one with an argument error, changing the command for its current users. Such an inventory, or one declaring no owned database, now stages no passwords and runs as on main; only an inventory that declares owned databases reads the secrets, and a missing file or key there still stops with PRECONDITION. Part of #122 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@romamo Decided without waiting: the owner password reaches Ansible as the path of a 0600 file staged outside the project for the run (the "own role variable" of spec 006 is that path, never the value), and converge refuses an empty password value like a missing key, and one role owning databases with two different Why: spec 006 says the value reaches the host in a 0600 file removed after the run and never appears in Ansible output (S-006-6); passing the path keeps Ansible from holding the value at all, as Reply with a comment starting with |
|
@romamo Decided without waiting: Why: spec 006 names Reply with a comment starting with |
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Run Component build steps in artifact build artifact build packaged the git tree only, so a Component could not ship a built frontend such as a gitignored web/dist. It now checks every spec.buildTools entry on PATH at its minimum version before cloning, runs each spec.build step in the clean checkout through env -i with only PATH, HOME, LANG, TMPDIR and CI=true, streams step output to stderr so stdout stays one JSON envelope, and packages the tracked files plus every spec.artifact.include path. A failed or timed-out step or a missing include writes no archive. The Artifact metadata gains an optional spec.build with the steps and tool versions. A Component without build is packaged exactly as before. Spec 002 is now built: every criterion has its test and its verification. Fixes #115 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Bound artifact build by its declared step timeouts; refuse deleted tracked files A fixed 1800 s command limit cut long builds short with treaty's TIMEOUT, and the SDK/MCP path capped the engine call at 3600 s, so a step never reached its own timeoutSeconds. artifact build now declares no fixed limit and applies its own: 1800 s for the clone plus every declared step timeout (ComponentInventory.artifact_timeout_seconds), shared by every child, so a step's own limit is the one that fires (artifact_build_step_failed). An explicit --timeout still replaces it. The SDK's artifact step carries no cap of its own and leaves the limit to the engine. A step that deletes a tracked file broke S-002-3 by leaving it out of the archive. The build now refuses it with artifact_tracked_file_missing, naming the path, before any archive is written. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Keep artifact build --timeout 0 meaning no limit Treaty's --timeout reaches a handler only as the effective timeout, the same whether the flag was omitted or 0, so the computed limit silently replaced an explicit 0. The command now takes --timeout as a field of its own, which treaty allows in place of its framework flag, parsed with Timeout.parse: omitted applies the clone time plus every declared step timeout, 0 disables the limit, and any other value replaces it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Refuse an artifact.include path that leaves the checkout through a symlink A tracked symlink above an include path (web/link -> /outside, include web/link/secrets) let the build package controller files outside the checkout. The include path itself may still be a symlink, archived as one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Sort the engine CLI's type-only imports after the rebase onto #139 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: shipmill-romamo[bot] <338557544+shipmill-romamo[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Summary
Implements S-006's converge of a database owned by a declared role (
docs/specs/006-standalone-postgresql.md), criteria S-006-5, S-006-6, S-006-7, S-006-12 and S-006-13. Network access over TLS (S-006-8 to S-006-10) is #123, so the spec staysapproved.cloudfall_postgresqlcreates theownerlogin role along with the application roles, sets its password on every converge (tasks/owner_password.yml), and gives it the database:CREATE DATABASE ... OWNER, plusALTER DATABASE ... OWNER TOwhen an existing database belongs to someone else. The password reaches the host as a0600file owned bypostgres.psqlreads that file through\set ... \cat`, afterSET password_encryption = 'scram-sha-256', then runsALTER ROLE ... PASSWORD :'cloudfall_password', so the value is in no task argument, no stdin, and no command line. Every task that sees it isno_log, and analways` removes the file. A converge that has no staged password for an owner fails before any changeapplicationoptional and takespasswordRef. It still refusesremoteAccess, which Open a PostgreSQL service to declared sources over TLS (S-006-8, 9, 10) #123 convergespasswordRef(SopsSecretProvider.fetch_value, thesecrets renderlayout) and writes it to a0600file in a0700private temp dir outside the project (staged_password_files). It passes only the paths, as--extra-vars '{"cloudfall_services_password_files": {service: {owner: path}}}', and removes the dir when the run ends, pass or fail. This is the patterndata copyalready uses for its source URL file: a controller-side file whose path crosses the command line, never its contentservices.ymlgo through it, each with a--secrets-dir(defaultsecrets): theconverge_servicesMCP tool (mcp serve --secrets-diralready existed), theservicesstep ofcloudfall migrate(new flag),operator approve/operator run(new flag onOperatorEngineArgs;engine_executornow takes the provider), andcloudfall-engine playbook run services.yml(new flag; reads the owned databases from the rendered inventory, decrypts throughctx.run sops. An inventory that isn't a rendered Cloudfall one, or declares no owned database, stages nothing and runs exactly as on main)PRECONDITIONnaming the expected path and key before any playbook starts.migratechecks before its first step, so the baseline doesn't run eitherowned_roleslist: theconverge_servicestool (newServicesConvergedanswer,high_entropy=False),playbook run(Played.owned_roles, empty for other playbooks), andmigrate's services step detail (ownedRoles, only when an owner exists). All of these are additiveoperator approve's answer is the proposal receipt, so itsspec.outcomegains an optionalownedRoleslist (same shape: service, owner,passwordRef, key,sha256) once aconverge-servicesrun sets an owner password.operator runaddsownedRolesto each executed proposal's entry.operator-proposal.schema.jsongets the optional property only, so every existing receipt still validates. The executor now returns theOwnedRoleRecords it set (internal;engine_executoris only called fromapp.py)data copyneeded no change. They already select bynameand restore with--role <owner>; tests now prove it for an owned databasedocs/secrets-guide.mdgains "Database owner passwords"Fixes #122
Part of #81
Test plan
engine/tests/test_examples_render_unchanged.py(S-006-12): rendersconfig/examplesthroughcloudfall-engine inventory renderand thecloudfall.conf,nftables.conf, backup and restore-check templates, and compares them byte for byte with goldens underengine/tests/golden/examples/. The goldens were captured fromorigin/mainbefore any change on this branchengine/tests/test_postgresql_owner_role.py:no_logon every task that sees the password (S-006-6)ansible-playbook -vvvon localhost with a stand-inpsql. The value reaches the stand-in via a0600file, the file is gone afterwards, and the value is in neither the Ansible output nor psql's stdin or argv (S-006-6)billing(S-006-13)billingand receiptspec.owner: billing(S-006-13)engine/tests/test_services_passwords.py:cloudfall-engine playbook run serviceswith a missing file or key answersPRECONDITIONwith path and key, and Ansible never starts (S-006-7). With a plain INI inventory, or JSON of another shape,playbook run services.ymlexits 0 as on main, with no secrets read and no argument error. A real run against a local host with a stand-insopson PATH (--tagsthat match no task) answersowned_roleswith the hash; the value is in neither stdout nor stderr at-vvv, and the staged file is gone (S-006-6)sdk/tests/test_owned_postgresql_converge.py:converge_serviceswith a recording step runner that reads the staged file mid-run (value,0600, dir0700, outside the project, removed afterwards), with the value in no argv, environment, project file, rendered inventory or answer (S-006-6). Missing file, missing key and empty value run no step (S-006-7). The operator executor fails with no step.approve()answers and receiptsownedRoleswith the hash, the receipt round-trips throughProposalStore.load, and the value appears in neither the answer, the receipt, nor any project file.operator run's autonomy report names each executed proposal's roles.cloudfall migrate --yesanswersPRECONDITIONbefore any step and writes no plan. The migrate services step recordsownedRolesand keeps the value out of the plan file. Theconverge_servicesMCP tool, over its real stdio transport, answersPRECONDITIONnaming path and keyspecs.py coverage --spec 006lists tests for S-006-5, 6, 7, 12 and 13uv run ruff check .,uv run mypy,uv run cloudfall config validate --project config/examples,cloudfall-engine inventory render, theservices.ymlanddata.ymlsyntax checks,ansible-lint engine/ansible(withengine/ansible/ansible.cfg), andengine/testsplus the SDK tests of touched modules (migrate, data migration, operator, secrets, agent server, lifecycle, owned PostgreSQL, commands, app, arguments). The full suite is left to CIpg_rolesshowsbillingwithrolcanloginand aSCRAM-SHA-256hash; that psql's backquote\setreads the staged file aspostgresandALTER ROLEstores it; theALTER DATABASE ... OWNER TOpath on a real cluster;pg_dumpofbilling, the drill passing, andpg_restore --role billingagainst a real source; and a realsopsdecryptPlan
secrets.py,converge_servicesin lifecycle, the four callers, the role tasks and argument spec, and tests per criterion (see the spec's Behaviour)specs.py coverage --spec 006; S-006-8 to S-006-10 stay with Open a PostgreSQL service to declared sources over TLS (S-006-8, 9, 10) #123)🤖 Generated with Claude Code