Converge PostgreSQL databases owned by a declared role (S-006-5, 6, 7, 12, 13)

changelog.d/122-owned-postgresql-converge.md

@@ -0,0 +1,3 @@
+### Added
+
+- `services.yml` converges a PostgreSQL database owned by a declared role: it creates the `owner` login role, sets its password from `passwordRef` as SCRAM-SHA-256 on every converge, and gives it the database; backups, the restore drill, and `data copy` (restoring as the owner, its receipt's `spec.owner`) cover it like any other database. Every path that runs `services.yml` (the `converge_services` MCP tool, the `services` step of `cloudfall migrate`, `operator approve` and `operator run`, and `cloudfall-engine playbook run services.yml`) reads the password from a `--secrets-dir` flag (default `secrets`) and stages it in a 0600 file in a private temp dir outside the project, removed after the run; the value is in no inventory, command line, Ansible output, answer, or receipt, and the answer's `owned_roles` names the reference, the key, and the value's SHA-256; `operator approve` records them as `ownedRoles` in the proposal receipt's `outcome` (an optional property) and `operator run` in each executed proposal's entry. A missing secrets file or key fails with `PRECONDITION` naming the expected path and key before any playbook runs (#122)

config/schemas/v1/operator-proposal.schema.json

@@ -247,6 +247,62 @@
               "type": "string",
               "minLength": 1,
               "maxLength": 1000
+            },
+            "ownedRoles": {
+              "description": "Owner role passwords the run set: the reference, the key, and the value's SHA-256, never the value",
+              "type": "array",
+              "items": {
+                "type": "object",
+                "properties": {
+                  "service": {
+                    "type": "string",
+                    "pattern": "^[a-z][a-z0-9]*(-[a-z0-9]+)*$"
+                  },
+                  "owner": {
+                    "type": "string",
+                    "pattern": "^[a-z][a-z0-9_]{0,62}$"
+                  },
+                  "passwordRef": {
+                    "type": "object",
+                    "properties": {
+                      "environment": {
+                        "type": "string",
+                        "pattern": "^[a-z][a-z0-9]*(-[a-z0-9]+)*$"
+                      },
+                      "path": {
+                        "type": "string",
+                        "pattern": "^/"
+                      },
+                      "key": {
+                        "type": "string",
+                        "pattern": "^[A-Za-z_][A-Za-z0-9_]{0,254}$"
+                      }
+                    },
+                    "required": [
+                      "environment",
+                      "path",
+                      "key"
+                    ],
+                    "additionalProperties": false
+                  },
+                  "key": {
+                    "type": "string",
+                    "pattern": "^[A-Za-z_][A-Za-z0-9_]{0,254}$"
+                  },
+                  "sha256": {
+                    "type": "string",
+                    "pattern": "^[0-9a-f]{64}$"
+                  }
+                },
+                "required": [
+                  "service",
+                  "owner",
+                  "passwordRef",
+                  "key",
+                  "sha256"
+                ],
+                "additionalProperties": false
+              }
             }
           },
           "required": [

docs/secrets-guide.md

@@ -107,6 +107,38 @@ with `EnvironmentFile=`.
 A missing source fails fast with the exact expected path; a malformed line
 fails with its line number; an unreadable value never half-renders a file.
 
+## Database owner passwords
+
+A PostgreSQL database owned by a declared role names its password with a
+`passwordRef`, one key of one source in the same layout:
+
+```yaml
+# services/postgresql-billing.yaml (excerpt)
+databases:
+  - name: billing
+    owner: billing
+    passwordRef:
+      environment: production
+      path: /billing/postgresql
+      key: BILLING_PASSWORD
+```
+
+Every path that runs `services.yml` (the `converge_services` MCP tool, the
+`services` step of `cloudfall migrate`, `cloudfall operator approve` and
+`operator run`, and `cloudfall-engine playbook run services.yml`) reads
+`secrets/production/billing/postgresql.env` from its `--secrets-dir` (default
+`secrets`) before any playbook starts. A missing file or key stops it with
+`PRECONDITION`, naming the expected path and key.
+
+The value is written to a `0600` file in a private temp directory outside the
+project, and only that file's path reaches Ansible. The role copies the file
+to the host as `0600`, `psql` reads it there, PostgreSQL stores the password as
+SCRAM-SHA-256, and both files are removed when the run ends. Every task that
+sees it runs with `no_log`. The answer's `owned_roles` names the reference,
+the key, and the value's SHA-256 (an operator proposal keeps them as
+`ownedRoles` in its receipt's `outcome`); the password is set on every converge, so a
+rotation is a commit in the private repository and a services converge.
+
 ## Drift detection
 
 Rendering writes an `EnvironmentReceipt` (component, content hash, key

engine/ansible/playbooks/services.yml

@@ -11,6 +11,13 @@
         name: cloudfall_postgresql
       vars:
         cloudfall_postgresql_service: "{{ cloudfall_service_item }}"
+        # Controller paths of the staged owner passwords cloudfall passes
+        # as cloudfall_services_password_files; never the values.
+        cloudfall_postgresql_password_files: >-
+          {{
+            (cloudfall_services_password_files | default({}))
+            [cloudfall_service_item.id] | default({})
+          }}
       loop: >-
         {{
           cloudfall_services | default([])

engine/ansible/roles/cloudfall_postgresql/defaults/main.yml

@@ -4,3 +4,5 @@ cloudfall_postgresql_repository_key_url: >-
   https://www.postgresql.org/media/keys/ACCC4CF8.asc
 cloudfall_postgresql_repository_key_path: /etc/apt/keyrings/postgresql.asc
 cloudfall_postgresql_repository_url: https://apt.postgresql.org/pub/repos/apt
+cloudfall_postgresql_password_files: {}
+cloudfall_postgresql_password_path: /var/lib/postgresql/.cloudfall-owner-password

engine/ansible/roles/cloudfall_postgresql/meta/argument_specs.yml

@@ -52,10 +52,21 @@ argument_specs:
                     required: true
                   application:
                     type: str
-                    required: true
                   owner:
                     type: str
                     required: true
+                  passwordRef:
+                    type: dict
+                    options:
+                      environment:
+                        type: str
+                        required: true
+                      path:
+                        type: str
+                        required: true
+                      key:
+                        type: str
+                        required: true
           backup:
             type: dict
             required: true
@@ -77,6 +88,16 @@ argument_specs:
               enabled:
                 type: bool
                 required: true
+      cloudfall_postgresql_password_files:
+        type: dict
+        default: {}
+        description: >-
+          Controller path of each owner role's staged password file, by
+          role; the values never enter a variable
+      cloudfall_postgresql_password_path:
+        type: path
+        default: /var/lib/postgresql/.cloudfall-owner-password
+        description: Host path an owner role's password is staged at, 0600
       cloudfall_postgresql_metrics_role:
         type: str
         default: alloy

engine/ansible/roles/cloudfall_postgresql/tasks/main.yml

@@ -12,6 +12,22 @@
       PostgreSQL deployment requires its declared Debian server and a
       loopback-only bind address
 
+- name: Require a staged password for every owner role
+  ansible.builtin.assert:
+    that:
+      - >-
+        cloudfall_postgresql_service.postgresql.databases
+        | selectattr('passwordRef', 'defined')
+        | map(attribute='owner')
+        | reject('in', cloudfall_postgresql_password_files)
+        | list
+        | length == 0
+    fail_msg: >-
+      An owner role of service {{ cloudfall_postgresql_service.id }} has no
+      staged password; run services.yml through cloudfall (converge_services,
+      cloudfall migrate, operator approve, or cloudfall-engine playbook run
+      services.yml), which reads each passwordRef from --secrets-dir
+
 - name: Validate the declared backup calendar expression
   ansible.builtin.command:
     argv:
@@ -132,7 +148,7 @@
   changed_when: false
   when: not ansible_check_mode
 
-- name: Create missing application login roles
+- name: Create missing database owner login roles
   ansible.builtin.command:
     argv:
       - /usr/bin/psql
@@ -154,6 +170,20 @@
     - item not in cloudfall_postgresql_existing_roles.stdout_lines
   changed_when: true
 
+- name: Set each owner role password from its staged file
+  ansible.builtin.include_tasks: owner_password.yml
+  loop: >-
+    {{
+      cloudfall_postgresql_service.postgresql.databases
+      | selectattr('passwordRef', 'defined')
+      | map(attribute='owner')
+      | unique
+      | list
+    }}
+  loop_control:
+    loop_var: cloudfall_postgresql_owner
+  when: not ansible_check_mode
+
 - name: Create the metrics monitoring role
   ansible.builtin.command:
     argv:
@@ -221,6 +251,51 @@
     - item.name not in cloudfall_postgresql_existing_databases.stdout_lines
   changed_when: true
 
+- name: Read the owners of existing PostgreSQL databases
+  ansible.builtin.command:
+    argv:
+      - /usr/bin/psql
+      - --port
+      - "{{ cloudfall_postgresql_service.bind.port }}"
+      - --tuples-only
+      - --no-align
+      - --command
+      - SELECT datname || '=' || pg_get_userbyid(datdba) FROM pg_database
+  become: true
+  become_user: postgres
+  register: cloudfall_postgresql_database_owners
+  changed_when: false
+  when:
+    - not ansible_check_mode
+    - >-
+      cloudfall_postgresql_service.postgresql.databases
+      | selectattr('passwordRef', 'defined') | list | length > 0
+
+- name: Give each owned database to its declared owner role
+  ansible.builtin.command:
+    argv:
+      - /usr/bin/psql
+      - --port
+      - "{{ cloudfall_postgresql_service.bind.port }}"
+      - --command
+      - ALTER DATABASE "{{ item.name }}" OWNER TO "{{ item.owner }}"
+  become: true
+  become_user: postgres
+  loop: >-
+    {{
+      cloudfall_postgresql_service.postgresql.databases
+      | selectattr('passwordRef', 'defined')
+      | list
+    }}
+  loop_control:
+    label: "{{ item.name }}"
+  when:
+    - not ansible_check_mode
+    - >-
+      item.name ~ '=' ~ item.owner
+      not in cloudfall_postgresql_database_owners.stdout_lines
+  changed_when: true
+
 - name: Create the backup directory
   ansible.builtin.file:
     path: "{{ cloudfall_postgresql_service.backup.directory }}"

engine/ansible/roles/cloudfall_postgresql/tasks/owner_password.yml

@@ -0,0 +1,40 @@
+---
+# Sets one owner role's password on every converge, so a rotation is a commit
+# in the secrets repository and a services converge. The value reaches the
+# host in a 0600 file removed after the run; psql reads the file itself, so
+# the value is in no command line, no task argument, and no Ansible output.
+- name: Set the password of owner role {{ cloudfall_postgresql_owner }}
+  block:
+    - name: Stage the owner role password on the host
+      ansible.builtin.copy:
+        src: "{{ cloudfall_postgresql_password_files[cloudfall_postgresql_owner] }}"
+        dest: "{{ cloudfall_postgresql_password_path }}"
+        owner: postgres
+        group: postgres
+        mode: "0600"
+      no_log: true
+
+    - name: Store the owner role password as SCRAM-SHA-256
+      ansible.builtin.command:
+        argv:
+          - /usr/bin/psql
+          - --port
+          - "{{ cloudfall_postgresql_service.bind.port }}"
+          - --no-psqlrc
+          - --quiet
+          - --set
+          - ON_ERROR_STOP=1
+        stdin: |
+          SET password_encryption = 'scram-sha-256';
+          \set cloudfall_password `cat {{ cloudfall_postgresql_password_path | quote }}`
+          ALTER ROLE "{{ cloudfall_postgresql_owner }}" WITH LOGIN PASSWORD :'cloudfall_password';
+      become: true
+      become_user: postgres
+      changed_when: true
+      no_log: true
+  always:
+    - name: Remove the staged owner role password
+      ansible.builtin.file:
+        path: "{{ cloudfall_postgresql_password_path }}"
+        state: absent
+      no_log: true