[shipmill-romamo]

Summary

cloudfall data copy dumped every schema of the source, but its empty-target check and its row-count query read only public, so the receipt's "per-table row counts matched" vouched for one schema of many (S-007). This PR adds a repeatable source-schema selection:

• data copy --source-schema NAME (repeatable; the data_copy MCP tool gets a source_schema list). pg_dump gets one --schema per name, and the empty-target check and the row-count check read exactly the selected schemas
• Without it, the pg_dump command is byte-for-byte the 0.7.0 one, and the plan's extra vars are unchanged. The selected schemas are every source schema except pg_catalog, information_schema, pg_toast, and the pg_temp_* / pg_toast_temp_* families, read from the source before the empty-target check
• The row-count query returns one schema,table,rows line per base table, ordered by schema and table
• The receipt gains spec.schemas (sorted) and a schema on each spec.tables[] entry beside table and rows. This is additive: no JSON schema, schema lock, or SDK reader exists for DataMigrationReceipt, and table/rows keep their meaning
• The plan without --yes says either "dump schemas app, public of the database ..." or "dump every non-system schema of the database ..."

Injection safety. Schema names never go into SQL text. They reach psql as a single --set cloudfall_data_schemas=<JSON array> value (one argv element, or one quoted shell word on the source side), and the SQL reads it back as :'cloudfall_data_schemas', which psql quotes as a literal. The queries now run from stdin with --no-psqlrc --set ON_ERROR_STOP=1. On the pg_dump side, each name is double-quoted as a literal pattern (internal " doubled) and then shell-quoted. This holds without #126's name check: a test passes Bad;Name"$(touch x) and gets back one quoted argument.

Spec deviation. S-007 calls the flag --schema. treaty reserves --schema for printing a command's schema (REQ-F-079), so registering it fails at import. The flag is therefore --source-schema, which pairs with --source-url-file. The spec text and the #125/#126 wording should follow, or treaty would need to free the name.

Out of scope here: the preflight refusals and pg_restore --single-transaction (#125), and name validation and migrate --data-schema (#126). Until #125 adds --extension, a --source-schema copy dumps no extensions, so a selected table that needs one fails at restore.

Fixes #124

Test plan

• uv run ruff check ., uv run mypy: clean
• ansible-lint run from engine/ansible (where its ansible.cfg lives): passed, production profile
• ansible-playbook --syntax-check on playbooks/data.yml against the rendered example inventory: passed
• pytest engine/tests sdk/tests/test_data_migration.py sdk/tests/test_lifecycle.py sdk/tests/test_app.py sdk/tests/test_commands.py sdk/tests/test_agent_server.py sdk/tests/test_project.py: 179 passed
• New engine/tests/test_data_migration_role.py runs the role's tasks through Ansible's own Templar: the dump command with and without names, the schema selection, the psql variable, the empty-target assertion, and the receipt built from CSV lines (including a quoted odd, "name" table)
• No live PostgreSQL ran. The repo has no live-database test for data copy, and this machine has no psql/postgres. S-007-1 to S-007-5 are covered only as rendered and unit tests. The SQL itself (the jsonb_array_elements_text(:'var'::jsonb) filter, the pg_namespace query, psql running the queries from stdin) has not been executed against a server. A proving run should check them
• The full suite is left to CI

Plan

• Serves: quality bar
• Files: engine/ansible/roles/cloudfall_data_migration/{tasks,defaults,meta/argument_specs}, engine/ansible/playbooks/data.yml, sdk/src/cloudfall/{app,lifecycle}.py, engine/tests/test_data_migration_role.py, sdk/tests/{test_data_migration,test_lifecycle}.py, docs/{neon,render}-migration-guide.md, changelog.d/124-data-copy-schema.md
• Order: the role's schema selection and queries first, then the receipt, then the SDK flag and plan text, then the tests per criterion
• Proof: S-007-1, S-007-2, S-007-3, S-007-4, S-007-5, S-007-14 (specs.py coverage --spec 007)

🤖 Generated with Claude Code

[shipmill-romamo]

@romamo Decision needed: the approved spec 007 names the flag data copy --schema, but the CLI framework (treaty) already uses --schema on every command to print that command's schema, so the CLI won't load with it. Which name should the flag have?

1. --source-schema (recommended): this PR as it stands, with the MCP parameter source_schema. A follow-up docs change amends S-007-1, S-007-2, S-007-14 and the wording of Refuse a data copy the target can't hold before restoring (S-007-6..11) #125 and Check --schema values and add migrate --data-schema (S-007-12, S-007-13) #126 (--source-schema, and migrate --data-schema keeps its name). Nothing else in the spec changes
2. Another name, such as --only-schema or --include-schema: the PR is revised to it, and the spec is amended the same way
3. Keep --schema: this waits on a treaty release that frees the name, and Copy only the selected schemas with data copy --schema (S-007-1..5, S-007-14) #124 stays open until then

Also in this PR, for your information (decided under the spec's own wording): without the flag, the empty-target and row-count checks now cover every non-system schema, not only public, as S-007-2 and S-007-5 read.

Reply here with a number or your own answer; shipmill takes this up on the tick after your reply.

[romamo]

Decision by @romamo, relayed by Claude Code

1. --source-schema, with the MCP parameter source_schema; amend S-007-1, S-007-2, S-007-14 and the wording of Refuse a data copy the target can't hold before restoring (S-007-6..11) #125 and Check --schema values and add migrate --data-schema (S-007-12, S-007-13) #126 in a follow-up docs change.

[shipmill-romamo]

Review before merge, on @romamo's answer (option 1, --source-schema as it stands):

• Rebased onto main (import conflicts in sdk/src/cloudfall/app.py and sdk/src/cloudfall/lifecycle.py)
• Security fix (46b57b1): a --source-schema name reached Ansible as an extra var, and Ansible templates extra vars, so --source-schema "{{ lookup('pipe','id') }}" (or the same in the MCP source_schema) ran a command on the management host. Names are now a PostgresSchemaName: non-empty, at most 63 bytes, no control character and no brace, refused as a usage error before any plan
• f9dfff2: the S-006-13 test replays the receipt with the new schema,table,rows lines; S-006-13's behaviour is unchanged
• Spec 007 and Refuse a data copy the target can't hold before restoring (S-007-6..11) #125/Check --schema values and add migrate --data-schema (S-007-12, S-007-13) #126 now name the flag --source-schema (Name the data copy flag --source-schema in spec 007 #157)

Left for #125: a named schema missing from the source is skipped while the receipt's spec.schemas still lists it (S-007-6 refuses it). Left for #126: a repeated name is de-duplicated, not refused (S-007-12). No live PostgreSQL ran; the SQL is checked by rendered templates only.