Skip to content

Refuse a bad --source-schema with PRECONDITION and add migrate --data-schema (S-007-12, S-007-13) - #169

Merged
shipmill-romamo[bot] merged 2 commits into
mainfrom
feat/126-source-schema-checks
Oct 30, 2026
Merged

shipmill-romamo[bot] merged 2 commits into
mainfrom
feat/126-source-schema-checks

Conversation

@shipmill-romamo

Copy link
Copy Markdown
Contributor

Summary

Builds S-007-12 and S-007-13 of docs/specs/007-data-copy-schemas.md.

  • PostgresSchemaName now takes exactly the spec's names: ^[a-z_][a-z0-9_]*$, at most 63 characters (read with fullmatch, so a trailing newline can't pass). Braces, quotes, semicolons and control characters fall outside it, so the Copy only the named schemas in data copy (#124) #134 guard against Ansible templating stays.
  • DataCopySchemas refuses a name given twice instead of silently dropping the repeat.
  • Behaviour change: a malformed or repeated --source-schema (CLI and the data_copy MCP tool) now exits PRECONDITION (exit 4, context code data_copy_schema_invalid) from the command, before any playbook, as the spec's Behaviour and S-007-12 say. Since Copy only the named schemas in data copy (#124) #134 it was a usage error (exit 2) raised from DataMigrateArgs.__post_init__. One exception: a value holding a newline is still refused by the CLI framework's own argv guard as ARG_ERROR (exit 2) before the command runs; no playbook runs either way.
  • cloudfall migrate --data-schema DATABASE=NAME (repeatable, MCP data_schema) narrows the data:<database> step to the named schemas, passing the same DataCopySchemas to migrate_data that data copy --source-schema does; the plan step's description names them. A --data-schema for a database with no --data exits PRECONDITION with code migrate_data_schema_without_data, in plan mode as with --yes, before any step. A --data-schema value without = is a usage error, as a malformed --data already is.
  • The Neon and Render guides name the rule and --data-schema.

Plan

  • Serves: G-3
  • Files: sdk/src/cloudfall/domain.py, sdk/src/cloudfall/lifecycle.py, sdk/src/cloudfall/app.py, sdk/src/cloudfall/migrate.py, sdk/tests/test_lifecycle.py, sdk/tests/test_data_migration.py, sdk/tests/test_migrate.py, sdk/tests/test_agent_server.py, sdk/tests/test_app.py, docs/neon-migration-guide.md, docs/render-migration-guide.md, changelog.d/126-source-schema-checks.md
  • Order: failing tests first (S-007-12 over argv and the MCP data_copy and migrate tools, S-007-13 on the migrate data step's playbook argv), then PostgresSchemaName to ^[a-z_][a-z0-9_]*$ and at most 63 characters, DataCopySchemas refusing a repeat, both answered as PRECONDITION in the command instead of a usage error, then migrate --data-schema DATABASE=NAME through MigrateOptions to the data:<database> step; Behaviour as in spec 007, "Choosing the schemas"
  • Proof: S-007-12, S-007-13

Test plan

  • S-007-12: test_s007_12_data_copy_refuses_a_malformed_or_repeated_schema (Bad;Name, app app, empty, Jinja, tab, upper case, leading digit, quote, 64 characters; with and without --yes: exit 4, no inventory render), test_s007_12_data_copy_takes_the_longest_schema_name, test_s007_12_cli_migrate_refuses_a_bad_data_schema_before_any_playbook (no --data, malformed, repeated; with --yes: no plan file, no inventory), test_s007_12_migrate_refuses_a_data_schema_without_its_data (no engine step started), test_s007_12_data_copy_schemas_refuse_a_repeat_and_a_malformed_name, test_s007_12_the_data_tools_refuse_a_bad_schema_as_the_cli_does (over the real MCP stdio transport)
  • S-007-13: test_s007_13_migrate_copies_only_the_data_schema_as_data_copy_does (the resumed data step's playbook argv equals data copy --source-schema public's), test_s007_13_migrate_groups_each_data_schema_under_its_database
  • uv run ruff check ., uv run mypy, ansible-lint engine/ansible, and the touched test files pass locally; the full suite is left to CI
  • Not run against a PostgreSQL server: nothing here changes the engine, and the copy itself is Copy only the named schemas in data copy (#124) #134's

Fixes #126

🤖 Generated with Claude Code

…-schema

A schema name must match ^[a-z_][a-z0-9_]*$ and be at most 63
characters, and may be named once; a malformed or repeated name now
exits PRECONDITION from the command before any playbook, as spec 007
says, where it was a usage error and a repeat was silently dropped.
cloudfall migrate takes --data-schema DATABASE=NAME beside --data, so
the data step copies only the named schemas as data copy
--source-schema does; one for a database with no --data is refused
before any step (S-007-12, S-007-13).

Fixes #126

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A pending step of a persisted plan kept the description it was saved with,
so a data step saved with --data-schema crm=public and resumed without it
reported "schema public" while it copied every schema (#126, S-007-13).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@shipmill-romamo
shipmill-romamo Bot merged commit de17114 into main Oct 30, 2026
1 check passed
@shipmill-romamo
shipmill-romamo Bot deleted the feat/126-source-schema-checks branch October 30, 2026 14:25
shipmill-romamo Bot added a commit that referenced this pull request Oct 30, 2026
…11) (#171)

* Accept a command health check in the component schema (#170)

A component can declare healthCheck type: command (command argv,
timeoutSeconds 1-300, attempts 1-20, no other key). The SDK parses it into
value objects and the deploy, restart and rollback plans name the check and
its command in wouldRun; the plan JSON gains no field (S-009-1, S-009-13).

The engine does not run the check yet (#164), so the deploy role's entry
points refuse a command check on their first task instead of passing an
unrun gate.

Fixes #161

Co-authored-by: shipmill-romamo[bot] <338557544+shipmill-romamo[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

* Refuse a bad --source-schema with PRECONDITION and add migrate --data-schema (S-007-12, S-007-13) (#169)

* Refuse a bad --source-schema with PRECONDITION and add migrate --data-schema

A schema name must match ^[a-z_][a-z0-9_]*$ and be at most 63
characters, and may be named once; a malformed or repeated name now
exits PRECONDITION from the command before any playbook, as spec 007
says, where it was a usage error and a repeat was silently dropped.
cloudfall migrate takes --data-schema DATABASE=NAME beside --data, so
the data step copies only the named schemas as data copy
--source-schema does; one for a database with no --data is refused
before any step (S-007-12, S-007-13).

Fixes #126

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Describe a resumed migrate step by this run's options

A pending step of a persisted plan kept the description it was saved with,
so a data step saved with --data-schema crm=public and resumed without it
reported "schema public" while it copied every schema (#126, S-007-13).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: shipmill-romamo[bot] <338557544+shipmill-romamo[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

* Validate Domain routes (S-003-1, S-003-8..12) (#168)

* Validate Domain routes (S-003-1, S-003-8..12)

A Domain can now declare spec.routes: proxy routes (path + upstream) and
static routes (path + static component, directory, spaFallback). The schema
holds each route's shape and the release-relative directory and fallback;
config validate checks the route source, path syntax, reserved and duplicate
paths, and that a static component exists and is deployed on the proxy
server. The inventory parses routes into value objects and serializes them
only when present, so a Domain with only proxy.upstream is unchanged.
Rendering routes comes with #162.

Fixes #159
Part of #90

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Refuse a trailing newline in a static route's directory or spaFallback

The releasePath schema pattern ends with $, which Python's re.search lets
match before one trailing newline, so config validate accepted
directory: "web\n" that ReleaseRelativePath then refused with a ValueError
when the inventory loaded. Refuse control characters in the schema, as
component.schema.json's controlCharacterFree does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: shipmill-romamo[bot] <338557544+shipmill-romamo[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

* Refuse a data copy the target can't hold before dumping it

A preflight on the target host, after the empty-target check and
before pg_dump, reads the source and target catalogs and names every
schema the source lacks, extension and policy role the target lacks,
and auth. reference, taking every name only as a psql variable. Any
finding refuses the copy with nothing dumped: data copy and migrate's
data step exit PRECONDITION with data_copy_preflight_refused and the
four lists, read whole from a findings file under --data-migrations.
pg_dump takes the needed extensions with named schemas, and pg_restore
runs in one transaction, so a failed restore leaves no table and no
receipt. Spec 007 is built (S-007-6 to S-007-11).

Fixes #125

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: shipmill-romamo[bot] <338557544+shipmill-romamo[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Check --schema values and add migrate --data-schema (S-007-12, S-007-13)

0 participants