Skip to content

Add test/install foundation for update+upgrade testing (fresh/update/existing) - #34

Merged
jnasbyupgrade merged 7 commits into
masterfrom
advanced-testing/foundation
Aug 8, 2026
Merged

Add test/install foundation for update+upgrade testing (fresh/update/existing)#34
jnasbyupgrade merged 7 commits into
masterfrom
advanced-testing/foundation

Conversation

@jnasbyupgrade

@jnasbyupgrade jnasbyupgrade commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds test/install/load.sql, using pgxntool's PGXNTOOL_ENABLE_TEST_INSTALL feature, as the single place that installs the extension for the whole regression run, in one of three modes (TEST_LOAD_SOURCE, default fresh):

  • fresh -- drops both extensions first (idempotent re-run against a non-fresh DB), lands pgtap in a dedicated tap schema, then CREATE EXTENSION test_factory_pgtap CASCADE.
  • update -- CREATE EXTENSION test_factory VERSION :from then ALTER EXTENSION UPDATE, then installs test_factory_pgtap at current (it's only ever shipped one version, so no update path of its own to exercise yet). make test-update is a shorthand for make test TEST_LOAD_SOURCE=update.
  • existing -- the extension is already installed (a real pg_upgrade target, or an out-of-band update) -- only asserts it's present at the current version, plants a dependency guard, and proves it. Run against a real pre-existing install with make test TEST_LOAD_SOURCE=existing CONTRIB_TESTDB=<db> EXTRA_REGRESS_OPTS=--use-existing PGXNTOOL_ENABLE_TEST_BUILD=no.

All three modes leave the same observable end state, so test/sql/base.sql and test/sql/pgtap.sql no longer install anything themselves or branch on mode -- they just assume both extensions are already present. test/helpers/create_extension.sql (the old install-or-skip wrapper they used) is deleted.

Dependency guard

Planted only in existing mode: a view in schema test_factory_drop_guard depending on tf.tap(text,text) blocks a non-CASCADE DROP EXTENSION test_factory_pgtap. test_factory itself doesn't need an artificial guard -- test_factory_pgtap's own control file (requires = 'pgtap, test_factory') already blocks a non-CASCADE DROP EXTENSION test_factory as long as test_factory_pgtap is installed; load.sql proves that natural protection too.

test/sql/pgtap.sql's dependency check

Now a pg_depend catalog inspection (works uniformly in every mode) instead of the old "try to CREATE EXTENSION test_factory_pgtap bare and see if it drags test_factory in" approach, which only worked when this file could assume test_factory_pgtap wasn't installed yet -- an assumption that no longer holds once load.sql installs both extensions in every mode.

Test plan

  • make test passes on PG17 and PG12, in fresh, update, and existing modes
  • Existing mode verified against a manually pre-populated database
  • make lint passes

@coderabbitai

coderabbitai Bot commented Aug 4, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 2952126d-1656-44da-82ac-f6642b583c43

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@jnasbyupgrade
jnasbyupgrade force-pushed the advanced-testing/foundation branch 5 times, most recently from 3599d45 to 56fdc2e Compare August 6, 2026 23:38
jnasbyupgrade added a commit that referenced this pull request Aug 6, 2026
head.repo.owner.login only distinguishes "who owns the fork" for
fork-headed PRs. For an upstream-branch-headed PR (base and head both in
this repo -- what `gh stack` requires), head.repo.owner.login is always
this repo's own org, never the actual author, so the gate silently skipped
review on every PR in this session's stack regardless of who opened it.

Confirmed via the check-runs API that claude-review's conclusion was
"skipped" on PRs #33/#34/#35 -- all legitimately jnasbyupgrade's own work,
recreated as upstream-branch PRs specifically so `gh stack` could link
them.

PR author can't be spoofed by a third party any more than head repo owner
can, and it's the more direct question for this gate's actual purpose:
trusting the PERSON asking for a review to run with this repo's secrets,
not the repository their branch happens to live in.
Base automatically changed from add-test-build to master August 7, 2026 18:42
jnasbyupgrade and others added 5 commits August 7, 2026 13:42
…existing)

Implements the load-mode switch, dependency guard, and single-source-of-truth
role name from the advanced update+upgrade testing pattern (modeled on
Postgres-Extensions/cat_tools PR #16 and what has landed on its master
since), scoped to what test_factory can actually exercise today:

- test/install/load.sql (pgxntool's PGXNTOOL_ENABLE_TEST_INSTALL) now owns
  getting the extension to its target state via TEST_LOAD_SOURCE=
  fresh|update|existing, propagated as GUCs (test_factory.test_load_mode
  etc), validated at both make-parse-time and read time.
- test/helpers/create_extension.sql skips CREATE EXTENSION when load.sql
  already installed it (update/existing), while keeping fresh mode's
  original no-IF-NOT-EXISTS behavior (hard error on stale state) unchanged.
- test/sql/install.sql and the install-ordering half of test/sql/pgtap.sql
  are skipped under existing mode: their own non-CASCADE DROP EXTENSION is a
  deliberate fresh-mode-only test that would otherwise trip the guard.
- Dependency guard (existing mode only): a view depending on
  tf.tap(text,text) blocks a stray non-CASCADE DROP EXTENSION
  test_factory_pgtap; test_factory itself already has a natural guard for
  free via test_factory_pgtap's own `requires` clause, which load.sql also
  proves still holds.
- test/roles.sql is now the single source of truth for the test_role name,
  \i'd via test/helpers/deps.sql and test/install/load.sql.
- Alternate expected output (test/expected/{base,install,pgtap}_1.out) for
  the existing-mode leg, since it legitimately produces different (but
  equally valid) output -- generated from real `existing`-mode runs, no raw
  "not ok" TAP lines in either leg.

Verified against both PG12 and PG17: fresh (default), TEST_LOAD_SOURCE=update
(currently a no-op -- see comment in load.sql for why no CI job drives it
yet), and TEST_LOAD_SOURCE=existing against a real pre-populated database
(the make test ... --use-existing recipe). Fresh mode's expected output is
byte-for-byte unchanged from before this change.

Skipped/deferred (see PR description): TEST_SCHEMA (test_factory is
non-relocatable with hardcoded schema names, so the ambient-search_path
failure mode it protects against can't occur here); an update-path CI job
(no second version has ever shipped); the bridge-update/multi-origin
machinery from PR #16 (cat_tools-specific technical debt, not applicable).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Several review comments added by the test/install foundation work used
consecutive -- lines for what was really one continuous remark. This
repo's convention (see the pre-existing test/helpers/create.sql and
test/sql/install.sql) is to use C-style /* */ blocks for any comment
spanning more than one line, reserving -- for single-line remarks.

While converting test/roles.sql, reworded "test/sql/*.sql" to "*.sql
files under test/sql/" -- the original phrasing contained a literal
/* immediately after test/sql, which Postgres's nesting-aware block
comment parser reads as an unwanted nested comment opener, leaving the
enclosing comment unterminated.
… crash

test/CLAUDE.md still referenced test/sql/install.sql (deleted by
add-test-build, now this branch's base) in the existing-mode section
and the alternate-expected-file list; updated both to reflect that
packaging/dependency-declaration checks now live in
test/build/install.sql instead.

Also fixes a real, pre-existing bug in test/sql/pgtap.sql surfaced by
testing TEST_LOAD_SOURCE=update locally (not just the fresh/existing
legs the rebase itself required): test/install/load.sql's update mode
installs test_factory only, never test_factory_pgtap. pgtap.sql's
non-existing-mode branch calls test/helpers/create_extension.sql for
test_factory (a no-op when already installed, per its own
already_installed check) and then unconditionally DROP TABLEs
pre_install_role/post_install_role -- which that no-op never created,
so the DROP errored under update mode. Guarded the same way
test/helpers/create.sql already guards its own read of those tables
(to_regclass(...) IS NOT NULL), rather than inventing a new pattern.
…dependency check

test/install/load.sql is now the single place that installs the extension
in every TEST_LOAD_SOURCE mode (fresh/update/existing), not just
update/existing. This follows from deleting test/build/install.sql
(rebase fallout from the add-test-build branch): test/build exists only to
run scripts "bare" for better error context with throwaway results, so
packaging/dependency checks belong in test/install/load.sql, which is
meant to commit and persist.

test/sql/base.sql and test/sql/pgtap.sql no longer branch on load mode or
install anything themselves -- test/helpers/create_extension.sql (the
install-or-skip wrapper they used) is deleted outright. This also means
base.out/pgtap.out no longer need per-mode alternate expected files
(test/expected/{base,pgtap}_1.out), since existing mode now produces the
same output as fresh/update.

pgtap.sql's dependency check moved from "try to CREATE EXTENSION
test_factory_pgtap and see if it drags test_factory in" (only valid when
pgtap.sql itself does the installing) to inspecting pg_depend directly,
which works regardless of when/how installation happened. Confirmed
against a live database that extension-requires-extension edges use
deptype 'n' (normal), not 'e' (DEPENDENCY_EXTENSION, which instead means
"this object belongs to this extension").

Fresh mode's CASCADE install surfaced a real ordering bug: it would
cascade-install pgtap into the ambient search_path (public) before
tap_setup.sql got a chance to create it in schema "tap", so the latter's
CREATE EXTENSION IF NOT EXISTS silently no-op'd and every pgTAP-based test
failed with "function no_plan() does not exist". Fixed by having load.sql
create the tap schema and install pgtap into it before installing
test_factory_pgtap in any mode.

Verified fresh, update, and existing (against a manually pre-populated
database) all pass with the same expected output.
Never ran `make lint` locally after the foundation redesign, so these two
multi-line -- comment blocks (added by that redesign) went unnoticed until
CI caught them. Converted both to /* */ per the linter's rule.
@jnasbyupgrade
jnasbyupgrade force-pushed the advanced-testing/foundation branch from e05623f to 0176a3b Compare August 7, 2026 18:42
jnasbyupgrade added a commit that referenced this pull request Aug 7, 2026
head.repo.owner.login only distinguishes "who owns the fork" for
fork-headed PRs. For an upstream-branch-headed PR (base and head both in
this repo -- what `gh stack` requires), head.repo.owner.login is always
this repo's own org, never the actual author, so the gate silently skipped
review on every PR in this session's stack regardless of who opened it.

Confirmed via the check-runs API that claude-review's conclusion was
"skipped" on PRs #33/#34/#35 -- all legitimately jnasbyupgrade's own work,
recreated as upstream-branch PRs specifically so `gh stack` could link
them.

PR author can't be spoofed by a third party any more than head repo owner
can, and it's the more direct question for this gate's actual purpose:
trusting the PERSON asking for a review to run with this repo's secrets,
not the repository their branch happens to live in.
…ts own line

The four test_factory.test_load_mode/test_update_from/test_update_to reads
and the has_update_to derived flag were 4 independent round-trips for no
reason -- one row, one \gset. Also moved every remaining \gset in this file
onto its own line, for consistency.
jnasbyupgrade added a commit that referenced this pull request Aug 7, 2026
head.repo.owner.login only distinguishes "who owns the fork" for
fork-headed PRs. For an upstream-branch-headed PR (base and head both in
this repo -- what `gh stack` requires), head.repo.owner.login is always
this repo's own org, never the actual author, so the gate silently skipped
review on every PR in this session's stack regardless of who opened it.

Confirmed via the check-runs API that claude-review's conclusion was
"skipped" on PRs #33/#34/#35 -- all legitimately jnasbyupgrade's own work,
recreated as upstream-branch PRs specifically so `gh stack` could link
them.

PR author can't be spoofed by a third party any more than head repo owner
can, and it's the more direct question for this gate's actual purpose:
trusting the PERSON asking for a review to run with this repo's secrets,
not the repository their branch happens to live in.
…y to load.sql

New CLAUDE.md rule: any \if/\else/\endif block spanning more than ~a dozen
lines needs a short block-naming comment on each control statement,
including \endif, so a reader landing on \else/\endif alone can tell which
\if it belongs to. Applied it to load.sql's two long blocks
(existing-vs-fresh/update, update-vs-fresh install).

The comment must go on its own line above the control statement, not
trailing on the same line: unlike SQL statements, \if/\else/\endif don't
strip a trailing -- as a comment. \if parses the whole rest of the line as
its boolean expression (a trailing comment breaks parsing outright, caught
by re-running the suite after the first attempt used trailing comments --
test/install/load.out picked up literal "unrecognized value ... Boolean
expected" and "extra argument ... ignored" lines). Documented this gotcha
in the new CLAUDE.md rule so it isn't rediscovered the same way again.

Also shortened test/sql/pgtap.sql's dependency-check comment -- kept only
the one genuinely non-obvious fact (deptype 'n' vs 'e'), dropped the
now-irrelevant history of the previous bare-CREATE-EXTENSION approach.
jnasbyupgrade added a commit that referenced this pull request Aug 7, 2026
head.repo.owner.login only distinguishes "who owns the fork" for
fork-headed PRs. For an upstream-branch-headed PR (base and head both in
this repo -- what `gh stack` requires), head.repo.owner.login is always
this repo's own org, never the actual author, so the gate silently skipped
review on every PR in this session's stack regardless of who opened it.

Confirmed via the check-runs API that claude-review's conclusion was
"skipped" on PRs #33/#34/#35 -- all legitimately jnasbyupgrade's own work,
recreated as upstream-branch PRs specifically so `gh stack` could link
them.

PR author can't be spoofed by a third party any more than head repo owner
can, and it's the more direct question for this gate's actual purpose:
trusting the PERSON asking for a review to run with this repo's secrets,
not the repository their branch happens to live in.
@jnasbyupgrade

Copy link
Copy Markdown
Collaborator Author

@claude pls review

@claude

claude Bot commented Aug 7, 2026

Copy link
Copy Markdown

Claude finished @jnasbyupgrade's task in 3m 1s —— View job


Review complete

  • Gather context (PR description, diff, changed files)
  • Read changed files in detail
  • Review test/install/load.sql (new load-mode switch logic)
  • Review test/helpers/create.sql and create_extension.sql deletion
  • Review test/sql/pgtap.sql gating changes
  • Review test/roles.sql, Makefile, CLAUDE.md docs
  • Post final review

Summary

This is a well-executed, carefully reasoned foundation piece. The fresh/update/existing split in test/install/load.sql is clean, the GUC validation (parse-time $(error...) in the Makefile and read-time rejection with no missing_ok) is defensive in the right places, and the extensive inline comments explain non-obvious decisions (nested-comment gotcha, why the dependency guard is scoped to existing mode only, why test_factory gets its drop-guard "for free" via test_factory_pgtap's requires clause) rather than just restating the code. test/CLAUDE.md is fully consistent with the actual diff (no per-mode alternate expected files needed since base.sql/pgtap.sql behave identically across modes now).

Findings

1. PR description is stale relative to the final commits
The "Rebase note" and "What this adds" sections describe things that no longer match the code in this branch:

  • They claim test/expected/{base,pgtap}_1.out alternate files were added for the existing-mode leg with "different-but-correct output (skipped sections, a skipped role-restore check)". Those files don't exist anywhere in this diff, and test/CLAUDE.md (also part of this PR) explicitly says the opposite: "no per-mode alternate expected files" because all three modes leave the same observable end state.
  • The rebase note describes a crash fix in test/sql/pgtap.sql involving DROP TABLE pre_install_role/post_install_role guarded by to_regclass(...) IS NOT NULL. That code path doesn't exist in the final pgtap.sql either — it was superseded by the unconditional pg_depend-based check (commit ae91d20).

Both seem to be leftover narrative from an earlier iteration of the stack that got simplified in the later commits (ae91d20, 04e71aa) without the description being updated. Worth cleaning up before merge so the PR history/description doesn't mislead future readers (or git log archaeology) about what actually shipped.

2. existing-mode leaves permanent artifacts in the target database
In test/install/load.sql's existing branch, CREATE SCHEMA IF NOT EXISTS test_factory_drop_guard + CREATE OR REPLACE VIEW ...guard are committed (per the file's own header comment, load.sql runs "committed") and never dropped afterward. Since existing mode is specifically meant to run against "a real pg_upgrade target" per the docs, this leaves a schema/view behind in that database indefinitely that isn't part of either extension. Might be intentional (cheap, harmless test residue), but worth a deliberate call — either document it as an accepted side effect of running the suite against a persistent target, or drop the schema at the end of the existing branch once the guard proofs are done.

Neither of these is a functional bug — the load-mode mechanism itself, the GUC plumbing, the security-context handling, and the dependency-guard proof logic all look correct and match the verification the author describes doing locally on PG12/PG17.

@jnasbyupgrade
jnasbyupgrade merged commit a6e4c6b into master Aug 8, 2026
20 checks passed
@jnasbyupgrade
jnasbyupgrade deleted the advanced-testing/foundation branch August 8, 2026 23:05
jnasbyupgrade added a commit that referenced this pull request Aug 8, 2026
head.repo.owner.login only distinguishes "who owns the fork" for
fork-headed PRs. For an upstream-branch-headed PR (base and head both in
this repo -- what `gh stack` requires), head.repo.owner.login is always
this repo's own org, never the actual author, so the gate silently skipped
review on every PR in this session's stack regardless of who opened it.

Confirmed via the check-runs API that claude-review's conclusion was
"skipped" on PRs #33/#34/#35 -- all legitimately jnasbyupgrade's own work,
recreated as upstream-branch PRs specifically so `gh stack` could link
them.

PR author can't be spoofed by a third party any more than head repo owner
can, and it's the more direct question for this gate's actual purpose:
trusting the PERSON asking for a review to run with this repo's secrets,
not the repository their branch happens to live in.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant