Skip to content

Add test/build: extension-metadata tests + raw SQL syntax checks - #33

Merged
jnasbyupgrade merged 13 commits into
masterfrom
add-test-build
Aug 7, 2026
Merged

Add test/build: extension-metadata tests + raw SQL syntax checks#33
jnasbyupgrade merged 13 commits into
masterfrom
add-test-build

Conversation

@jnasbyupgrade

@jnasbyupgrade jnasbyupgrade commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds test/build, using pgxntool's existing PGXNTOOL_ENABLE_TEST_BUILD feature (this repo never had a test/build directory before), and removes test/sql/install.sql, whose packaging checks it makes redundant.

1. Remove test/sql/install.sql

Tested extension packaging (dependency declarations, clean install/uninstall) rather than test_factory's actual business logic. Deleted outright, along with the leading dependency-check block it left in test/sql/pgtap.sql (the rest of that file, which exercises tf.tap(), is untouched).

2. Add test/build/syntax.sql

Runs sql/test_factory.sql and sql/test_factory_pgtap.sql -- the actual source files a developer edits -- directly via \i (not CREATE EXTENSION), so a genuine syntax error is reported clearly instead of being obscured by a generic CREATE EXTENSION failure. Deliberately not the pgxntool-generated, version-suffixed copies of these files (sql/*--VERSION.sql): confirmed those are byte-identical except for one auto-generated header line, so there's no reason to resolve a version number here at all.

Known/expected errors baked into its expected output (documented in the file itself):

  • Two pg_extension_config_dump() can only be called from an SQL script executed by CREATE EXTENSION errors -- that function always errors when its containing script is \i'd directly instead of run via CREATE/ALTER EXTENSION, regardless of whether the SQL is otherwise valid.
  • A handful of NOTICE: type reference ... converted to text lines, harmless artifacts of the same bare-execution path.

ON_ERROR_ROLLBACK savepoints each statement so those expected errors don't abort the rest of the file, while a real syntax error still surfaces clearly on its own. VERBOSITY is knocked down from psql.sql's verbose to default since the verbose LOCATION line differs across PG majors, which would otherwise break matching a single expected-output file across this repo's PG10-18 matrix.

3. Wire it up

  • PGXNTOOL_ENABLE_TEST_BUILD = yes explicit in the Makefile (avoids silently disabling the check if test/build/'s contents were ever accidentally deleted).
  • pgxn-tools' pg-build-test never calls make test, so test-build (wired only into test's dependency chain) would never run in CI. Replaced with a direct make verify-results step, which also correctly detects failures that pg-build-test's make installcheck call cannot (pgxntool marks installcheck .IGNORE, so its exit code doesn't reflect real regression failures).

Test plan

  • make test passes on PG17 and PG12 (including the new test-build step)
  • Simulated CI's exact sequence from a fully clean state -- no pre-installed extension, no leftover roles/databases
  • Deliberately introduced a syntax error and confirmed test/build/syntax.sql surfaced it clearly and immediately; reverted before committing
  • test/build/expected/syntax.out generated from a real run (never hand-authored)

🤖 Generated with Claude Code

@coderabbitai

coderabbitai Bot commented Aug 4, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: b8f025af-fe40-4ed6-b4ff-1ea7cd89e71d

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Base automatically changed from sync-pgxntool-2.3.0 to master August 5, 2026 23:13
jnasbyupgrade and others added 8 commits August 5, 2026 18:13
Moves install-mechanics testing out of test/sql/install.sql and the top of
test/sql/pgtap.sql (which tested extension packaging -- dependency
declarations, clean install/uninstall -- not test_factory's actual
registering/getting business logic) into test/build/install.sql, using
pgxntool's test/build feature. Quoting its own purpose comment in
pgxntool/base.mk:

  Validates that extension SQL files are syntactically correct by running
  files from test/build/ through pg_regress. This provides better error
  messages than CREATE EXTENSION failures.

Also adds test/build/syntax.sql, which \i's the raw generated versioned SQL
files (sql/test_factory--0.5.0.sql, sql/test_factory_pgtap--0.1.0.sql)
directly rather than through CREATE EXTENSION, so a genuine syntax error
shows up immediately instead of being obscured by a generic CREATE EXTENSION
failure. Verified this works by deliberately introducing a typo locally and
confirming both new tests surfaced it clearly, then reverting.

Both new files hit known, harmless errors baked into their expected output
(see comments in each file for the full explanation):
  - pg_extension_config_dump() always errors when its script is \i'd
    directly instead of run via CREATE/ALTER EXTENSION.
  - `SET ROLE ""` fails because the role-restore GUC set via
    pg_catalog.set_config(..., true) is transaction-scoped, and plain \i
    (autocommit) gives each statement its own implicit transaction.
Also dials VERBOSITY down to "default" (from psql.sql's "verbose") since
verbose mode's backend source LOCATION lines differ across PG major
versions/builds, which would break single-expected-file matching across the
PG12-17 CI matrix.

Enables PGXNTOOL_ENABLE_TEST_BUILD explicitly (matching this Makefile's
existing preference for explicit-over-implicit config) and adds a CI step
that runs `make test-build` directly: pgxn-tools' pg-build-test invokes
`make installcheck`, never `make test`, so test-build's dependency chain
(wired only into `test`) would otherwise silently never run in CI.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
pgxntool/run-test-build.sh (vendored) shells out to rsync to sync
test/build/*.sql into test/build/sql/, but the pgxn/pgxn-tools container
image doesn't ship rsync -- caught by the first real CI run on this PR
("rsync: command not found", pgxntool/base.mk:473: test-build, Error 127).
Not reproducible locally since this dev container already has rsync.

Worth a pgxntool-test issue since any project enabling
PGXNTOOL_ENABLE_TEST_BUILD on this same image would hit the identical gap;
not fixed here (that's vendored, not this repo's own script).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Both raw SQL files print a query-result row for their
pg_catalog.set_config('..._role', current_user, true) call (the
original-role save at the top of each). current_user is whichever role
actually connects -- this dev container connects as root, CI's
pgxn/pgxn-tools connects as postgres -- so the expected output generated
locally didn't match CI's real run (confirmed by the first real CI run
after the rsync fix: "root" expected, "postgres" got, both PG13's own
failure and independently confirmed by a peer session's diagnosis of the
same run).

Fixed with `\o /dev/null`: discards normal query-result output for the rest
of the file, but NOTICE/WARNING/ERROR go through a separate stream `\o`
doesn't touch, so every error this test actually cares about -- including a
genuine future syntax error -- still shows up in the diff untouched.
Regenerated test/build/expected/syntax.out from a real run (never
hand-authored); confirmed clean on both PG12 and PG17.

Note: `make results` does not handle test/build's separate results/expected
directories (it only copies test/results/*.out -> test/expected/*.out) --
had to copy test/build/results/syntax.out -> test/build/expected/syntax.out
by hand, still from a real run, never typed by hand. Worth a pgxntool-test
issue: `make results` silently doesn't cover test-build's own output.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…rim comments

CI (.github/workflows/ci.yml): pg-build-test's own failure detection
(`make installcheck || status=$?`) can never actually trip -- pgxntool marks
`installcheck` .IGNORE, so `make installcheck` always exits 0 regardless of
real regression failures (verified locally with a deliberately broken
test). Confirmed from pg-build-test's actual source
(pgxn/docker-pgxn-tools bin/pg-build-test): `make all; sudo make install;
make installcheck || status=$?` -- it never calls `make test` either, so
test-build was never reachable through it regardless. Replaced with a
direct `make verify-results`, which inspects the real TAP output instead of
trusting an exit code, and pulls in test-build via the same `test`
dependency chain it already runs. Confirmed locally (4 repeated runs) that
a single `make verify-results` with no separate `make install` first is
deterministic once pg-build-test's own `MAKEFLAGS=-j $(nproc)` export is out
of the picture -- that export (not anything the container sets globally)
was the actual source of the install/installcheck race PR #23 found and
worked around; not re-litigating that fix there, just noting root cause
now confirmed.

test/build/syntax.sql: wrapped the \i calls in BEGIN/ROLLBACK with
ON_ERROR_ROLLBACK on, so nothing persists in the database whether this runs
under pg_regress or ad hoc locally (previously: autocommit, no rollback,
would leave a real mess in a developer's own database). Bonus: running the
whole thing as one transaction is also more faithful to how CREATE
EXTENSION actually behaves, so `current_setting('..._role')` now correctly
survives to the end of each file -- the third known/expected error (SET
ROLE "" from the role-restore code) is gone entirely, not just documented
away. Regenerated expected output from a real run.

Trimmed both test/build/*.sql files' comments substantially and added a
one-line cross-reference between them (install.sql tests packaging via a
real CREATE EXTENSION; syntax.sql tests raw SQL syntax by bypassing it) --
the previous version buried that distinction in multi-paragraph essays.

CLAUDE.md: noted that this PGXN distribution ships two structurally similar
extensions (test_factory, test_factory_pgtap) and that the test suite
shares infrastructure between them accordingly.

Verified on both PG12 and PG17: full make test passes, and a plain
`psql -f test/build/syntax.sql` against a scratch database leaves zero
test_factory objects behind afterward.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The previous commit (95d8be4) removed this based on local testing that
turned out to be invalid: a concurrent process in this dev container had
installed PostgreSQL 18, silently shifting the default `pg_config` on PATH
out from under an unrelated test, which produced a false "it works without
the extra step" result.

Re-tested properly (PG_CONFIG pinned to match the actual target cluster,
extension deliberately removed from the system first) and found a REAL,
pre-existing pgxntool bug, independent of anything in this PR: on a
genuinely fresh system, `make test`/`make verify-results` runs the main
suite's `installcheck` before `install` ever copies the extension's files
into place. Root cause: `TEST_DEPS` textually ends up as `testdeps
check-stale-expected test-build install installcheck`, but
`check-stale-expected: installcheck` is its own direct dependency edge --
Make resolves that early (2nd in the list), running the main suite against
a not-yet-installed extension, well before it ever reaches the literal
`install installcheck` pair at the end. `test-build` avoids this by luck,
not shared design: it declares its own `test-build: install`, which
protects only itself.

Filed as Postgres-Extensions/pgxntool-test#62 (not fixed in the vendored
copy here). Also corrects the record on PR #23's similar-looking fix: that
one attributed the same symptom to "ambient parallel make" from
pg-build-test's `MAKEFLAGS=-j $(nproc)` export -- confirmed here, with a
plain serial `make -n` dry run (no -j anywhere), that the real cause is
this TEST_DEPS ordering issue instead. The fix (explicit `make install`
first) happened to be correct either way.

Reproduced 3/3 times with the extension deliberately removed from
/usr/share/postgresql/*/extension first, and confirmed fixed 3/3 times,
on both PG12 and PG17, with PG_CONFIG correctly pinned throughout this
time.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…change

Rebasing onto sync-pgxntool-2.3.0 (built on current upstream master)
pulls in upstream PR #20 ("Drop the original_role temp table from
install"), which this branch's fork-based master predated. That commit
only touched sql/*.sql, not test/, but it shifts every subsequent line
of the auto-generated sql/test_factory--0.5.0.sql by a constant offset
-- and test/build/syntax.sql's expected output embeds literal line
numbers from psql's \i output. Regenerated from a real run (test/build
has no `make results` equivalent to gate this the way test/expected/
does); confirmed the diff is a pure line-number shift, not a content
change: verified locally that a deliberately broken test/build syntax
error is still caught immediately and clearly.
pgxntool 2.3.0 (sync-pgxntool-2.3.0, this branch's new base) adds a
real installcheck: install dependency edge to base.mk, fixing the
ordering bug this workaround existed for (Postgres-Extensions/pgxntool-test#62).
Reproduced the original failure once more on this rebased branch to
confirm the fix holds standalone: deleted test_factory's installed
files from /usr/share/postgresql/17/extension/, ran the now-simplified
`make verify-results` alone with no prior `make install`, and it
passed -- where the old base required the explicit install step first.
Same root cause as the syntax.out regeneration two commits back: these
files were authored on this branch's original fork-based master, which
predates upstream PR #24 (adds the sql-lint tool and its
comment-stacked-dashes rule, 3+ consecutive -- lines must use /* */).
Rebasing onto sync-pgxntool-2.3.0 makes the linter apply to them for
the first time.

Watch for the same nested-block-comment hazard PR #22's own comment
conversion hit (see commit 6a0627d "Convert multi-line SQL comments to
/* */ style"): syntax.sql's original wording included the literal
substring "sql/*--*.sql", where the "/*" opens a second, unwanted
nested comment (Postgres block comments nest) -- reworded to avoid any
"/*"/"*/" substrings appearing inside comment bodies, not just at the
intended block boundaries.
jnasbyupgrade and others added 5 commits August 6, 2026 15:57
test/build exists for exactly one purpose: running extension scripts
"bare" via \i, so a genuine SQL syntax error is reported clearly instead of
being obscured by a generic CREATE EXTENSION failure. Its results are
always meant to be thrown away -- unlike test/install, which is intended
to commit and persist for the rest of the suite.

test/build/install.sql (the CASCADE-pulls-in-dependency, bare-create-fails,
clean-drop checks) was a different kind of test entirely: packaging/
dependency-declaration correctness via a real CREATE EXTENSION. Putting it
in test/build bought nothing over test/install (which already owns getting
the extension installed) and confused the two directories' actual purposes
-- test/build/install.sql is also just a name collision with test/install/
waiting to trip someone up.

This content moves to test/install/load.sql (which will actually perform
the CASCADE install in fresh mode too, once that lands) in a follow-up
commit on advanced-testing/foundation -- not duplicated here. test/build
now contains exactly the one file it was designed for.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Caught in review: the previous commit's git add -A picked up this
asciidoctor-generated build artifact (only doc/*.asc is meant to be
tracked -- confirmed never committed before on master). Removed from
tracking and added a gitignore pattern so a future git add -A can't
repeat this.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
… in ci.yml

test/build/syntax.sql hardcoded sql/test_factory--0.5.0.sql and
sql/test_factory_pgtap--0.1.0.sql. A real version bump would silently break
this the moment the old versioned file is removed, or worse, keep testing a
stale version forever if it isn't. Resolve both filenames from
pg_available_extensions.default_version instead (the same technique
test/install/load.sql already uses for its own version assertions) --
those catalog rows reflect whatever's in the .control files test-build's
own `install` dependency just put in place, so there's nothing to remember
to update alongside a version bump.

ci.yml's CI-job comment used "pgxn-tools'" (the pgxn/pgxn-tools CONTAINER
IMAGE this job runs in, a completely unrelated project) right next to
"pgxntool" (this repo's own vendored Make-based build system) without ever
spelling out that distinction -- reads as a typo of the same name instead
of two different tools. Made every reference to the container image
explicit ("CONTAINER IMAGE") and clarified pgxntool/run-test-build.sh,
pgxntool 2.3.0, and make verify-results are pgxntool's own.
…ffixed copies

Reverts the version-resolution approach from the previous commit entirely,
rather than just fixing its hardcoded version number. sql/test_factory.sql
and sql/test_factory--0.5.0.sql are byte-identical except for one
auto-generated "DO NOT EDIT" header line -- there was never a reason to
reference the generated, version-suffixed copy here at all, let alone
resolve its version dynamically. \i sql/test_factory.sql directly: no
version lookup, no version to go stale, nothing to resolve.

Also hit the same nested-block-comment gotcha documented in
test/install/load.sql while writing the explanatory comment here: writing
out the generated filename's glob pattern with a literal wildcard right
after a slash opens an unbalanced nested comment and silently swallows the
rest of the file.

Regenerated test/build/expected/syntax.out to match (filenames and line
numbers shift since sql/test_factory.sql lacks the generated header line).
make results has no copy mechanism for test/build's separate
results/expected dirs (confirmed in pgxntool/base.mk), so this is a direct
copy from a verified-passing real test/build/results/syntax.out.
The previous comment was long enough, and placed early enough (right
before the "Install rsync" step, which it has nothing to do with), that it
read as an explanation of rsync rather than of the later "make
verify-results" step it actually justifies. State the reasons briefly and
put the comment directly above the step it explains instead.
jnasbyupgrade added a commit that referenced this pull request Aug 6, 2026
head.repo.owner.login only distinguishes "who owns the fork" for
fork-headed PRs. For an upstream-branch-headed PR (base and head both in
this repo -- what `gh stack` requires), head.repo.owner.login is always
this repo's own org, never the actual author, so the gate silently skipped
review on every PR in this session's stack regardless of who opened it.

Confirmed via the check-runs API that claude-review's conclusion was
"skipped" on PRs #33/#34/#35 -- all legitimately jnasbyupgrade's own work,
recreated as upstream-branch PRs specifically so `gh stack` could link
them.

PR author can't be spoofed by a third party any more than head repo owner
can, and it's the more direct question for this gate's actual purpose:
trusting the PERSON asking for a review to run with this repo's secrets,
not the repository their branch happens to live in.
@jnasbyupgrade
jnasbyupgrade merged commit 1aeb68e into master Aug 7, 2026
20 checks passed
@jnasbyupgrade
jnasbyupgrade deleted the add-test-build branch August 7, 2026 18:42
jnasbyupgrade added a commit that referenced this pull request Aug 7, 2026
head.repo.owner.login only distinguishes "who owns the fork" for
fork-headed PRs. For an upstream-branch-headed PR (base and head both in
this repo -- what `gh stack` requires), head.repo.owner.login is always
this repo's own org, never the actual author, so the gate silently skipped
review on every PR in this session's stack regardless of who opened it.

Confirmed via the check-runs API that claude-review's conclusion was
"skipped" on PRs #33/#34/#35 -- all legitimately jnasbyupgrade's own work,
recreated as upstream-branch PRs specifically so `gh stack` could link
them.

PR author can't be spoofed by a third party any more than head repo owner
can, and it's the more direct question for this gate's actual purpose:
trusting the PERSON asking for a review to run with this repo's secrets,
not the repository their branch happens to live in.
jnasbyupgrade added a commit that referenced this pull request Aug 7, 2026
head.repo.owner.login only distinguishes "who owns the fork" for
fork-headed PRs. For an upstream-branch-headed PR (base and head both in
this repo -- what `gh stack` requires), head.repo.owner.login is always
this repo's own org, never the actual author, so the gate silently skipped
review on every PR in this session's stack regardless of who opened it.

Confirmed via the check-runs API that claude-review's conclusion was
"skipped" on PRs #33/#34/#35 -- all legitimately jnasbyupgrade's own work,
recreated as upstream-branch PRs specifically so `gh stack` could link
them.

PR author can't be spoofed by a third party any more than head repo owner
can, and it's the more direct question for this gate's actual purpose:
trusting the PERSON asking for a review to run with this repo's secrets,
not the repository their branch happens to live in.
jnasbyupgrade added a commit that referenced this pull request Aug 7, 2026
head.repo.owner.login only distinguishes "who owns the fork" for
fork-headed PRs. For an upstream-branch-headed PR (base and head both in
this repo -- what `gh stack` requires), head.repo.owner.login is always
this repo's own org, never the actual author, so the gate silently skipped
review on every PR in this session's stack regardless of who opened it.

Confirmed via the check-runs API that claude-review's conclusion was
"skipped" on PRs #33/#34/#35 -- all legitimately jnasbyupgrade's own work,
recreated as upstream-branch PRs specifically so `gh stack` could link
them.

PR author can't be spoofed by a third party any more than head repo owner
can, and it's the more direct question for this gate's actual purpose:
trusting the PERSON asking for a review to run with this repo's secrets,
not the repository their branch happens to live in.
jnasbyupgrade added a commit that referenced this pull request Aug 8, 2026
head.repo.owner.login only distinguishes "who owns the fork" for
fork-headed PRs. For an upstream-branch-headed PR (base and head both in
this repo -- what `gh stack` requires), head.repo.owner.login is always
this repo's own org, never the actual author, so the gate silently skipped
review on every PR in this session's stack regardless of who opened it.

Confirmed via the check-runs API that claude-review's conclusion was
"skipped" on PRs #33/#34/#35 -- all legitimately jnasbyupgrade's own work,
recreated as upstream-branch PRs specifically so `gh stack` could link
them.

PR author can't be spoofed by a third party any more than head repo owner
can, and it's the more direct question for this gate's actual purpose:
trusting the PERSON asking for a review to run with this repo's secrets,
not the repository their branch happens to live in.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant