Skip to content

Regenerate test fixtures during release - #83

Open
sophieleewu wants to merge 1 commit into
devfrom
fix/regenerate-fixtures-on-release
Open

Regenerate test fixtures during release#83
sophieleewu wants to merge 1 commit into
devfrom
fix/regenerate-fixtures-on-release

Conversation

@sophieleewu

Copy link
Copy Markdown
Contributor

Problem

release.yml runs yarn build, which regenerates generated/ from the live schema at https://api.us-west-2.fragment.dev/schema.graphql. The four generated clients under tests/fixtures/ are regenerated by a different command — yarn update-test-schema — which the release workflow never ran.

Both read the same live schema, so they only agree when they run together:

command rewrites
yarn buildgen-clients generated/
yarn update-test-schema the 4 clients in tests/fixtures/

updateSDKQueries.yml runs both. release.yml ran only the first.

What that caused

If the schema changes between the last Update SDK queries run and a release, the release commits a fresh generated/ next to stale fixtures. tests.yml verifies both sets, so the release PR fails:

  • Verify generated methods are up-to-date → passes, the release rebuilt it
  • Verify test schema generated files are up-to-datefails, nothing rebuilt it

This hit the 2.2.0 release (#82). The LedgerEntryInput.typeVersion doc comment changed server-side in the ~15 minutes between #80 merging (17:38 UTC, everything consistent) and the release running (17:53 UTC):

- /** Experimental: This field is reserved for an upcoming feature and is not yet supported. */
+ /** The version of the Ledger Entry type to post. Defaults to 1. */

generated/generated.ts picked up the new wording; all four fixtures kept the old one. The fixtures had to be regenerated by hand to unblock the release.

Fix

Run yarn update-test-schema alongside yarn build so both sets of generated files come from the same fetch of the schema, and install the Fragment CLI, which scripts/update-test-schema.sh needs for fragment gen-graphql (matching how updateSDKQueries.yml installs it).

peter-evans/create-pull-request already commits everything in the working tree, so any regenerated fixtures ride along in the release PR automatically.

Notes

  • Verified locally that the codegen is idempotent — a second run is byte-identical — so this adds no churn when the schema hasn't changed.
  • tests/fixtures/test-schema-queries.graphql comes from the static test-schema.json, not the live schema, so it doesn't drift; it's regenerated here anyway since it's part of the existing script.
  • Separately, PR Bump SDK version (2.2.0) #81's runs showed flaky 5s vitest timeouts on live-API tests. Not addressed here.

🤖 Generated with Claude Code

`yarn build` regenerates `generated/` from the live schema at
https://api.us-west-2.fragment.dev/schema.graphql, but the four generated
clients under `tests/fixtures/` are only refreshed by `yarn update-test-schema`,
which the release workflow never ran. Both read the same live schema, so they
match only when they run together.

When the schema changed between the last `Update SDK queries` run and a
release, the release committed a fresh `generated/` alongside stale fixtures.
`tests.yml` verifies both, so the release PR failed on "Verify test schema
generated files are up-to-date" while "Verify generated methods" passed. This
happened on the 2.2.0 release (#82), where a `typeVersion` doc comment changed
in the 15 minutes between #80 merging and the release running.

Run `yarn update-test-schema` alongside `yarn build` so both sets of generated
files come from the same fetch of the schema, and install the Fragment CLI,
which the script needs for `fragment gen-graphql`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant