Skip to content

Publish over R2's S3 API; require the CDN to serve the uploaded pair - #20

Merged
dweekly merged 2 commits into
mainfrom
fix/r2-s3-publish
Nov 2, 2026
Merged

dweekly merged 2 commits into
mainfrom
fix/r2-s3-publish

Conversation

@dweekly

@dweekly dweekly commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

Summary

The R2 credential for this bucket is an S3-style key pair (access key ID, secret, endpoint), which Wrangler can't use, so the weekly publish from #18 could never authenticate. This switches CI publishing to R2's S3 API and fixes a verification gap found while testing: the script could report success after a failed upload.

Changes

  • Scripts/publish-database.sh: uploads with aws s3api put-object when R2_ACCESS_KEY_ID, R2_SECRET_ACCESS_KEY and R2_ENDPOINT are all set (exit 64 if only some are), and with Wrangler otherwise, so maintainer machines are unchanged.
  • Verification fix: after upload it used to check only that the CDN's pair matched itself, so a failed upload passed on the previous pair. It now requires the CDN to serve the exact uploaded SHA-256s, retrying every 30 s for up to 6 min to cover the 300 s cache.
  • publish-database.yml (new): workflow_call from the weekly update after merge, and workflow_dispatch with a ref to publish main now or roll back to an earlier commit.
  • CONTRIBUTING and CHANGELOG updated.

Verification

With stub aws and wrangler binaries on PATH (nothing uploaded):

Case Result
Only R2_ACCESS_KEY_ID set exit 64 with message
Stub upload, CDN still serves older pair exit 65, "CDN still serves a different database" (old script: exit 0)
Stub upload, CDN serves the local pair (aws path) exit 0; put-object args: bucket, key, body, content type, cache-control, AWS_REGION=auto
Same, Wrangler path exit 0; same wrangler r2 object put calls as before

shellcheck and actionlint are clean apart from the existing SC2129 note.

Not yet verified: a real upload with the R2 keys. That happens when publish-database.yml is dispatched once the three secrets are set.

🤖 Generated with Claude Code

https://claude.ai/code/session_01A8mvWkYd1YBhFyvbFx7UF7

dweekly and others added 2 commits November 2, 2026 11:32
The repo's R2 credential is an S3 access key pair, which Wrangler cannot
use. publish-database.sh now uploads with the aws CLI when R2_ACCESS_KEY_ID,
R2_SECRET_ACCESS_KEY and R2_ENDPOINT are set, and with Wrangler otherwise.

Post-upload verification checked only that the CDN pair was internally
consistent, so a failed upload would pass on the previous pair. It now
requires the CDN to serve the exact uploaded bytes, retrying past the
300 s cache.

Publishing moves to publish-database.yml (workflow_call from the weekly
update, or workflow_dispatch with a ref to publish or roll back).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A8mvWkYd1YBhFyvbFx7UF7
Rollback dispatches with an older ref checked out that commit's
publish script, which predates S3 support and fails (Codex P1). Check
out the publisher from the workflow's own revision and only the
Resources directory from ref, and verify the pair before publishing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A8mvWkYd1YBhFyvbFx7UF7
@dweekly
dweekly merged commit c40f887 into main Nov 2, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant