Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
110 changes: 68 additions & 42 deletions .github/workflows/publish-site.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,10 +31,16 @@ on:
- 'scripts/release-notes.ps1'
- '.github/workflows/publish-site.yml'
# A release changes what the manifests say without changing anything in site/, so it has
# to redeploy the page as well. This is what keeps the download links current.
# to redeploy the page as well. This is what keeps the download links current. It does
# not deploy from here, though - see the redeploy job.
release:
types: [published]
workflow_dispatch:
inputs:
tag:
description: 'Release tag this deployment is for. Leave empty unless deploying for a release.'
required: false
type: string

permissions:
contents: read
Expand All @@ -48,8 +54,35 @@ concurrency:
cancel-in-progress: false

jobs:
# A Pages deployment made from a tag does not become the one the site serves unless it is
# the first deployment of that commit - and by the time a release is published, that
# commit has almost always been deployed from main already. The deployment is created,
# reported as succeeded, and never served. Every release ran into this; deploying a second
# time from the same tag, which is what this workflow used to do, did not help either.
#
# A deployment from a branch supersedes whatever is live, first or not. So a release does
# not deploy: it asks this same workflow to run on main, and that run deploys. The tag
# goes with it, because the manifests still have to wait for the release to appear.
#
# workflow_dispatch is one of the two events GitHub deliberately exempts from the rule
# that GITHUB_TOKEN cannot start another workflow run, so this needs no personal token.
redeploy:
name: Redeploy the site from main
if: github.event_name == 'release'
runs-on: ubuntu-latest
permissions:
actions: write
steps:
- name: Ask for a deployment from main
shell: bash
run: gh workflow run publish-site.yml --repo "$GITHUB_REPOSITORY" --ref main --field "tag=$RELEASE_TAG"
env:
GH_TOKEN: ${{ github.token }}
RELEASE_TAG: ${{ github.event.release.tag_name }}

deploy:
name: Deploy to GitHub Pages
if: github.event_name != 'release'
runs-on: ubuntu-latest
environment:
name: github-pages
Expand All @@ -59,28 +92,42 @@ jobs:

# Generated into the checkout, so the manifests ship with the page in one
# deployment. Both files are gitignored: this is the only thing that writes them.
# ExpectTag is empty for every trigger except a release, and that is the point: a
# ExpectTag is empty except when deploying for a release, and that is the point: the
# release event fires seconds before the releases API lists the release, so without
# it this step reads the previous version and writes a stale manifest while
# reporting success. That happened to 0.9.5.
# reporting success. That happened to 0.9.5. The tag arrives as an input because the
# release itself does not deploy - the redeploy job passes it here.
- name: Generate the release manifests
shell: pwsh
run: ./scripts/build-release-manifests.ps1 -Repository "$env:GITHUB_REPOSITORY" -OutputFolder site -Token "$env:GH_TOKEN" -ExpectTag "$env:EXPECT_TAG"
env:
GH_TOKEN: ${{ github.token }}
EXPECT_TAG: ${{ github.event.release.tag_name }}
EXPECT_TAG: ${{ inputs.tag }}

# A released version with no entry in CHANGELOG.md would publish a What's new page
# that does not mention the version people are being offered. Checked here as well as
# on the pull request that bumps VersionPrefix, because a release does not have to
# have come from one. Pre-releases are exempt: Dev builds come from every merge and
# are deliberately not in the changelog.
- name: Check this release has notes
if: github.event_name == 'release' && github.event.release.prerelease == false
# have come from one.
#
# The shape of the tag is what says whether it is a pre-release: the pipeline gives
# released versions v<major>.<minor>.<patch> and Dev builds a -dev.<build> suffix.
# Dev builds come from every merge and are deliberately not in the changelog.
- name: Check the released version has notes
shell: pwsh
run: ./scripts/release-notes.ps1 -Mode Verify -Version "$env:RELEASE_TAG"
run: |
if (-not $env:RELEASE_TAG) {
Write-Host 'Not deploying for a release, so there is nothing to check.'
return
}

if ($env:RELEASE_TAG -notmatch '^v\d+\.\d+\.\d+$') {
Write-Host "$env:RELEASE_TAG is a pre-release; the changelog covers released versions only."
return
}

./scripts/release-notes.ps1 -Mode Verify -Version $env:RELEASE_TAG
env:
RELEASE_TAG: ${{ github.event.release.tag_name }}
RELEASE_TAG: ${{ inputs.tag }}

# Written into the page rather than fetched from the API by the page itself: the
# notes are then in the HTML that ships, so they are readable without scripting and
Expand All @@ -89,47 +136,26 @@ jobs:
shell: pwsh
run: ./scripts/release-notes.ps1 -Mode Html

- uses: actions/configure-pages@v5
- uses: actions/configure-pages@v6

- uses: actions/upload-pages-artifact@v3
# v4 of this action stopped putting dotfiles in the artifact. site/ has none, so
# nothing is silently dropped; if one is ever added, it needs include-hidden-files.
- uses: actions/upload-pages-artifact@v5
with:
path: site

- id: deployment
uses: actions/deploy-pages@v4
uses: actions/deploy-pages@v5

# A Pages deployment can report success, be recorded as the active deployment, and
# still leave the site serving the previous one. Every step above went green for
# 1.0.0 while the download page offered 0.9.5, and nothing said so. Reading the site
# back is the only place that failure is visible.
#
# It is neither rare nor random: it is what a release does to itself. A Pages
# deployment is identified by commit SHA - deploy-pages sends the SHA as
# pages_build_version and then polls a status keyed on that same SHA - and a release
# redeploys a commit Pages already holds, because the release tag, the pre-release
# tag and main are all one commit. The second deployment of that commit is reported
# as succeeded, since the status read back belongs to the first one, and is then
# never served. That is why release-triggered runs failed while pushes and manual
# runs passed, and why re-running by hand always fixed it.
# back is the only place that failure is visible, so it fails the run that caused it
# rather than waiting to be noticed.
#
# So the first look is deliberately short - a deployment that is going to appear is
# serving within about ten seconds, and a long wait here only delays the thing that
# actually fixes it - and the deploy that a person used to do by hand is done here.
- id: firstlook
name: Verify the site is serving this deployment
continue-on-error: true
shell: pwsh
run: ./scripts/verify-published-site.ps1 -Folder site -TimeoutSeconds 90

- id: redeployment
name: Deploy again
if: steps.firstlook.outcome == 'failure'
uses: actions/deploy-pages@v4

# The full wait this time. If the same files are still not served after a second
# deployment then something other than the collision is wrong, and the run says so
# rather than passing quietly.
- name: Verify the second deployment
if: steps.firstlook.outcome == 'failure'
# Deploying a second time from the same run was tried as a way to recover from that
# and does not work: dev 3286 deployed twice from its tag and the site took neither.
# Only the ref matters, which is what the redeploy job is for.
- name: Verify the site is serving this deployment
shell: pwsh
run: ./scripts/verify-published-site.ps1 -Folder site
38 changes: 23 additions & 15 deletions docs/release-management.md
Original file line number Diff line number Diff line change
Expand Up @@ -190,21 +190,29 @@ from the live domain after deploying and fails if they are not the ones it just
release, because a pre-release deployment leaves `stable.json` untouched and that is
correct.

**Why that kept happening on releases, and what the workflow now does about it.** It was
not bad luck. A Pages deployment is identified by commit SHA: `deploy-pages` sends the SHA
as `pages_build_version` and then polls a deployment status keyed on that same SHA. A
release redeploys a commit Pages already holds — the release tag, the pre-release tag and
`main` are all one commit — so the second deployment of that commit reads the *first* one's
status, reports success within seconds, and is then never served. The numbers said so
plainly: pushes passed 9 out of 9 and manual runs 8 out of 8, while release-triggered runs
passed 7 out of 23, and the stalled deployment sat at `in_progress` for five minutes where
a healthy one reaches `success` in about ten seconds.

Deploying the same files again clears it, so the workflow does that itself rather than
waiting to be asked: a short first check, a second `deploy-pages` if the site is still
stale, then the full check. A run that fails now has failed twice, and the collision is not
the explanation — look under **Settings → Environments → `github-pages`** for a deployment
stuck in progress on that commit.
**Why that kept happening on releases: the ref, not the repetition.** A Pages deployment
made from a **tag** is not served unless it is the first deployment of that commit. A
deployment from a **branch** supersedes whatever is live, first or not. By the time a
release is published, its commit has usually been deployed from `main` already — the tag,
the pre-release tag and `main` are all one commit — so the release's own deployment is
created, reported as succeeded by `deploy-pages`, and never served.

The evidence, once the verify step existed to make it visible: pushes passed 9 of 9 and
manual runs 8 of 8, while release-triggered runs passed 7 of 23. Commit `f169379` settles
it — deployed from `main` at 13:59 it went live; deployed from tag `v1.2.2-dev.3286` at
14:20 it stalled, and stalled again on a second attempt in the same run. Deploying twice
was tried as a fix and does not work, because both attempts are on the same tag.

So **a release does not deploy the site.** The `redeploy` job asks this same workflow to
run on `main`, passing the release tag as an input so the manifests still wait for the
release to appear. `workflow_dispatch` is one of the two events GitHub exempts from the
rule that `GITHUB_TOKEN` cannot start another workflow run, so this needs no personal
access token. A release therefore produces two runs: a three-line dispatcher on the tag,
and the deployment on `main`.

A failure in the verify step should now mean something genuinely new. Re-run **Publish
site** on `main` with the tag left empty; if that fails too, look under **Settings →
Environments → `github-pages`** for a deployment stuck in progress on that commit.

### Azure Pipelines

Expand Down
28 changes: 15 additions & 13 deletions scripts/verify-published-site.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -16,15 +16,15 @@
deployment leaves stable.json untouched, which is correct, and a check written
against the newest release would fail on every one of them.

The usual cause is a release redeploying a commit Pages already holds: a Pages
deployment is identified by commit SHA, and the release tag, the pre-release tag and
main are all one commit, so the second deployment of it reports the first one's status
and is never served. The workflow answers that by deploying again, which is why this
runs twice there - once on a short timeout before the retry, once on the full one
after.
The cause was found in September 2026: a Pages deployment made from a tag is not served
unless it is the first deployment of that commit, and by the time a release is
published the commit has usually been deployed from main already. Deploying a second
time from the same tag does not help - dev 3286 did exactly that and the site took
neither. publish-site.yml therefore deploys from main and never from a release tag, so
a failure here should now mean something genuinely new.

Read-only. It cannot repair a deployment - the remedy is to deploy the same files
again - so its whole job is to make a silent staleness loud.
Read-only. It cannot repair a deployment - the remedy is to deploy again from main - so
its whole job is to make a silent staleness loud.

.PARAMETER Folder
Folder holding the manifests that were deployed, and the CNAME naming where they were
Expand Down Expand Up @@ -120,9 +120,11 @@ throw @"
$BaseUrl is still serving an older deployment after $TimeoutSeconds seconds ($stale).

The deployment this run made succeeded, so the files are right and the site has not
picked them up. Deploying the same files again clears it, and the workflow already does
that once by itself - so if this is the second attempt, the collision it works around is
not the whole story. Run it again by hand - Actions -> Publish site -> Run workflow - and
look for a deployment still in progress on this commit under the github-pages
environment.
picked them up.

Deployments from a release tag behave this way and are the reason this check exists, but
publish-site.yml no longer makes any - it deploys from main - so this is something else.
Run it again by hand: Actions -> Publish site -> Run workflow, on main, leaving the tag
empty. If that also fails, look under Settings -> Environments -> github-pages for a
deployment still in progress on this commit.
"@