Skip to content

ci(site): test the pages in a real browser, check the live site, widen the em-dash check - #71

Merged
TMHSDigital merged 1 commit into
mainfrom
ci/site-smoke-and-live-checks
Sep 23, 2026
Merged

TMHSDigital merged 1 commit into
mainfrom
ci/site-smoke-and-live-checks

Conversation

@TMHSDigital

Copy link
Copy Markdown
Owner

These are the parts of #54 that were still open, plus #49, which sits in the same workflow file.

Fixes #49. Part of #54.

A browser smoke test that installs nothing (scripts/smoke_site.mjs)

  • It serves the assembled site at /plumbline/, as Pages does, and drives the runner's own Chrome over the DevTools protocol. It uses Node 22's built-in WebSocket, which the runner already has (v22.23 in the last run), so the site workflow keeps its no-install rule.
  • It checks the flows a reader actually uses:
    • the explainer's result card, and the worked example deriving the report's line;
    • an invalid input showing its message and setting aria-invalid;
    • a calculator run writing its inputs into the address;
    • a link carrying those inputs reproducing the result;
    • search opened from a doc page finding a section of that doc.
  • It fails on any console error, uncaught exception, or CSP violation, on every page, including the 404.
  • I checked that it catches what it is for. On a copy of the build with the old [object Object] bug put back and an inline script added to a doc page, it reported all three: the wrong status text, Chrome's CSP error, and the recorded violation.
  • It runs in site.yml's docs job, against the same unpacked archive the deploy publishes.

A check of the live site after deploy

  • A new live job, which needs deploy, runs check_site_links.mjs against the Pages URL. Only the live site can show that a missing path gets the 404 page with a 404 status.
  • It retries once after 60 s, because the CDN can lag a deploy.

A wider em-dash check

  • It now covers site/, scripts/ and .github/ as well as the markdown and src/. datasets/public/ and site/vendor/ stay excluded, since both hold vendored files.
  • It also catches the HTML entity and JavaScript escape spellings. Each is written in the pattern with a bracketed character, so the workflow does not match itself.
  • The tree is clean today. A probe file with —, — and — matched all three.

#49

  • The quickstart's traceback guard was grep -qv Traceback, which succeeds on any output with more than one line, so it could never fail. It now fails when the word appears. I checked it fires on a real traceback.

Left open in #54
Whether -- as prose punctuation should be banned too. It stands in for an em dash in 22 places, including METHODOLOGY and one report string in loader.py. Changing the report string means regenerating the example report, so that is better as its own decision.

🤖 Generated with Claude Code

…n the em-dash check

scripts/smoke_site.mjs serves the assembled site at /plumbline/, as Pages
does, and drives the runner's own Chrome over the DevTools protocol with
Node 22's built-in WebSocket, so the site workflow still installs
nothing. It checks the flows a reader uses: the explainer's result card
and derived example, an invalid input's message and aria-invalid, a
calculator run writing its inputs into the address, a link carrying them
reproducing the result, and search from a doc page. It fails on any
console error, uncaught exception, or CSP violation on every page.
Checked against a copy of the build with the old "[object Object]" bug
and an inline script put back: it names both.

A live job after deploy runs check_site_links.mjs against the Pages URL,
which is the only way to see that a missing path really returns the 404
page with a 404 status. It retries once, since the CDN can lag a deploy.

The em-dash check now covers site/, scripts/, and .github/, and the HTML
entity and JS escape spellings, written with bracketed characters so the
workflow does not match itself.

The quickstart's traceback guard used grep -qv, which succeeds on any
multi-line output, so it could never fail. It now fails when the word
appears (fixes #49).

Part of #54.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@TMHSDigital
TMHSDigital merged commit 219691d into main Sep 23, 2026
17 checks passed
@TMHSDigital
TMHSDigital deleted the ci/site-smoke-and-live-checks branch September 23, 2026 16:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CI's traceback guard can never fail

1 participant