Skip to content

feat(scripts): a dense pull request gets a readable front page - #1615

Open
fairchild wants to merge 5 commits into
mainfrom
workspace/claude-1613-pr-review-page
Open

feat(scripts): a dense pull request gets a readable front page#1615
fairchild wants to merge 5 commits into
mainfrom
workspace/claude-1613-pr-review-page

Conversation

@fairchild

@fairchild fairchild commented Sep 12, 2026

Copy link
Copy Markdown
Owner

Summary

Mergeability

  • Surface: agent-runtime / scripts — scripts/pr-review-page.py (new), scripts/upload-evidence.py (one extension and its content type), fixtures and tests under scripts/tests/
  • User-facing behavior changed: No for the app. New operator tooling: a PR can now carry a Review page: line, and .html is an uploadable evidence type.
  • Non-happy paths considered: unavailable is reported as unavailable, never as none, for reviews, review threads and checks alike; a PR with no Summary falls back to its title; a marker and fence nested inside an outer HTML comment carry no diagram, because GitHub shows that block to nobody; only http: and https: become an href, and an <img> only for https: on the evidence store or GitHub, which is what the page's own policy allows — anything else is shown as its address in text; the head is re-read before the upload and before the body edit, and a head that moved refuses both; a hostile filename cannot close a mermaid label; --head refuses to build a page that would claim a head the PR does not have; --link re-reads the body at write time and rewrites its own line rather than adding a second; with no renderer present the page shows the escaped diagram source and says it did not draw it.
  • Release/ops preconditions: None. Nothing here runs in the release lane; --upload needs EVIDENCE_UPLOAD_TOKEN, which evidence.sh already sources.
  • Residual risk or follow-up: Authored diagrams are allowlisted, so a fence outside the subset (boxes, arrows, words) is shown as escaped source rather than drawn — a deliberate trade of a picture for keeping the build host's filesystem out of a published PNG. The evidence store serves uploaded HTML as active pages on its shared origin with no content security policy #1616 — the evidence store serves uploaded HTML as an active page on a shared origin with no response headers of its own; this page's own <meta> policy is the only layer until that lands. The grouping heuristic can file a hunk under a neighbouring sentence (Give WorkspaceServiceTests each their own preferences domain instead of UserDefaults.standard #1602: 5 hunks under the first bullet, 22 under the second, 1 under the third), which Factory step: build, host, and link the PR review page at every ready flip of an April PR #1614's model turn is what fixes. The review-thread query reads the first 100 and does not page; the page says so when it hits that cap. --disable-remote-fonts is documented in the renderer's flags as intended-but-absent: mermaid-cli 11.12 fails every render with it, and a closed network denies remote fonts anyway. .gitattributes in the fixture directory turns off the whitespace check there, because a recorded diff's blank context line is a single space.

Validation

  • uv run --script scripts/tests/test_pr_review_page.pyRan 72 tests in 29.647s / OK; with mmdc off PATH, the path CI takes, Ran 72 tests in 0.050s / OK (skipped=6)
  • uv run --script scripts/tests/test_upload_evidence.pyRan 12 tests in 0.009s / OK
  • for f in scripts/tests/test_*.py; do uv run --script "$f"; done — 52 suites, every one exit 0
  • python3 scripts/check-subprocess-timeouts.pyOK: no un-timed ProcessRunner.run calls in 318 files
  • uv run --script scripts/pr-readiness.py --body-file <this> --base origin/mainPR readiness passed.
  • git diff --cached --check — clean
  • curl -s -D - on the hosted specimen — HTTP/2 200, content-type: text/html; charset=utf-8; the fetched bytes are identical to the local build
  • The built specimen carries one CSP, one PNG data URI, zero <svg> elements and zero <script> tags

Performance

  • Not a performance-sensitive change

Evidence

  • UI evidence attached (screenshot or recording from the exact commit under review)
  • Tests named below: the command and the line it printed

The specimen page, fetched from its hosted URL and rendered in Chrome: https://evidence.cloudcompute.com/workspaces/pr-1613/27-GHucz5wfotPnTe0G5Sg/20260912-060005-specimen-1602-review-page-browser-v3.png

The synthetic fixture's page, which covers an inline evidence image, an open review thread, a red check and the "Everything else" group: https://evidence.cloudcompute.com/workspaces/pr-1613/dHRx2kNMb6g9fFCozX2p3A/20260912-040055-synthetic-fixture-page-browser.png

Evidence Status

Blockers

  • None

🤖 Generated with Claude Code

https://claude.ai/code/session_01T3CL23nN7Fmu8mE5TB9CdT

Orchestrator note (workspaces)

scripts/pr-review-page.py builds one self-contained HTML page per PR: the
Summary's own sentences as plain language, a diagram of the shape, the diff
grouped under the sentence that explains it, the evidence shown rather than
linked, and where the PR stands. It reads the PR through gh or a recorded
fixture, uploads the page to the evidence store, and writes `Review page: <url>`
under the body's byline leaving every other byte alone.

The evidence store serves an uploaded .html as text/html, so the page hosts
directly; upload-evidence.py just needed the extension and its content type.

Grouping is a heuristic and says so: a bullet claims a hunk by the names it
shares with what the hunk adds, a cited line range claims outright, and what
nothing claims lands under "Everything else". The prose all comes from one
function, so the factory turn that writes better sentences (#1614) changes that
function and nothing else.

Closes #1613

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3CL23nN7Fmu8mE5TB9CdT
@fairchild fairchild added the author:claude-code PRs authored by the claude-code agent (interactive Claude Code sessions) label Sep 12, 2026
@vercel

vercel Bot commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
workspaces Ready Ready Preview Sep 12, 2026 6:01am UTC

Request Review

Comment thread scripts/pr-review-page.py Fixed
april-clearwater[bot]
april-clearwater Bot previously approved these changes Sep 12, 2026

@april-clearwater april-clearwater Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

April Clearwater, Application Lead

🟡 Approve with follow-ups — solid tool, two things worth tracking

scripts/pr-review-page.py turns a dense PR (the #1602 specimen) into a page a person reads in minutes, and all three requested-evidence items from #1613 are answered with specific, checkable detail rather than assertion.

  • Worth noting: the diff handed to me cut off at a line limit partway through the pr.json fixture, so I never saw the actual diffs for scripts/upload-evidence.py, scripts/tests/test_pr_review_page.py, or scripts/tests/test_upload_evidence.py — only the generator itself, the docs, and part of the fixture. My code read below is scoped to what was visible.
  • Worth noting: the grouping heuristic in group_hunks/claim_score is honest about its own limits (already flagged in the PR body and tracked as a follow-up in #1614) — a bullet claims a hunk by shared tokens with what it adds, which is approximate by design.
  • Worth noting: body_with_link's insertion point only inspects the first non-blank line as a possible byline; if that line isn't italic, the link goes at line 1, which is fine for this repo's convention but silent about any other case.

Evidence: ✅ all three items from #1613 are [complete] with specifics — hosted URL plus HTTP/2 200/content-type: text/html, before/after test counts (27 failing → 31 passing), and 52/52 suites exit 0. This is a visual product (a page someone reads) plus python logic, so a screenshot for the former and named test counts for the latter is the right bar, and both are here. I can't fetch the hosted screenshot myself from this runner (no browser/network tool), but the author's own check (curl -D -, byte-identical fetch) is the kind of verification that satisfies the gate.

Code read of pr-review-page.py and docs
  • claim_score/group_hunks: cited line ranges outrank token overlap, which outranks a bare filename match — reasonable priority order, and ties correctly go to the earlier bullet.
  • body_with_link/write_link: re-reads the live body at write time rather than the one captured at build time, so a concurrent edit isn't reverted — matches the PR's own claim, and the code backs it up.
  • render_diagram: falls back cleanly from mmdc (inline SVG) to a <pre class="mermaid"> block with the cdnjs script when the renderer isn't present; no external assets beyond that script, matching the "no PNG fallback needed" summary line.
  • _run/subprocess calls: all go through subprocess.run with argument lists (no shell) and explicit timeouts, consistent with the check-subprocess-timeouts.py pass reported in Validation.
  • DIAGRAM_RE's comment explaining why it reads to a terminator line rather than the first --> is good self-documentation given mermaid diagrams themselves contain -->.
  • No blocking findings in what I could read. The parts I couldn't see (upload-evidence.py's one-line extension addition, and the new test file) are lower-risk by description, but I'm flagging that I didn't verify them directly.

@april-clearwater april-clearwater Bot added the mergeable Agent-approved, ready for owner merge label Sep 12, 2026
…ermaid fence

Where an HTML comment ends is not one answer: a browser ends it at `--!>` as
well as at `-->`, and `-->` is also a mermaid edge. A parser reading a comment
for content therefore disagrees with the renderer about where that content
stopped, and the text after the disagreement is prose to a reader of the PR and
diagram source here. The `<!-- review-page:diagram -->` marker above a ```mermaid
fence has one reading, and GitHub draws it in the body as well, so it is now the
only carrier.

CodeQL alert #28 (py/bad-tag-filter) on 7dd7478 removed with the comment-held form.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3CL23nN7Fmu8mE5TB9CdT
@fairchild

Copy link
Copy Markdown
Owner Author
  • 2026-09-12T04:13:38Z progress | gate on head c0e463b (origin/main 5b6bb9d)

Codex triage — head c0e463b

Adjudicated: taken 1, declined 0 — head c0e463b (answering CodeQL alert #28 on 7dd7478)

  • taken — py/bad-tag-filter (high): the comment-held diagram form and its regex are removed; an authored diagram is a ```mermaid fence under the <!-- review-page:diagram --> marker only, which a parser and a browser read the same way. CodeQL's Python analysis reports no open alert on this head.

Gate, bare in the worktree: test_pr_review_page.py Ran 33, OK; test_upload_evidence.py Ran 10, OK; full scripts/tests loop 52 files, 0 failed; check-subprocess-timeouts.py OK; readiness passed; #1613's three requested items read complete by the factory parser. The specimen page for #1602 rebuilt from the recorded fixture is byte-identical to the hosted page apart from the footer time, before and after this fix. I read the hosted page (GET 200, text/html; the store refuses HEAD) and its browser render: five sections in order, a build-time SVG diagram with no script dependency, the diff collapsed under the Summary sentences that claim its hunks. MERGEABLE with headRefOid equal to the worktree HEAD at flip time.

Flipping ready. Merge is the portfolio steward's under B4, or Michael's click.

Orchestrator note (workspaces)

@fairchild
fairchild marked this pull request as ready for review September 12, 2026 04:13
april-clearwater[bot]
april-clearwater Bot previously approved these changes Sep 12, 2026

@april-clearwater april-clearwater Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

April Clearwater, Application Lead

🟡 Approve with follow-ups — CodeQL fix lands clean, same visibility gap as my last pass

Head c0e463b2 replaces the comment-scraping diagram parser with a <!-- review-page:diagram --> ```mermaid fence requirement (DIAGRAM_MARKER_RE), closing the py/bad-tag-filter alert from my earlier round. The diff I can see is sound; the diff I still can't see hasn't changed in nature.

  • Worth noting: the diff handed to me again truncates partway through the 1602/pr.json fixture, so I still haven't seen the actual diffs for scripts/upload-evidence.py, scripts/tests/test_pr_review_page.py, or scripts/tests/test_upload_evidence.py — same gap as my first review. Lower risk by description (a one-line extension add, and test-only files), but unverified by me directly.
  • Worth noting: the PR body's Ran 31 tests line predates this fix commit; the author's own progress comment on this head reports Ran 33 tests ... OK, which is a stronger number than what's printed in the body, not a weaker one — not a concern, just worth reconciling in the body text before merge so the two don't disagree.
  • Worth noting: the fix comment explaining the marker rationale (DIAGRAM_MARKER_RE's docstring) is good self-documentation — it names the exact disagreement (--> vs --!> vs mermaid's own edge syntax) that made comment-scraping unsound in the first place.

Evidence: ✅ all three items from #1613 remain [complete] with specifics unchanged from my last pass (hosted URL + HTTP/2 200/text/html, before/after test counts, 52/52 suites). This is a visual product plus python logic, so a screenshot for the page and named test counts for the logic is the right bar, and both are present. The author's own triage comment on this head reports the same gate re-run (33/10/52 passing) plus a rebuild-and-byte-diff check against the hosted #1602 page — that's the kind of self-check that satisfies the gate for a change this size.

What changed since my first review, and why it's sufficient
  • DIAGRAM_MARKER_RE now requires the mermaid content live inside a fenced code block under the marker, rather than reading raw text up to a terminator line inside an HTML comment. A parser and a browser now agree on where the diagram source starts and ends, which is exactly the class of bug CodeQL's py/bad-tag-filter flags (regex-based HTML/comment parsing disagreeing with a real renderer).
  • Nothing else in pr-review-page.py changed in the parts visible to me — claim_score/group_hunks, body_with_link, render_diagram's mmdc fallback, and the subprocess calls are identical to what I read before and still hold up.
  • I can't re-verify the two test files or upload-evidence.py's extension addition directly, but their described scope (a fence-based fixture update, a one-line MIME addition) doesn't raise new concerns beyond what a green test_pr_review_page.py count already covers.

@fairchild

Copy link
Copy Markdown
Owner Author

Codex pass — 2026-09-12, head c0e463b

A senior read over scripts/pr-review-page.py and the rest of the diff, with the untrusted-input question first. Every claim below was reproduced in a clean worktree at this head before it was written down; where the codex run overstated something, the corrected version is what appears here, and finding 5 says so explicitly.

Blocking

  1. No content security policy anywhere, on a page built from untrusted text and served from a shared origin. build_page emits no <meta http-equiv="Content-Security-Policy">, and the evidence store supplies none either: infra/cloudflare-evidence-store/src/index.ts:33-40 sets Content-Type, Cache-Control and Access-Control-Allow-Origin: * on a GET and nothing else. The live specimen confirms it — HTTP/2 200, content-type: text/html; charset=utf-8, no CSP, no X-Content-Type-Options, no frame policy. Widening ALLOWED_EXTENSIONS to html (scripts/upload-evidence.py:21) means any holder of EVIDENCE_UPLOAD_TOKEN can serve active HTML on evidence.cloudcompute.com, an origin every other artifact shares. The escaping in this script is the only thing standing between a PR body and script execution there, with no second layer behind it.

  2. The build-time renderer fetches URLs chosen by PR text, and the reviewer's browser fetches them again. render_diagram (scripts/pr-review-page.py:454-477) hands the body's mermaid source to mmdc, which renders it in a headless Chromium. A loopback probe put <img src="http://127.0.0.1:PORT/probe"> in a diagram label; the build host requested it twice. The URL then survived into the inlined SVG, so it fires again for every person who opens the page. out_file.read_text() at line 471 is interpolated verbatim, with no sanitization of the returned XML: <foreignObject>, remote images and data: images all come through. Script tags, on* handlers and javascript: links were stripped, but only because mermaid-cli 11.12 defaults to securityLevel: strict. Nothing in this repo pins that version or that setting, so the page's safety on this path is an inherited default.

  3. A diagram hidden from the PR body still reaches the page. The point of c0e463b was that a parser and a browser read a mermaid fence the same way. They still do not agree on where it lives. Nesting the marker and its fence inside an outer HTML comment is accepted by DIAGRAM_MARKER_RE (scripts/pr-review-page.py:90-98) and returned by diagram_source:

    <!--
    <!-- review-page:diagram -->
    ```mermaid
    graph LR
      hidden --> payload
    ```
    -->
    

    GitHub renders nothing for that block. The review page renders hidden --> payload. So the page can show a diagram no reviewer sees in the body it claims to front, which combined with finding 2 is the delivery vehicle. CodeQL is genuinely quiet on this head, checked directly and not taken from the gate comment: zero open alerts on refs/pull/1615/head, and alert Complete refinement hardening and start maintainability pass #28's most recent instance sits on 7dd7478, the commit before the fix. The alert is closed. The property it stood for is not.

  4. A failed review-thread query renders as a clean pull request. review_threads returns [] on RuntimeError, SubprocessError or JSONDecodeError (scripts/pr-review-page.py:192-194), and build_page then prints No open review threads. (line 755). The same query asks for first: 100 with no pagination. A page whose fifth section exists to say where a PR stands reports unavailable as none, which is the one failure mode that makes a summary page worse than no page. The PR body names this as an accepted degradation ("review threads degrade to 'none reported' when the GraphQL call fails"); the fix is to distinguish the two states in the output, not to widen the catch.

Minor

  1. Scheme handling on URLs taken from PR text, narrower than it first reads. Escaping itself is clean. Every interpolation runs through _esc (html.escape(..., quote=True)), and a title carrying "onload="alert(1) could not break out of an attribute. A javascript: link does not reach an href, because url.startswith("http") at line 524 filters it — the codex run reported a parsed javascript: anchor, and that did not reproduce. What is true: that gate is a prefix test rather than a scheme check, so httpjavascript:alert(1) renders as an <a href>, and markdown images (IMAGE_MD_RE, line 509) take any scheme with no check at all, so ![c](javascript:...) renders as <img src="javascript:...">. Both are inert in a browser. Fix the gate anyway; it is one urlsplit and it is currently load-bearing by accident.

  2. mmdc failure falls through to a different, older mermaid. The except (subprocess.SubprocessError, OSError): pass at line 472 catches a timeout, a non-zero exit and a parse rejection alike, and every one of them lands on the CDN fallback: mermaid 10.9.1 from cdnjs, no subresource integrity attribute, no explicit securityLevel. Source the build-time renderer refused gets a second parser in the reviewer's browser. Malformed input does not fail closed; it fails to a different engine.

  3. Generated diagram labels are built by string interpolation. diagram_source puts Path(path).name and the closed-issue reference straight inside mermaid label quotes (lines 429-431) with no escaping for that context. A filename carrying a quote or markup reaches the mermaid source without anyone authoring a diagram at all.

  4. The escaping tests are two exact-string assertions. test_pr_body_markup_cannot_reach_the_page_unescaped (line 98) checks one <script>alert('xss')</script> in a summary bullet, and it exercises the plain-language list rather than a group heading. test_a_hostile_label_cannot_become_markup (line 234) checks one exact label. Neither would go red for a remote image, a data URL, a handler with different text, a hostile filename, an attribute-breaking caption, or a marker hidden in an outer comment. Worth adding: a title and a group heading with quotes and angle brackets; a hostile filename and a diff hunk containing markup; scheme cases for every href and src; the nested-marker body; a build-time network assertion; and a CSP assertion once one exists. scripts/tests/test_upload_evidence.py has no html case at all, so deleting either the extension or its content-type mapping leaves all ten tests green.

  5. Dead weight. Bullet.text (line 130) and Group.bullet (line 141) are written and never read. PR_FIELDS fetches comments and labels that no rendering path touches, which is 24.6 KB of the 63.9 KB recorded fixture.

Checked and clean

PR titles, summary sentences, group headings, filenames, diff lines, evidence captions, author logins, check names and review-thread bodies all pass through _esc. No PR-derived text is concatenated into an inline script or an event handler. mmdc receives a fixed argv and a fixed temp path, so PR text reaches no shell and selects no flag or filesystem location. Every subprocess call carries a timeout. --head refuses to build a page claiming a head the PR does not have. The uploader change widens exactly .html and its content type, and touches no other extension, the 50 MiB limit, key construction or path handling. There is no scripts/_helpers.py to have duplicated. Scope matches #1613. Both suites pass here: test_pr_review_page.py Ran 33, OK; test_upload_evidence.py Ran 10, OK.

Verdict: FINDINGS

Not verified

  • The hosted specimen's HTML bytes and its visual browser render. Only its response was checked: HTTP/2 200, text/html; charset=utf-8, no CSP, no nosniff, access-control-allow-origin: *.
  • Mermaid 10.9.1's behavior on hostile source in a real browser. All hostile rendering here used local mmdc 11.12.
  • Whether the evidence Worker sets different headers for a path it wrote itself versus one uploaded through the client.
  • Any network reach beyond the loopback probe. Nothing was sent to an external host.
  • CodeQL's own workflow run log. The alerts API was read directly instead.
  • Suites beyond the two named above.
  • Unrelated but adjacent: alert feat: harden tart-gui-automation + context attachments UX #19, js/bad-tag-filter at web/scripts/docs-sync.mjs:287, is open on main and untouched by this PR.

No uploads, no edits, no pushes, no deploys were made in this pass.

Steward note (steward v9)

@april-clearwater

Copy link
Copy Markdown
Contributor

This blocks merge. The codex pass found four issues the c0e463b fix didn't reach: no Content-Security-Policy on a page built from untrusted PR text and served from a shared evidence origin; render_diagram lets PR-authored mermaid source trigger a live network fetch during the build and then inlines the returned SVG unsanitized, so a build-time fetch becomes a reviewer-time fetch too; a diagram marker nested inside an outer HTML comment still renders on the page even though GitHub shows nothing for that block, which combined with the SVG-fetch issue is a real delivery path, not a theoretical one; and a failed review-thread GraphQL query renders identically to zero open threads, so the section meant to show where the PR stands can silently misreport.

The "ready" flip in the earlier comment predates this review and should be treated as reversed — this needs another pass, not a merge. Whoever picks it up next should add a CSP meta tag (and push for one on the evidence store's responses too), pin mmdc's version and securityLevel instead of relying on its default, reject or flatten markers nested in an outer comment so the parser and GitHub agree on what's hidden, and make the review-threads section distinguish "query failed" from "no threads found." The minor items — a proper URL scheme check via urlsplit instead of a prefix test, the silent fallback to an older unpinned mermaid on mmdc failure, unescaped diagram labels, thin escaping test coverage, and the dead Bullet.text/Group.bullet fields — are worth folding into the same pass but don't block on their own.

No code, labels, or PR state changed here; that's for whoever takes the next pass.

…s offline to images, and reports unavailable as unavailable

The page is built from text nobody vetted and served from an origin every other
evidence artifact shares, so it now emits its own Content-Security-Policy as the
first thing in its head and loads nothing from anywhere: no script, no font, no
CDN. The diagram is drawn once at build time in a renderer with no route out to
a network, and leaves as a PNG, so nothing a mermaid label asked for reaches a
reader. A marker and fence nested inside an outer HTML comment now carry no
diagram, because GitHub shows that block to nobody and a page that draws it
shows a diagram no reviewer can see. Reviews, review threads and checks report
three states rather than two: what is there, that there is none, or that the
query failed.

Also: only http and https become an href or an img src and anything else is
shown as text, generated mermaid labels are escaped for mermaid's own syntax,
two dead dataclass fields and two unread API fields are gone, and the uploader's
html handling has a test of its own.

Codex round 1 on c0e463b: four blocking and five minor taken; the origin's headers are #1616.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3CL23nN7Fmu8mE5TB9CdT
…, and the fallback has a test of its own

`mmdc` is on a developer's machine and not on the hosted runner, so two tests
asserted a drawn diagram that CI could not produce and read the documented
fallback as a failure. A shared `requires_renderer` skip now gates every test
that needs a renderer, the way the no-network test already did.

The path CI does take is now a contract rather than an absence: with the lookup
forced to find nothing, the page shows the mermaid source escaped inside a
`<pre>`, says it did not draw the diagram, and that block carries no `<img>`,
`<svg>` or `<script>`. That test needs no renderer, so it runs everywhere.

CI run 34674313148 on ae4ee8c: two tests assumed a renderer the hosted runner lacks.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3CL23nN7Fmu8mE5TB9CdT
@fairchild

Copy link
Copy Markdown
Owner Author
  • 2026-09-12T05:08:35Z progress | gate on head 048487f (origin/main 5b6bb9d)

Codex triage — head 048487f

Adjudicated: taken 9, declined 0, filed #1616 — head 048487f (answering round 1 on c0e463b)

  • taken — (1) CSP: the page emits default-src 'none'; img-src data: https:; style-src 'unsafe-inline'; base-uri 'none'; form-action 'none' as the first element of its head and loads nothing external; no CDN, no script, no font. The origin's own headers are the store's to set: filed as The evidence store serves uploaded HTML as active pages on its shared origin with no content security policy #1616.
  • taken — (2) untrusted mermaid: every diagram renders offline (Chromium with --host-resolver-rules=MAP * ~NOTFOUND; a loopback-listener test proves no request leaves the build) to a PNG data URI; no SVG or markup is inlined. Without a renderer the page shows the escaped source and says the diagram was not rendered, and that fallback has its own test.
  • taken — (3) comment-nested fence: HTML comments are stripped with the terminator-complete pattern (--> and --!>) before the marker is searched; a marker inside an outer comment carries nothing (tested).
  • taken — (4) a failed thread, review, or check query renders unavailable (the query failed), never as none (tested).
  • taken — the five minors: http/https-only hrefs by parse, escaped generated labels, the dead fields and unused fixture JSON removed, an .html upload test, escaping tests on the escaped form.

Gate, bare in the worktree: test_pr_review_page.py Ran 50, OK (skipped=3 with the renderer hidden from PATH, as the hosted runner sees it); test_upload_evidence.py Ran 12, OK; full scripts/tests loop 52 files, 0 failed; check-subprocess-timeouts.py OK; readiness passed. CI test green on this head; CodeQL Python analysis reports no open alert. The #1602 specimen was rebuilt and re-uploaded on this code (GET 200 text/html; the page carries the policy and a PNG diagram) and #1602's link updated. MERGEABLE with headRefOid equal to the worktree HEAD at flip time.

Flipping ready. Merge is the portfolio steward's under B4, or Michael's click.

Orchestrator note (workspaces)

@fairchild
fairchild marked this pull request as ready for review September 12, 2026 05:08
april-clearwater[bot]
april-clearwater Bot previously approved these changes Sep 12, 2026

@april-clearwater april-clearwater Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

April Clearwater, Application Lead

🟡 Approve with follow-ups — the codex pass's four blockers are fixed and test-verified

Head 048487fc closes all four blocking findings from the codex security pass (no CSP, network-reaching diagram render with unsanitized SVG, a marker nested in an outer comment still rendering, a failed review-thread query reporting as "none") plus the five minor ones. I read the current scripts/pr-review-page.py, scripts/tests/test_pr_review_page.py, and scripts/tests/test_upload_evidence.py directly from the checked-out branch rather than trusting the gate comment's word for it.

  • Worth noting: the PR body's own ## Evidence Status/## Validation text still reads "Ran 49 tests" from before this fix round; the actual suite at this head is 50 tests (3 skipped when mmdc is absent) — reconcile the number before merge so the body doesn't disagree with the code it's describing.
  • Worth noting: the hosted screenshots linked in the PR's ## Evidence section were captured before 048487fc's security fixes (timestamps ~04:05, the fix landed at 05:08) — the page's visual layout didn't change, so this doesn't undermine the visual claim, but a fresh capture would remove the doubt.
  • Worth noting: #1616 (the evidence store's own response headers) and the review-threads first:100 pagination cap are both real, both correctly named as accepted follow-ups rather than silently ignored.

Evidence: ✅ all three requested items from #1613 are [complete]. This PR mixes a security-isolation claim (needs a test that actually proves the property, not just a green suite) with a visual product (needs a screenshot) and ordinary logic (needs named test counts) — all three bars are met: test_the_renderer_reaches_no_network_during_a_build runs a real loopback listener and asserts zero hits, the specimen screenshot exists and is linked, and I independently counted 50 test methods in test_pr_review_page.py and 12 in test_upload_evidence.py by reading the files — both match the gate comment's numbers exactly.

What I checked against each of the codex pass's four blockers, and the fifth-through-ninth minors
  1. CSPCONTENT_SECURITY_POLICY is defined and build_page emits it as the very first element inside <head>, before charset or viewport. Policy.test_the_policy_is_the_first_thing_in_the_head asserts this by locating the first <meta> in the head and checking it's the CSP one with all four directives present.
  2. Untrusted mermaid reaching the network / unsanitized SVGrender_diagram now renders to a PNG via base64.b64encode, never reads out_file as text, and PUPPETEER_ARGS routes everything (including loopback, via --proxy-bypass-list=<-loopback>) through a proxy that doesn't exist. test_the_renderer_reaches_no_network_during_a_build stands up a real HTTP listener on loopback, puts its URL in a diagram label, and asserts the listener saw zero hits — this is exactly the class of proof that "the named tests passed" alone wouldn't give me.
  3. Comment-nested marker_marker_is_its_own_comment walks COMMENT_RE.finditer(body) and only accepts a marker match whose start coincides with a comment's own start, rejecting one where a comment's span merely contains it. test_a_marker_nested_in_an_outer_comment_carries_nothing reproduces the codex-provided repro nearly verbatim and confirms diagram_source falls back to the generated graph, not payload.
  4. Failed queries reporting as "none"pr.get("reviews") is None, source.threads is None, and pr.get("statusCheckRollup") is None are each checked before falling to the "no X" branch, all emitting UNAVAILABLE. Five tests in Unavailable cover both the raising path (read_threads catching RuntimeError/SubprocessError/JSONDecodeError) and the empty-vs-missing distinction for all three fields.
  5. Scheme check_safe_url now does urlsplit(url).scheme.lower() against SAFE_SCHEMES = {"http", "https"} rather than a startswith prefix test. test_the_scheme_gate_is_a_parse_not_a_prefix directly proves httpjavascript: now fails where a prefix test would pass it.
  6. CDN mermaid fallback removedrender_diagram's no-renderer branch shows escaped source text and nothing else; there's no second mmdc-failure path that reaches for cdnjs anymore. test_without_a_renderer_the_page_shows_the_source_and_says_so asserts no <img>, <svg>, or <script> in that block.
  7. Escaped diagram labels_mermaid_label entity-escapes & " < > [ ] before interpolation. test_a_hostile_filename_cannot_escape_a_mermaid_label uses a filename containing "]--x[ and confirms it can't close a label early.
  8. Escaping test coverage — the Escaping class now covers a hostile title, a hostile group heading, a hostile filename in a diagram label, markup in a diff hunk, and a hostile evidence caption — five targeted cases where before there were two generic ones.
  9. Dead codeBullet.text and Group.bullet are gone from both dataclasses; PR_FIELDS no longer requests comments or labels. Confirmed by reading the file directly.

Nothing else in the visible code changed shape since my last pass: claim_score/group_hunks's honest-heuristic scoring, body_with_link's live-body re-read, and the subprocess timeout discipline are all unchanged and still hold up.

@fairchild

Copy link
Copy Markdown
Owner Author

Verdict: FINDINGS on 048487f

Codex pass, round 2 of 2 — head 048487f

Round 2 of the two-round bound, on head 048487f, after the nine round-1 items on c0e463b were taken (the adjudication at 05:08:36Z). Codex (gpt-5.6-sol, reasoning xhigh) read the diff against round 1 and the adjudication; every claim below was reproduced in a clean worktree at this head before it was written down, and where the codex run overstated something, the corrected version is what appears here. There is no round 3: the in-fence findings are the repo steward's to adjudicate on this PR with an Adjudicated: line before Michael's click.

Round 1, item by item

  1. CSP: closed. scripts/pr-review-page.py:61-64 and :798 emit the policy as the first element of <head>; no script, link or font anywhere. The hosted Give WorkspaceServiceTests each their own preferences domain instead of UserDefaults.standard #1602 specimen carries it (GET 200, text/html; one policy, one PNG data URI, zero <svg>, zero <script>, zero <link>).
  2. Untrusted mermaid rendered offline: partially closed. The network is closed (the loopback test passes here with mmdc 11.12; codex's extra [::1] probe had no hits), no SVG is inlined, and the no-renderer fallback shows escaped source. The filesystem is not closed: finding 1.
  3. Comment-nested marker: closed for the diagram (_marker_is_its_own_comment, :469-491; the nested, --!>, unterminated and <!--> forms all yield the generated graph). Not closed for prose and evidence: finding 2.
  4. Unavailable as unavailable: closed as to misreporting. A failed thread query renders unavailable (the query failed); a failed gh pr view aborts the build with no page, which is not a misreport. The residue is finding 5.
  5. Scheme gate by parse: closed (_safe_url, :745-757).
  6. No CDN fallback: closed (:527-575; the fallback is escaped source in a <pre>).
  7. Generated labels escaped: closed (_mermaid_label, :456-465).
  8. Escaping tests: closed (Escaping and Schemes classes; test_html_uploads_as_a_page_not_as_a_download). The gap that remains is finding 9.
  9. Dead fields and fixture keys: closed (Bullet, Group; neither fixture carries comments or labels).

Findings

  1. [in-fence, blocking; reopens round-1 Add Sparkle auto-update #2 in part] Local files on the build host reach the diagram PNG. render_diagram (scripts/pr-review-page.py:527-575) closes the network, but mmdc loads its render page with page.goto(file://…/dist/index.html), so the page has a file:// origin and a bare path in the source resolves against it. Reproduced with a 120×120 magenta PNG at a known path: an HTML label a["<img src='/abs/path/probe.png'>"] and a relative ../../…/probe.png both put the probe's pixels in the rendered diagram (size=(550, 252) magenta_pixels=24025 against a baseline of 0), and the flowchart image shape A@{ img: "/abs/path/probe.png" } does the same (near_magenta=44160). A file:// URL is refused on both paths by mermaid's sanitizer; a bare path is not. The PNG is what --upload publishes to the shared origin, so any image or SVG the build user can read at a guessable path (evidence captures, screenshots, exported diagrams) can be carried into a hosted page by a PR body. Fix: confine the renderer's filesystem (a sandbox-exec profile or a container exposing only the temp dir and the mmdc package), or refuse authored source that names an image at all (<img, @{ img:), since a diagram of a PR's shape needs none. A mermaid config passed with -c (htmlLabels: false at the top level and per diagram, with secure listing those keys) closes the HTML-label half and an %%{init}%% directive cannot reopen it (verified), but it does not touch the image shape.

  2. [in-fence, blocking; round-1 Code signing and notarization for distribution #3 by its principle] Content inside an HTML comment reaches the page everywhere except the diagram. body_sections, summary_bullets and evidence_items (:250-290, :594-626) read raw lines, so a ## Summary and its bullets inside <!-- … --> become the page's opening lines and group headings, and an ![…](https://…) or […](…) inside a comment becomes an <img> every reader's browser fetches, or a link they click. Reproduced: all three carried; GitHub shows none of them. Fix: compute visible = COMMENT_RE.sub("", body) once in build_page and read sections, bullets and evidence from it (the marker scan keeps the raw body; write_link already re-reads the live body); test with the hidden-summary and hidden-image bodies.

  3. [in-fence, minor] A marker and fence inside a fenced code block (a ```text or four-backtick fence), a <![CDATA[ block or a <?…?> instruction is drawn by the page while GitHub shows the same text as literal source. Not hidden, so not the round-1 case, but the two readers disagree. Fix: skip a marker that falls inside a fence extent, with the same scan _marker_is_its_own_comment does for comments.

  4. [in-fence, minor] read_pr (:190-196) takes headRefOid from gh pr view, then fetches the diff in a second call; --head checks the first and nothing rechecks before --upload (:1033) or --link (:1043). A push between the two calls publishes a page naming one head over another head's diff. Fix: re-read headRefOid after gh pr diff and again in write_link, and refuse when it moved.

  5. [in-fence, minor] read_threads (:199-231) catches the query failing but not a payload without the expected keys (KeyError/TypeError escape at :231 and abort the build), and asks first:100 (:212) with no page cursor, which the PR body discloses. Fix: add the two to the except (returning None) and either paginate or print "first 100" in the section.

  6. [in-fence, minor] body_with_link (:926-951) rewrites the first line starting Review page: wherever it is, a fenced example or a comment included (:935), and then adds no visible line. Reproduced. Fix: consider only lines before the first ## heading and outside comment extents.

  7. [in-fence, minor] _safe_url admits http: for images (SAFE_SCHEMES, :83) while the policy's img-src admits only https: and data:, so an http: evidence image renders as a broken image rather than the "not shown" note. Fix: a separate IMAGE_SCHEMES = {"https"}.

  8. [in-fence, minor] img-src data: https: lets a body's image URL make every reader's browser fetch from any https host with nothing in between (on the body itself, GitHub's camo proxy is what hides a reader's address). Fix: pin img-src to the evidence origin(s) a body may cite and apply the same host rule to images in _safe_url.

  9. [in-fence, minor] No test that runs on the hosted runner asserts the puppeteer config carries the network flags: the three renderer tests skip without mmdc, and nothing patches shutil.which and subprocess.run to read the config file render_diagram writes. Fix: one such test, so the wiring half of the offline proof survives a refactor even where the render half cannot run.

On the offline proof without the local run

What the tests prove, on a machine with mmdc: a loopback IP literal in a label is not fetched and does not survive into the page (test_the_renderer_reaches_no_network_during_a_build; reproduced here, Ran 50 tests, OK; codex's [::1] probe also had no hits). What code reading proves: PUPPETEER_ARGS (:76-81) are written to puppeteer.json and mmdc Object.assigns that file onto {headless: 'shell'} before puppeteer.launch, with no default args to clobber. What CI proves: only the fallback (Ran 50 tests, OK (skipped=3) with mmdc hidden from PATH, reproduced). Since CI never builds a page and the laptop that does is where the suite runs bare, the skip is honest and the proof holds for the network on the machine that matters; finding 9 is what carries the wiring half onto the runner. What the proof does not cover is the filesystem, which is finding 1. Unverified assumptions: that the flags behave the same on a Chromium other than the one installed here, and that nothing listens on 127.0.0.1:1.

Outside the fence, for the repo steward to file

  • scripts/evidence.sh:275 prints the image-markdown hint ![name](url) on stderr for an .html upload; the URL on stdout is correct, so upload() is unaffected. Fix: add html to the link case beside webm, mp4 and txt.

Checked and clean

Every PR-derived string in the page passes through _esc; no PR text reaches an inline script, handler or style; mmdc, gh and evidence.sh get a fixed argv with a timeout; --head refuses a head the PR does not have; --link refuses to run from a fixture; the uploader widens exactly .html and its content type, with a test for each half. CI is green on this head (test, readiness and the CodeQL analyses all pass) and the open code-scanning alerts are all on main in files this PR does not touch. Both suites pass here: test_pr_review_page.py Ran 50, OK with mmdc and OK (skipped=3) without; test_upload_evidence.py Ran 12, OK.

No uploads, no edits, no pushes, no deploys were made in this pass.

Steward note (steward v9.6)

@april-clearwater

Copy link
Copy Markdown
Contributor

Round 2 leaves two blocking items in the fence: local files on the build host reaching the diagram PNG through mmdc's file:// origin (finding 1), and comment-hidden summary text, headings, and links surfacing everywhere except the diagram (finding 2). Both need a fix and a re-run of the suite before this can flip back to ready — the repo steward should adjudicate the round-2 findings with an Adjudicated: line, since round 2 is the last round in the two-round bound here.

The seven minor findings (fence-hidden markers, head drift between the two gh calls, thread-query payload shape and pagination, the link-rewrite scope, the http: image scheme mismatch against the CSP's img-src, the missing host pin on img-src, and the untested puppeteer wiring on CI) are worth folding into the same pass rather than filed separately, given how much of the diff they touch. The one item outside the fence — evidence.sh's stderr hint not covering .html uploads — is a one-line, non-blocking fix for whoever picks this up.

@fairchild
fairchild marked this pull request as draft September 12, 2026 05:46
… and reads nothing from inside comments or fences

mmdc renders from a `file://` page, so a bare path in an image shape or an HTML
label resolves on the machine doing the build and is rasterised into the PNG the
page publishes: any image that user can read at a guessable path could be
carried into a hosted page by a PR body. Closing it by sanitising the render is
a race against mermaid's feature list, so an authored fence is now drawn only if
its syntax is on a strict allowlist — flowchart, graph, sequence and state
headers, plain ids, edges, and labels with no markup, no path and no directive.
Anything else is shown as escaped source, which costs a picture and keeps the
filesystem out of the page. Mermaid's own `securityLevel: strict` and
`htmlLabels: false` are passed with `secure` pinning both, as the layer behind
that one. The generated graph does not go through the allowlist: its labels are
file paths and are escaped by `_mermaid_label` instead.

Content inside an HTML comment is content GitHub shows to nobody, so the page no
longer reads it either: sections, bullets, evidence and the closing reference
all come from a comment-stripped body, and a marker inside a comment, a fenced
example, a CDATA block or a processing instruction draws nothing.

Also: the head is re-read before the upload and before the body edit and a head
that moved refuses both; a malformed review-thread payload reads as unavailable
rather than crashing, and the page states the 100-thread cap when it hits it;
`body_with_link` rewrites only a line above the first heading and outside
comments and fences; images are admitted only for `https:` on the evidence store
or GitHub, which is what the page's policy now names instead of all of https.

Codex round 2 on 048487f: both blocking taken; minors as recorded.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3CL23nN7Fmu8mE5TB9CdT
@fairchild

Copy link
Copy Markdown
Owner Author
  • 2026-09-12T06:05:51Z progress | gate on head d4fa12a (origin/main 5b6bb9d)

Codex triage — head d4fa12a

Adjudicated: taken 9, declined 0, filed #1618 — head d4fa12a (answering round 2 on 048487f)

  • taken — blocker 1, local files reaching the diagram PNG: refusal, not sanitisation. is_renderable_mermaid admits only flowchart/graph, sequence, and state headers, refuses a token list over the whole source and a second list inside labels, and requires every line to match a node-or-edge skeleton once labels are removed; securityLevel: strict and htmlLabels: false pinned as the second layer. Anything else is shown as escaped source with a notice. My own probe on this head: an image-shape node and an HTML-label img, each naming a bare path to a PNG on the build host, produced no image tag and no data URI, only the escaped source under the notice.
  • taken — blocker 2, comment-hidden content: sections, Summary bullets, evidence items, and the Closes reference read the comment-stripped body. My probe: a bullet, an evidence image, and a Closes #999 inside a comment reached nothing on the page.
  • taken — minors 3–9 as the report records: fence/CDATA/PI carriers refused; head rechecked before upload and link; tolerant thread payloads with the 100-cap stated; the link line rewritten only above the first heading and outside fences and comments; https-only images pinned to the evidence and GitHub asset hosts in both the policy and the parser; the puppeteer wiring asserted from the written config files so the no-network proof runs on the hosted runner too.
  • filed — evidence.sh prints image markdown for an uploaded .html file #1618, evidence.sh:275's image-markdown hint for an .html upload, outside this fence.

Gate, bare in the worktree: test_pr_review_page.py Ran 72, OK, and OK (skipped=6) with the renderer off PATH as CI sees it; test_upload_evidence.py Ran 12, OK; full scripts/tests loop 52 files, 0 failed; check-subprocess-timeouts.py OK; readiness passed. CI test green on this head; no open CodeQL alert. The #1602 specimen was rebuilt and re-uploaded on this code and relinked. MERGEABLE with headRefOid equal to the worktree HEAD at flip time.

Flipping ready. The portfolio steward runs its confirmation pass on the diff from 048487f; merge is Michael's click.

Orchestrator note (workspaces)

@fairchild
fairchild marked this pull request as ready for review September 12, 2026 06:05

@april-clearwater april-clearwater Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

April Clearwater, Application Lead

🟡 Approve with follow-ups — the two round-2 blockers read as fixed in the diff; same visibility gap as my last three passes

Head d4fa12ab (per the latest triage comment) closes both blocking findings from the codex round-2 pass — a file://-origin diagram renderer that could pull local build-host images into a published PNG, and comment-hidden Summary/evidence content surfacing on the page while GitHub shows nothing for it. I read the current scripts/pr-review-page.py diff directly to check both.

  • Worth noting: MERMAID_FORBIDDEN now refuses img, image, href, click, @{, file:, /, \, and .. anywhere in an authored fence's lowercased source, so a diagram naming any path — an image shape or an HTML-label <img> — is shown as escaped source instead of rendered. That's a real fix for the file://-origin path finding 1 named, not a narrower sanitizer.
  • Worth noting: body_sections, summary_bullets, evidence_items, and closes_issue all now read visible_body(body) (comments stripped) rather than the raw body; authored_diagram's marker scan correctly still uses the raw body, since it needs to know where comments are. _quoted_line_numbers and body_with_link's fence/comment-aware insertion point match the round-2 minors (3 and 6).
  • Worth noting: same gap as my last three reviews on this PR — the diff handed to me truncates before scripts/upload-evidence.py, scripts/tests/test_pr_review_page.py, and scripts/tests/test_upload_evidence.py, so I can confirm the fixes exist in the generator but not that a test asserts each one (the filesystem-refusal case, the comment-hidden-content case, the puppeteer-wiring assertion round 2's minor 9 asked for). The gate comments report matching test counts (Ran 72, OK/OK (skipped=6)), which is consistent with the generator's growth since my last pass, but it's their word, not something I read myself.

Evidence: ✅ all three requested items from #1613 remain [complete] with specifics (hosted URL + status/content-type, before/after test counts, 52/52 suites). This PR mixes a security-isolation claim (needs a test proving the property) with a visual product (screenshot) and ordinary logic (named test counts); the first two bars are met on what I can read, the third — the specific tests for round 2's fixes — I can't independently confirm because those files are outside the diff shown to me.

What I checked against each round-2 blocker, and what I couldn't
  1. Local files reaching the diagram PNG (blocking, round 2 finding 1)is_renderable_mermaid refuses any authored source containing img, image, href, click, @{, %%{, file:, /, \, or .. (whole-source, lowercased), plus a per-line skeleton check once labels are stripped. Combined with MERMAID_CONFIG's pinned securityLevel: strict/htmlLabels: False/secure list and the existing PUPPETEER_ARGS network closure, this is layered refusal rather than sanitization — matches the PR's stated fix and the triage comment's description. I can't run mmdc myself to reproduce the magenta-pixel probe codex describes, but the code path that would have let it through is now gated by a syntax check, not a runtime sanitizer trying to catch every renderer feature.

  2. Comment-hidden content everywhere but the diagram (blocking, round 2 finding 2)visible_body() strips COMMENT_RE matches once, and every prose-reading function (body_sections, summary_bullets, evidence_items, closes_issue) now calls it before scanning. The one function that still reads the raw body is authored_diagram, correctly, since it needs the comment boundaries to reject a nested marker.

3–9 (minors)** — refuse_if_head_moved is called after the diff fetch in read_pr, and again from main() before --upload/--link; read_threads's except clause now includes KeyError/TypeError; body_with_link only considers lines before the first ## heading and outside comment/fence extents; IMAGE_SCHEMES = {"https"} and _safe_image_url checks both scheme and IMAGE_HOSTS membership, matching the CSP's img-src allowlist. All read consistent with what the round-2 report and the 06:05:53 triage claim was taken.

Nothing else changed shape since my last pass: claim_score/group_hunks's heuristic scoring and the subprocess timeout discipline are unchanged and still hold up.

@fairchild

Copy link
Copy Markdown
Owner Author

Verdict: CLEAR on d4fa12a

Confirmation pass — head d4fa12a

This is the confirmation pass after the two-round bound: round 1 on c0e463b, round 2 on 048487f, adjudicated at 06:05:53Z with Adjudicated: taken 9, declined 0, filed #1618 — head d4fa12ab. It is scoped to git diff 048487fc..d4fa12ab, one commit and three files (scripts/pr-review-page.py, scripts/tests/test_pr_review_page.py, the synthetic fixture), and asks two questions: does the fix push close round 2's nine items, and does it introduce anything new. It is not a review round and there is no round 3. The residuals below are the repo steward's to file; none blocks the merge. Codex (gpt-5.6-sol, reasoning xhigh) read the same scope; every claim here was reproduced in a clean worktree at this head before it was written down.

Round 2, item by item

  1. Blocker 1, local files reaching the diagram PNG: closed. An authored fence is drawn only when is_renderable_mermaid admits it (scripts/pr-review-page.py:541): a header allowlist (flowchart/graph, sequence, state), a refused-token list over the whole source (@{, %%{, href, click, img, image, file:, /, \, ..), < and > refused inside labels, and a skeleton regex (:123) over each line with labels removed. Refused source is shown escaped under a notice (:694). Behind it, MERMAID_CONFIG (:102) pins securityLevel: strict and htmlLabels: false under secure and is handed to mmdc with -c (:712). Reproduced with a 240px magenta PNG at a known path on the build host: the image shape A@{ img: "<path>" }, the HTML label a["<img src='<path>'>"] (absolute and relative), a YAML front-matter config and a markdown-string image each came out as escaped source with no data URI. With the allowlist bypassed (authored=False, straight at mmdc) the HTML-label form renders as text (1224×166 against a 406×130 baseline; the image would add about 480px of height), and an %%{init}%% directive loosening securityLevel and htmlLabels produces the byte-identical PNG, so the secure pin holds. The image shape does render the file on that path (752×512) when the allowlist is bypassed, which is why the allowlist is the layer for it, as the adjudication says. Tests: AuthoredDiagramAllowlist (test_pr_review_page.py:395), two of them renderer-gated, run and pass here with mmdc 11.12.0.

  2. Blocker 2, comment-hidden content: closed. visible_body (:323) strips HTML comments once; body_sections (:334) and closes_issue (:414) read it, and summary_bullets, plain_language and evidence_items all read through body_sections. authored_diagram and generated_diagram keep the raw body for the marker scan, as intended. Reproduced: a hidden Summary bullet, a hidden evidence image and link on an allowed host, and a hidden Closes #4321 reached nothing on the page while the visible ones did. Tests: HiddenContent (:481).

  3. Fence, CDATA and PI carriers: closed for the shapes round 2 named, with residual R1. _quoted_line_numbers (:596) skips a marker inside a four-backtick fence, a terminated CDATA block or a processing instruction; QuotedMarkers (:694) covers each.

  4. Head rechecked: closed. refuse_if_head_moved (:239) runs after gh pr diff (:260), before --upload off a live PR (:1244) and at the top of write_link (:1188). Tests: HeadMoved (:734).

  5. Thread payload and cap: closed for the payload, with residual R2. KeyError and TypeError join the except (:299) and four malformed payloads read as unavailable rather than a crash (ThreadQuery, :766).

  6. Link line scope: closed for comments and a conventional fence, sharing R1's fence rule. body_with_link stops at the first ## heading (:1136) and skips commented and quoted lines. Tests: BodyLink (:888).

  7. http: images: closed. IMAGE_SCHEMES = {"https"} (:97) and _safe_image_url (:913); an http: image on an allowed host is named in text, not fetched (ImageHosts, :831).

  8. Image policy pinned: closed. IMAGE_HOSTS (:65) drives both the CSP and the parser. The emitted policy reads img-src data: plus the five https hosts (the evidence store, the three GitHub asset hosts and github.com for the redirecting asset links) with no bare https:.

  9. Renderer wiring on the hosted runner: closed. RendererWiring (:791) patches shutil.which and subprocess.run, reads both config files the build wrote, and asserts the three network flags plus securityLevel, htmlLabels and secure. It runs without mmdc, so CI carries the wiring half of the offline proof.

Residuals, for the repo steward to file (none blocking)

R1. _quoted_line_numbers closes a fence on any line that starts with the opening run (scripts/pr-review-page.py:610). CommonMark lets a closing fence carry no info string, so on GitHub a ```text line inside a three-backtick fence is content and the fence stays open to the next bare run. A marker and mermaid fence after such a line are literal text on GitHub and a drawn (allowlisted) diagram here, and body_with_link (:1117) rewrites a Review page: line in the same spot. Both reproduced. Fix: a closing fence is a bare run of at least the opening's length, re.fullmatch(r"({3,}|~{3,})\s*", line). Codex also notes the CDATA and PI patterns (:616`) require termination, so an unterminated carrier is not quoted; not reproduced against GitHub's rendering here.

R2. The "Only the first 100 review threads were read" line (:1081) sits inside the branch that needs at least one open thread. With 100 returned threads all resolved, the page says "No open review threads." and nothing about the cap, although later unqueried threads may be open. Reproduced. Fix: hoist the notice out of the elif; the test at test_pr_review_page.py:773 makes all 100 unresolved, so add the all-resolved case.

R3. AuthoredDiagramAllowlist.assertShownAsSource (test_pr_review_page.py:415) is defined and never called, and its middle assertion (:417) could not pass if it were. Delete it.

R4. Hardening, not a defect: the allowlist's token refusal can be walked around with entities in the text positions that admit < and > (sequence messages, state notes and descriptions) and in quoted labels. &#47; and mermaid's own #47; stand in for /, &lt; and #lt; for <, and a fully numeric-entity <img> is admitted. Every such form was rendered here with a magenta probe on disk and every PNG carried zero probe pixels, and codex's byte-compare across probe colours showed no change, so securityLevel: strict with htmlLabels: false is what stands there and it holds. The comment at :113 says refusing the syntax is the check; for these positions the pinned config is doing the work. Refusing & and #…; in authored source would make the comment true on its own; the cost is a label that cannot say #1615, which the generated graph's labels never need since they bypass the allowlist.

Nothing new

The three files add no script, sink or subprocess beyond the ones round 2 read. Every new string reaching the page passes _esc or html.escape, including the notice text and the cap line. The one new argument to a subprocess is -c on the existing timed mmdc call. The fixture change only moves the evidence host to one the policy now allows.

Runs

In a clean worktree at d4fa12a, Python 3.14.4. The test file declares requires-python = ">=3.11" in its script header; on the Xcode python3 (3.9) the two renderer-gated allowlist tests error on TestCase.enterContext, which is outside the declared floor and not a residual.

  • test_pr_review_page.py with mmdc 11.12.0 on PATH: Ran 72 tests, OK. With mmdc hidden from PATH: Ran 72 tests, OK (skipped=6).
  • test_upload_evidence.py: Ran 12 tests, OK.
  • Codex, same scope, same two runs: Ran 72, OK and OK (skipped=6).

No uploads, no edits, no pushes, no deploys were made in this pass.

Steward note (steward v9.6)

@april-clearwater

Copy link
Copy Markdown
Contributor

R1 deserves more than a residual note. The fence-closer regex only checks that a line starts with a backtick or tilde run of any length — it doesn't require the closer to match or exceed the opener's length. CommonMark does require that. So a three-backtick fence closed early by a shorter run stays open on GitHub but closes here, which is the same content-hiding mismatch this PR closed twice already for comments and CDATA, just in the fence-length dimension. Given two full review rounds were spent on exactly this failure mode, R1 reads as a third instance of it rather than ordinary polish, and it's a one-line fix (re.fullmatch(r"({3,}|~{3,})\s*", line)` per the pass's own suggestion, with a length check against the opener).

R2 and R3 are cosmetic and fine to leave for follow-up. R4 is worth a one-line comment fix at :113 regardless of code changes — it currently claims syntax refusal is what blocks the entity-encoded bypass, when the confirmation pass shows the pinned securityLevel: strict config is actually doing that work.

The CLEAR verdict otherwise stands — nothing here reopens rounds 1 or 2. Whoever holds merge authority should either fold R1's one-line fix into this PR before flipping ready, given it's the same bug class as the two blocking findings this PR exists to fix, or file it as a tracked follow-up issue explicitly scoped as "round 3 of the same defect" rather than a generic residual, so it doesn't get deprioritized as polish.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author:claude-code PRs authored by the claude-code agent (interactive Claude Code sessions) mergeable Agent-approved, ready for owner merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A readable front for a dense PR: a generated review page per factory PR, #1602 as the specimen

2 participants