Skip to content

fix(#669): make extractReportedMetadata robust to phrasing and Markdown emphasis - #670

Merged
BorisTyshkevich merged 1 commit into
mainfrom
fix/669-sha-extraction-markdown-bold
Aug 10, 2026
Merged

fix(#669): make extractReportedMetadata robust to phrasing and Markdown emphasis#670
BorisTyshkevich merged 1 commit into
mainfrom
fix/669-sha-extraction-markdown-bold

Conversation

@BorisTyshkevich

Copy link
Copy Markdown
Collaborator

What & why

extractReportedMetadata() used three fixed labelled-SHA regexes to find which SHA
ChatGPT reported reviewing, falling back to "the first bare 40-hex string anywhere in
the text" when none matched. Confirmed live on a real /ship code-review loop (issue
#642, PR #668) that this fell back incorrectly twice in the same run:

  • Pass 2: Pass-2 reviewed head: **\`**— the\s*` between the label's colon and
    the backtick required pure whitespace, so Markdown bold emphasis defeated every pattern.
  • Pass 3: Reviewed pass-3 head: **\`**` — a different word order ("reviewed" before
    the pass number, not after) that no pattern covered at all, bold or not.

Both times the fallback picked the wrong (earlier-mentioned, "previously reviewed")
SHA instead of the current one, because a multi-pass review always states the old SHA
first. This directly undermines /ship's own merge-gate check ("reviewed SHA equals this
unit's current PR head") — exactly the safety net meant to catch a stale-head
certification. (It did not affect #642's own merge correctness — the coordinator manually
verified the real GitHub PR comments before merging — but that defeats the point of having
this automated.)

Fix

Replaced the fixed pattern set with one flexible "[prefix] reviewed [pass-N] head/sha:"
pattern (word order and Markdown emphasis both tolerated), filtering out any match whose
preceding text on the same line mentions a previous/prior/anchor/old/earlier
qualifier, and taking the last surviving occurrence — every real multi-pass review
observed states the earlier SHA before the current one, never after. Deliberately checks
only text before each match, not the whole line: a later, unrelated sentence on the
same physical line can independently use one of these words (observed live: "...it is one
commit ahead of the requested anchor." following a valid, unqualified match) and must not
retroactively invalidate it.

Test plan

  • node --test skills/chatgpt-review/tests/*.test.mjs — 67/67 pass (65 + 2 new)
  • Verified against all three real captured response texts from the check-boundaries generic RULES do not fail closed on computed dynamic imports #642 run (pass
    1/2/3), each producing the correct SHA
  • All three pre-existing unit tests (including the "unrelated commit SHA later in the
    text" and "previous-head line exclusion" cases) still pass unchanged

Closes #669

🤖 Generated with Claude Code

https://claude.ai/code/session_01LwFPT465eDJqYcRa8HGNLz

…wn emphasis

extractReportedMetadata() used three fixed labelled-SHA regexes to find which
SHA ChatGPT reported reviewing, falling back to "the first bare 40-hex
string anywhere in the text" when none matched. Confirmed live on a real
/ship code-review loop (issue #642, PR #668) that this fell back
incorrectly twice in the same run:

- Pass 2: "Pass-2 reviewed head: **`<sha>`**" -- the `\s*` between the
  label's colon and the backtick required pure whitespace, so Markdown bold
  emphasis defeated every pattern.
- Pass 3: "Reviewed pass-3 head: **`<sha>`**" -- a different word order
  ("reviewed" before the pass number, not after) that no pattern covered at
  all, bold or not.

Both times the fallback picked the WRONG (earlier-mentioned, "previously
reviewed") SHA instead of the current one, because a multi-pass review
always states the old SHA first. This directly undermines /ship's own
merge-gate check ("reviewed SHA equals this unit's current PR head") --
exactly the safety net meant to catch a stale-head certification.

Replaced the fixed pattern set with one flexible "[prefix] reviewed
[pass-N] head/sha:" pattern (word order and Markdown emphasis both
tolerated), filtering out any match whose PRECEDING text on the same line
mentions a previous/prior/anchor/old/earlier qualifier, and taking the LAST
surviving occurrence -- every real multi-pass review observed states the
earlier SHA before the current one, never after. Deliberately checks only
text before each match (not the whole line): a later, unrelated sentence on
the same physical line can independently use one of these words (observed
live: "...it is one commit ahead of the requested anchor." following a
valid, unqualified match) and must not retroactively invalidate it.

Verified against all three real captured response texts from the #642 run
(pass 1/2/3) plus all three pre-existing unit tests; added two new
regression tests reproducing the exact real-world shapes found. 67/67
chatgpt-review tests pass (65 + 2 new).

Closes #669
@BorisTyshkevich
BorisTyshkevich merged commit e37c591 into main Aug 10, 2026
8 checks passed
@BorisTyshkevich
BorisTyshkevich deleted the fix/669-sha-extraction-markdown-bold branch August 10, 2026 15:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

chatgpt-review: extractReportedMetadata misextracts reviewed SHA when Markdown-bolded

1 participant