fix(#669): make extractReportedMetadata robust to phrasing and Markdown emphasis - #670
Merged
Merged
Conversation
…wn emphasis extractReportedMetadata() used three fixed labelled-SHA regexes to find which SHA ChatGPT reported reviewing, falling back to "the first bare 40-hex string anywhere in the text" when none matched. Confirmed live on a real /ship code-review loop (issue #642, PR #668) that this fell back incorrectly twice in the same run: - Pass 2: "Pass-2 reviewed head: **`<sha>`**" -- the `\s*` between the label's colon and the backtick required pure whitespace, so Markdown bold emphasis defeated every pattern. - Pass 3: "Reviewed pass-3 head: **`<sha>`**" -- a different word order ("reviewed" before the pass number, not after) that no pattern covered at all, bold or not. Both times the fallback picked the WRONG (earlier-mentioned, "previously reviewed") SHA instead of the current one, because a multi-pass review always states the old SHA first. This directly undermines /ship's own merge-gate check ("reviewed SHA equals this unit's current PR head") -- exactly the safety net meant to catch a stale-head certification. Replaced the fixed pattern set with one flexible "[prefix] reviewed [pass-N] head/sha:" pattern (word order and Markdown emphasis both tolerated), filtering out any match whose PRECEDING text on the same line mentions a previous/prior/anchor/old/earlier qualifier, and taking the LAST surviving occurrence -- every real multi-pass review observed states the earlier SHA before the current one, never after. Deliberately checks only text before each match (not the whole line): a later, unrelated sentence on the same physical line can independently use one of these words (observed live: "...it is one commit ahead of the requested anchor." following a valid, unqualified match) and must not retroactively invalidate it. Verified against all three real captured response texts from the #642 run (pass 1/2/3) plus all three pre-existing unit tests; added two new regression tests reproducing the exact real-world shapes found. 67/67 chatgpt-review tests pass (65 + 2 new). Closes #669
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
extractReportedMetadata()used three fixed labelled-SHA regexes to find which SHAChatGPT reported reviewing, falling back to "the first bare 40-hex string anywhere in
the text" when none matched. Confirmed live on a real
/shipcode-review loop (issue#642, PR #668) that this fell back incorrectly twice in the same run:
Pass-2 reviewed head: **\`**— the\s*` between the label's colon andthe backtick required pure whitespace, so Markdown bold emphasis defeated every pattern.
Reviewed pass-3 head: **\`**` — a different word order ("reviewed" beforethe pass number, not after) that no pattern covered at all, bold or not.
Both times the fallback picked the wrong (earlier-mentioned, "previously reviewed")
SHA instead of the current one, because a multi-pass review always states the old SHA
first. This directly undermines
/ship's own merge-gate check ("reviewed SHA equals thisunit's current PR head") — exactly the safety net meant to catch a stale-head
certification. (It did not affect #642's own merge correctness — the coordinator manually
verified the real GitHub PR comments before merging — but that defeats the point of having
this automated.)
Fix
Replaced the fixed pattern set with one flexible
"[prefix] reviewed [pass-N] head/sha:"pattern (word order and Markdown emphasis both tolerated), filtering out any match whose
preceding text on the same line mentions a previous/prior/anchor/old/earlier
qualifier, and taking the last surviving occurrence — every real multi-pass review
observed states the earlier SHA before the current one, never after. Deliberately checks
only text before each match, not the whole line: a later, unrelated sentence on the
same physical line can independently use one of these words (observed live: "...it is one
commit ahead of the requested anchor." following a valid, unqualified match) and must not
retroactively invalidate it.
Test plan
node --test skills/chatgpt-review/tests/*.test.mjs— 67/67 pass (65 + 2 new)1/2/3), each producing the correct SHA
text" and "previous-head line exclusion" cases) still pass unchanged
Closes #669
🤖 Generated with Claude Code
https://claude.ai/code/session_01LwFPT465eDJqYcRa8HGNLz