Skip to content

Read pull_requests.reviewed as could_not_check when any verdict's author reads as a non-member - #172

Open
paulfoley wants to merge 1 commit into
SoulCraft-Org:mainfrom
paulfoley:fix/reviewed-mixed-private-members
Open

paulfoley wants to merge 1 commit into
SoulCraft-Org:mainfrom
paulfoley:fix/reviewed-mixed-private-members

Conversation

@paulfoley

@paulfoley paulfoley commented Nov 20, 2026 •

Copy link
Copy Markdown

Closes #169

Item Detail
What changed privateMarkerAuthors returns true when any well-formed Review: MERGE|BLOCK at <sha> verdict (one naming a commit of its own PR) is by a non-member, not only when all are. pull_requests.fixed_after_review gets the same check pull_requests.reviewed already had. The could_not_check text now says a verdict cannot be told from an outsider's
Why A token outside the org reads a public member's verdict as MEMBER and a private member's as CONTRIBUTOR. One MEMBER verdict made the check pass, so every private member's verdict scored as not reviewed: a low score from a read the token cannot make
How it was verified node --test quality/collectors/github/github-more.test.mjs quality/collectors/github/github.test.mjs (79 pass). In "counts a member's Review: MERGE|BLOCK at comment as a review", the mix of two private-member verdicts and one public-member verdict (formerly asserted 33.3) now reads could_not_check on both metrics. With the old rule it fails
How to reverse it Revert the commit
Judgment call Reason How to reverse it
Any non-member verdict voids the metric, as the issue proposes, rather than accepting the PR's merger as the reviewer of record The token cannot tell a private member from an outsider, so a PR's review state is unknown, and the metric says so. A verdict naming no commit of its PR is still ignored, as before Restore found.length > 0 && !found.some(...)
Cost of that rule: an outsider's well-formed verdict can also void the metric It can never raise a score, and the reason names the cause. The old rule had the same effect when no member verdict existed Same as above
Test fixture: the forged outsider verdict on PR 3 now names a SHA the PR never had A forged verdict that names the PR's head now voids the metric, so the case that checks a forged verdict is not counted as a review needs a verdict naming no commit of its PR; counts are unchanged Point it back at the head and drop the forged-verdict count from the first test

🤖 Generated with Claude Code


Autofix with [code]smith
Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.

…hor reads as a non-member

A token outside the org reads a public member's verdict as MEMBER and a
private member's as CONTRIBUTOR. privateMarkerAuthors returned true only
when every well-formed verdict read as a non-member, so one public member's
verdict among the private ones made the rest score as not reviewed.

It now returns true when any verdict naming one of the PR's own commits is
by a non-member. pull_requests.fixed_after_review, which read the same
verdicts without the check, gets it too.

Closes SoulCraft-Org#169

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: paulfoley <268432743+paulfoley@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pull_requests.reviewed drops verdicts by private org members when some verdicts count

1 participant