Skip to content

fix(automation): dispatch review-comment and review-body agent mentions - #954

Closed
seonghobae wants to merge 8 commits into
mainfrom
fix/agent-mention-review-comment-dispatch
Closed

fix(automation): dispatch review-comment and review-body agent mentions#954
seonghobae wants to merge 8 commits into
mainfrom
fix/agent-mention-review-comment-dispatch

Conversation

@seonghobae

Copy link
Copy Markdown
Contributor

Trusted @cwl-noema-review and @opencode-agent mentions on pull-request review comments and submitted review bodies never reached the router. parse_event required issue.pull_request, GitHub's pull_request_review_comment / pull_request_review payloads have no issue object, the 5-minute sweep listed only issues/{n}/comments, and the local job if required a case-sensitive conversation-comment body substring.

Changes

  • Accept review comments and submitted reviews in parse_event while still ignoring plain issues (no pull_request marker).
  • Sweep pulls/{n}/comments and submitted pulls/{n}/reviews in the lookback window.
  • Trigger the local router on pull_request_review_comment and pull_request_review; hydrate PR_NUMBER from issue.number || pull_request.number; drop the case-sensitive body contains filter (Python remains case-insensitive).
  • Skip issue-comment eye reactions for review-comment/review IDs (those IDs are not issue-comment IDs); keep the receipt issue comment.
  • Require a per-changed-file walk and allocate review compute by stage/role/ablation (Fugu / Conductor / TRINITY). Reviewers stay edit: deny. LLM key remains NVIDIA_NIM_API_KEY.

Verification

  • GitHub-shaped review-comment and submitted-review fixtures drive parse_event and build_requests_for_pull_request.
  • Workflow contract pins the new triggers and the absence of the case-sensitive body filter.
  • scripts/ci/agent_mention_router.py and agent_mention_sweep.py are 100% statement/branch with 100% public docstrings.
  • Doctoring: docs/doctoring/review-agent-mention-surfaces.md (APA 7th).

Does not wait on Checks. Does not use COPILOT_GITHUB_TOKEN.

@cwl-noema-review

Trusted @cwl-noema-review and @opencode-agent mentions on pull-request
review comments and submitted reviews were ignored because parse_event
required issue.pull_request. Accept those surfaces, sweep them, drop the
case-sensitive workflow body filter, and require a per-changed-file
review walk with staged compute allocation.
@coderabbitai

coderabbitai Bot commented Aug 13, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@seonghobae, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 11 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: ec2dbd95-4234-4557-9ff2-bc1a4dc99d72

📥 Commits

Reviewing files that changed from the base of the PR and between 6eb06cd and 792c6ce.

📒 Files selected for processing (22)
  • .github/workflows/agent-mention-opencode-dispatch.yml
  • .github/workflows/agent-mention-router.yml
  • .gitignore
  • AGENTS.md
  • ARCHITECTURE.md
  • CHANGELOG.md
  • CLAUDE.md
  • ci-review-prompt.md
  • code-reviewer-prompt.md
  • docs/automation/review-agent-comment-invocation.md
  • docs/doctoring/review-agent-mention-surfaces.md
  • scripts/ci/agent_mention_router.py
  • scripts/ci/agent_mention_sweep.py
  • scripts/ci/materialize_base_python_requirements.py
  • scripts/ci/opencode_review_prompt_template.md
  • tests/test_agent_mention_complete_payload_binding.py
  • tests/test_agent_mention_idempotency.py
  • tests/test_agent_mention_router.py
  • tests/test_agent_mention_sweep.py
  • tests/test_agent_mention_workflow_contract.py
  • tests/test_materialize_base_python_requirements.py
  • tests/test_opencode_agent_contract.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@seonghobae

Copy link
Copy Markdown
Contributor Author

@cwl-noema-review please review this exact head. Independent Noema review is required for the two-approval gate; this comment is the conversation-comment path that already works on main while the review-comment/review-body path in this PR is still landing.

Live run 31670687388 queued @cwl-noema-review for #954 then failed the
job on POST .../reactions with 403 Resource not accessible by
integration, so no receipt was posted. Treat the reaction as optional,
give the local job pull-requests: write for conversation receipts, and
keep durable dispatch evidence authoritative.
Record that an exceptional eyes-reaction response cannot be treated as a
missed dispatch. Force the trusted-uv installer tests onto the linux
x86_64 runner path and add the control-plane architecture diagram.
Live router run 31672030631 queued Noema for #9560c253f0 and then
failed OpenCode with HTTP 422: repository_dispatch allows 10
client_payload properties and the review-only flags made 14. Nest
those flags under review_contract, bind the wrapper to that object,
and leave the flat invocation-key claim unchanged.
@seonghobae

Copy link
Copy Markdown
Contributor Author

@cwl-noema-review

Please review this exact current head (a9debe4b53c762bcea3a04f69959fdd216a3f4f0).

This head adds review-comment/review-body mention dispatch, keeps dispatch when the optional eyes reaction 403s, and nests OpenCode review-only flags under review_contract so repository_dispatch stays at GitHub's 10-key client_payload limit (live 422 on run 31672030631 after Noema queued for #956). Reviewers remain edit: deny. Do not lower the two-approval ruleset (#948).

Doctoring already required skipping pending reviews, but parse_event
and the sweep treated a dismissed body with submitted_at as a live
mention. Dispatch only submitted, non-dismissed review bodies.
@seonghobae

Copy link
Copy Markdown
Contributor Author

@cwl-noema-review Please review this exact current head (d450fc82). The head moved after the earlier mention. Independent Noema APPROVE on this SHA is required; do not lower the two-approval ruleset. Do not @opencode-agent until this PR is on main.

route-local-agent-mention on main dies after a successful dispatch
because GITHUB_TOKEN lacks the reactions permission (HTTP 403).
Keep the reaction non-fatal and declare the write so the eyes
receipt can actually appear.
@seonghobae

Copy link
Copy Markdown
Contributor Author

@opencode-agent @cwl-noema-review Review-only request for exact current head 686a4847; do not mutate or merge. Re-evaluate review-comment/review-body mention dispatch, pending/dismissed skip, and reactions: write so live route-local-agent-mention 403 on eyes does not fail after a successful queue. Independent current-head approval is required for the two-approval gate.

@seonghobae

Copy link
Copy Markdown
Contributor Author

@cwl-noema-review Please review this exact current head (686a4847). The head moved after the earlier mention. Independent Noema APPROVE on this SHA is required; do not lower the two-approval ruleset. Do not @opencode-agent until this PR is on main.

@seonghobae

Copy link
Copy Markdown
Contributor Author

@cwl-noema-review exact current head 686a4847. Independent Noema APPROVE on this SHA is required. Strix/evidence-status were cancelled, not a code failure.

Materialize a base Python lock only when every package line is an exact
SHA-256 pin or a two-token relative -r/--requirement include of a
candidate lock path. A lone --require-hashes directive, ./dotted paths,
and -r other-hashes.txt no longer enter the trusted build context.
@seonghobae

Copy link
Copy Markdown
Contributor Author

@cwl-noema-review exact current head fb6e5938 — review-comment/review-body mention dispatch; coverage now accepts only bounded relative requirement includes. Head moved after 686a4847. Independent Noema APPROVE on this SHA is required. Do not wait on two-approval. Reviewers remain edit: deny. NVIDIA_NIM_API_KEY only.

@opencode-agent opencode-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

OpenCode could not approve from deterministic current-head evidence because GitHub Checks have failed.

Findings

1. HIGH Current-head GitHub Checks - Fix failed required checks before approval

  • Problem: Failed same-head checks remain for fb6e59385853cf57ab78635ca0e186afb3673dde.
  • Root cause: The model-unavailable evidence fallback is allowed only when peer GitHub Checks are complete and clean.
  • Fix: Read and fix the failed check logs below, then rerun the current-head checks.
  • Regression test: Keep the model-unavailable fallback gated on an empty failed-check rollup.

Failed checks:

Changed-File Evidence Map

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Workflow (2 files)"]
  S1 --> I1["GitHub Actions review job"]
  I1 --> R1["Review risk: Workflow (2 files)"]
  R1 --> V1["actionlint plus required checks"]
  Evidence --> S2["Changed file (7 files)"]
  S2 --> I2["repository behavior"]
  I2 --> R2["Review risk: Changed file (7 files)"]
  R2 --> V2["required checks"]
  Evidence --> S3["Docs (2 files)"]
  S3 --> I3["operator or user guidance"]
  I3 --> R3["Review risk: Docs (2 files)"]
  R3 --> V3["docs review"]
  Evidence --> S4["CI script (4 files)"]
  S4 --> I4["review and security gate shell path"]
  I4 --> R4["Review risk: CI script (4 files)"]
  R4 --> V4["bash -n plus Strix self-test"]
  Evidence --> S5["Test (7 files)"]
  S5 --> I5["regression suite"]
  I5 --> R5["Review risk: Test (7 files)"]
  R5 --> V5["targeted test run"]
Loading

@opencode-agent

Copy link
Copy Markdown
Contributor

OpenCode Review Overview

  • Head SHA: fb6e59385853cf57ab78635ca0e186afb3673dde
  • Workflow run: 31725443019
  • Workflow attempt: 1
  • Gate result: REQUEST_CHANGES (approval step)

Pull request overview

OpenCode could not approve from deterministic current-head evidence because GitHub Checks have failed.

Findings

1. HIGH Current-head GitHub Checks - Fix failed required checks before approval

  • Problem: Failed same-head checks remain for fb6e59385853cf57ab78635ca0e186afb3673dde.
  • Root cause: The model-unavailable evidence fallback is allowed only when peer GitHub Checks are complete and clean.
  • Fix: Read and fix the failed check logs below, then rerun the current-head checks.
  • Regression test: Keep the model-unavailable fallback gated on an empty failed-check rollup.

Failed checks:

Changed-File Evidence Map

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Workflow (2 files)"]
  S1 --> I1["GitHub Actions review job"]
  I1 --> R1["Review risk: Workflow (2 files)"]
  R1 --> V1["actionlint plus required checks"]
  Evidence --> S2["Changed file (7 files)"]
  S2 --> I2["repository behavior"]
  I2 --> R2["Review risk: Changed file (7 files)"]
  R2 --> V2["required checks"]
  Evidence --> S3["Docs (2 files)"]
  S3 --> I3["operator or user guidance"]
  I3 --> R3["Review risk: Docs (2 files)"]
  R3 --> V3["docs review"]
  Evidence --> S4["CI script (4 files)"]
  S4 --> I4["review and security gate shell path"]
  I4 --> R4["Review risk: CI script (4 files)"]
  R4 --> V4["bash -n plus Strix self-test"]
  Evidence --> S5["Test (7 files)"]
  S5 --> I5["regression suite"]
  I5 --> R5["Review risk: Test (7 files)"]
  R5 --> V5["targeted test run"]
Loading

@opencode-agent
opencode-agent Bot disabled auto-merge August 13, 2026 20:17
@seonghobae
seonghobae enabled auto-merge (squash) August 13, 2026 20:17
@opencode-agent
opencode-agent Bot disabled auto-merge August 13, 2026 21:48
@seonghobae
seonghobae enabled auto-merge (squash) August 13, 2026 22:12
@opencode-agent
opencode-agent Bot disabled auto-merge August 13, 2026 22:17
@seonghobae
seonghobae enabled auto-merge (squash) August 13, 2026 22:20
@opencode-agent
opencode-agent Bot disabled auto-merge August 13, 2026 22:20
@seonghobae
seonghobae enabled auto-merge (squash) August 13, 2026 22:27
@opencode-agent
opencode-agent Bot disabled auto-merge August 13, 2026 22:36
Review-comment dispatch heads still declared a non-existent GITHUB_TOKEN
scope. Issue-comment eyes use issues: write; review-comment eyes use
pull-requests: write. NVIDIA_NIM_API_KEY only.
@seonghobae
seonghobae enabled auto-merge (squash) August 13, 2026 22:44
@seonghobae

Copy link
Copy Markdown
Contributor Author

@cwl-noema-review exact current head 792c6ce8 — review-comment/review-body mention dispatch; drop invalid reactions:write. Head moved after fb6e5938. Independent Noema APPROVE on this SHA is required. Do not wait on two-approval. Reviewers remain edit: deny. NVIDIA_NIM_API_KEY only.

@seonghobae
seonghobae marked this pull request as draft August 14, 2026 09:08
auto-merge was automatically disabled August 14, 2026 09:08

Pull request was converted to draft

Copy link
Copy Markdown
Contributor Author

Returned to Draft because the exact current tree mixes the review-comment/review-body mention routing repair with an unrelated trusted-uv materializer branch.

The buyer-visible slice is coherent: support GitHub-shaped pull_request_review_comment and pull_request_review events, sweep all three comment/review surfaces, hydrate PR identity correctly, keep trusted-actor/idempotency/read-only review authority, and remove the case-sensitive workflow prefilter. Head 792c6ce8c9040eff4892fbf50d922eb1f6b9fe10 additionally changes materialize_base_python_requirements.py and its tests.

Rebuild from protected main retaining only the two mention workflows, router/sweep production code, review prompt surfaces directly required by the new invocation contract, focused fixtures/regressions, doctoring/governance/changelog evidence, and no materializer changes. Then reacquire exact-head router/security/supply-chain and independent-review evidence.

Copy link
Copy Markdown
Contributor Author

Closing as superseded by canonical cumulative #991. #991 contains this review-comment/review-body dispatch foundation plus the correct review-comment REST reaction path, submitted-review GraphQL reaction with idempotent already-reacted handling, and cached node_id reuse with lookup fallback. Keep one reconstructed clean branch rather than merging four overlapping router trees.

@seonghobae seonghobae closed this Aug 14, 2026
@github-project-automation github-project-automation Bot moved this from In Progress to Done in naruon Platform Roadmap Aug 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant