Skip to content

ci: review PRs with the Nitpick flow - #9

Merged
queso merged 3 commits into
mainfrom
chore/nitpick-review
Aug 21, 2026
Merged

queso merged 3 commits into
mainfrom
chore/nitpick-review

Conversation

@queso

@queso queso commented Aug 9, 2026 •

Copy link
Copy Markdown
Owner

Adds the thin caller for nitpick-flow's reusable review workflow, so FlowSpec PRs get the multi-station AI review (sharded reviewer fan-out -> coordinator -> one batched PR review).

Modeled on print-farm's caller, which is the right precedent: print-farm and FlowSpec are both owned by queso, so the private ghcr.io/queso/nitpick-flow image needs only a package grant (nitpick-flow package -> Manage Actions access), not a registry_token. The GHCR_READ_PAT that conduit-harness carries is for the CROSS-ORG case (theaiteam-dev) and is not needed here — a long-lived PAT in a public repo would be cost without benefit.

One deviation from that template: the fork guard. FlowSpec is the first PUBLIC repo to consume this workflow, so it is the first to meet a case the private consumers never could — GitHub withholds secrets from fork PRs, and without the guard an outside contributor's PR fails on a missing conduit_api_key, showing a red check they cannot fix. Same-repo branches review normally. Not solved with pull_request_target, which would put secrets in scope against untrusted PR code.

Inert until both prerequisites are in place (package grant + FIREWORKS_AI_API_KEY secret); until then no run is triggered that could fail confusingly, because the workflow only fires on PRs to this repo.

Summary by CodeRabbit

  • Chores
    • Added automated pull request review checks for eligible pull requests.
    • Configured appropriate permissions and secure API key handling for the review workflow.

Adds the thin caller for nitpick-flow's reusable review workflow, so FlowSpec
PRs get the multi-station AI review (sharded reviewer fan-out -> coordinator ->
one batched PR review).

Modeled on print-farm's caller, which is the right precedent: print-farm and
FlowSpec are both owned by `queso`, so the private ghcr.io/queso/nitpick-flow
image needs only a package grant (nitpick-flow package -> Manage Actions
access), not a registry_token. The GHCR_READ_PAT that conduit-harness carries is
for the CROSS-ORG case (theaiteam-dev) and is not needed here — a long-lived PAT
in a public repo would be cost without benefit.

One deviation from that template: the fork guard. FlowSpec is the first PUBLIC
repo to consume this workflow, so it is the first to meet a case the private
consumers never could — GitHub withholds secrets from fork PRs, and without the
guard an outside contributor's PR fails on a missing conduit_api_key, showing a
red check they cannot fix. Same-repo branches review normally. Not solved with
pull_request_target, which would put secrets in scope against untrusted PR code.

Inert until both prerequisites are in place (package grant + FIREWORKS_AI_API_KEY
secret); until then no run is triggered that could fail confusingly, because the
workflow only fires on PRs to this repo.
@coderabbitai

coderabbitai Bot commented Aug 9, 2026 •

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@queso, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 10 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 110e2d21-0aca-457e-bb13-789e8ce2a912

📥 Commits

Reviewing files that changed from the base of the PR and between 74afdaa and d7cc7cb.

📒 Files selected for processing (1)
  • .github/workflows/pr-review.yml
📝 Walkthrough

Walkthrough

Added a GitHub Actions workflow for pull-request reviews. The workflow invokes queso/nitpick-flow, applies pull-request event filters, restricts execution to same-repository branches, configures permissions, and passes FIREWORKS_AI_API_KEY.

Changes

Pull Request Review Workflow

Layer / File(s) Summary
Nitpick workflow integration
.github/workflows/pr-review.yml
The workflow triggers on selected pull-request events, checks for same-repository branches, invokes queso/nitpick-flow, grants required permissions, and maps the API key secret.

Estimated code review effort: 2 (Simple) | ~10 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the CI change that enables Nitpick flow reviews for pull requests.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch chore/nitpick-review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/pr-review.yml:
- Line 38: Update the reusable workflow reference in the workflow’s uses
declaration to pin nitpick.yml to a full-length immutable commit SHA instead of
the mutable main branch, preserving the existing workflow path and permissions.
- Around line 25-26: Update the reusable workflow reference in the PR review job
so its uses target is accessible to the public FlowSpec repository. Move the
workflow to a public repository, add a public wrapper, or replace the current
nitpick-flow reference with an allowed private repository while preserving the
existing review behavior.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 683df2f6-01b8-4721-bf41-fc8935c7fae0

📥 Commits

Reviewing files that changed from the base of the PR and between 844c5b2 and 74afdaa.

📒 Files selected for processing (1)
  • .github/workflows/pr-review.yml

Comment thread .github/workflows/pr-review.yml Resolved
Comment thread .github/workflows/pr-review.yml Outdated
# If FlowSpec stops taking outside contributions, delete this `if` and the caller matches
# print-farm exactly.
if: github.event.pull_request.head.repo.full_name == github.repository
uses: queso/nitpick-flow/.github/workflows/nitpick.yml@main

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/usr/bin/env bash
set -euo pipefail

ref="$(rg -oP 'nitpick\.yml@\K[0-9A-Za-z._/-]+' .github/workflows/pr-review.yml | head -n1)"

if [[ ! "$ref" =~ ^[0-9a-f]{40}$ ]]; then
  echo "The reusable workflow is not pinned to a full commit SHA."
  exit 1
fi

gh api "repos/queso/nitpick-flow/commits/$ref" --jq '.sha' | grep -Fx "$ref"

Repository: queso/FlowSpec

Length of output: 210


🏁 Script executed:

#!/usr/bin/env bash
set -euo pipefail

echo "== workflow line =="
sed -n '30,45p' .github/workflows/pr-review.yml

echo
echo "== reusable workflow call refs =="
rg -n "uses:\s*queso/nitpick-flow/.github/workflows/nitpick\.yml@" .github/workflows/pr-review.yml

echo
echo "== resolved main ref (if gh is available) =="
if command -v gh >/dev/null 2>&1; then
  gh api repos/queso/nitpick-flow/git/refs/heads/main --jq '{ref, object_type: .object.type, object_sha: .object.sha}' || true
else
  echo "gh not installed"
fi

Repository: queso/FlowSpec

Length of output: 1165


Pin the reusable workflow to an immutable commit.

queso/nitpick-flow/.github/workflows/nitpick.yml@main is mutable, so downstream control of conduit_api_key or pull-requests: write could shift with an unrelated branch update. Use a full-length commit SHA for this reusable-workflow call.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/pr-review.yml at line 38, Update the reusable workflow
reference in the workflow’s uses declaration to pin nitpick.yml to a full-length
immutable commit SHA instead of the mutable main branch, preserving the existing
workflow path and permissions.

Source: MCP tools

The `uses:` caller could never run. FlowSpec is PUBLIC and queso/nitpick-flow is
PRIVATE, and a public caller cannot resolve a private reusable workflow — the
run died in 0s with no jobs scheduled, and Actions showed the file path instead
of the workflow name (PR #9, run 31291660149). nitpick-flow's Actions access is
already `access_level: user`, which is what made this look like it should work:
that setting shares its workflows with other queso-owned repos but does not
extend to a public caller. print-farm consumes the same workflow happily because
print-farm is private.

Inlined the job instead, which is what theaiteam-dev/conduit-harness already
does for the sibling reason (cross-org rather than public).

Inlined from nitpick.yml @ 6bbcb0b, NOT from conduit-harness's copy — that one
has drifted and is missing the paginated diff fallback, the 60k-line safety
valve, the spider step, and the whole-run retry. Diffed against upstream: every
step's shell body is byte-identical. The only changes are the four `with:`
inputs pinned as `env:`, `conduit_api_key` -> the FIREWORKS_AI_API_KEY secret,
`registry_token || GITHUB_TOKEN` -> GITHUB_TOKEN (same-owner package grant, so
no PAT — conduit-harness needs one only because it is cross-org), and the fork
guard.

The fork guard is new to this repo: FlowSpec is public, GitHub withholds secrets
from fork PRs, and without it an outside contributor's PR fails on a missing key
they cannot supply. Not pull_request_target, which would put secrets in scope
against untrusted PR code.

Still requires the package grant (nitpick-flow package -> Manage Actions access
-> queso/flowspec, Read); the image is private and that is independent of
workflow access. Header records provenance so upstream changes get re-inlined
rather than hand-patched.

Inlining is the stopgap. A GitHub App is the real fix for both the
workflow-access and image-access problems.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick review — approve

This change adds a 281-line GitHub Actions workflow for PR review, altering CI behavior. No specific defects were identified during the review. However, given the size and untested nature of this infrastructure change, its risk profile warrants human review to ensure it meets repository automation and security requirements.

0 inline comment(s).

Verdict was approve, posted as a comment — this repo does not allow GitHub Actions to approve PRs.

The inline carried nitpick-flow's comments over verbatim, on the reasoning that
they hold the operational knowledge. That reasoning was right for a private repo
and wrong here: FlowSpec is public, so those comments — and this repo's Actions
logs — are world-readable.

Removed from public view: the print-farm#48 diff statistics, the Fireworks
prefill measurements (~110k-token prompts, 60-158s per call, 2.5x variance) and
the Bun fetch-ceiling interaction, upstream engine issue numbers and what each
bug is, the escape-pilot conclusion about the diff cap, and the discarded-work
cost figures. None of that is needed to operate this workflow; all of it is
detail about how the reviewer works and what it cost to learn.

Kept everything needed to maintain the file: the fork-guard rationale, the
prerequisites, and the non-obvious mechanics a maintainer would otherwise break
(--entrypoint bun, the /tmp-vs-/flow ownership rule, why envsubst is scoped to
two vars, why GHCR login must precede the spider, why the retry needs a fresh
volume). The header now says explicitly that the full rationale lives upstream
and that this copy should be re-inlined rather than hand-patched.

Verified mechanical: all 140 non-comment lines are byte-identical to the
previous commit. 292 -> 232 lines. No behavior change.

Note the prompts themselves were never exposed — flow.yaml references
prompts/*.md by path rather than inlining them, they stay inside the private
image, and report-failure.mjs selects only span metadata (station, attempt,
name, model, tokens, duration), never prompt or completion text.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick review — approve

This change adds a new GitHub Actions workflow for PR review, introducing 231 lines of CI configuration to automate repository infrastructure. The modification does not affect application logic or secrets, and no issues were identified during the review. The change is approved.

0 inline comment(s).

Verdict was approve, posted as a comment — this repo does not allow GitHub Actions to approve PRs.

@queso
queso merged commit d304c12 into main Aug 21, 2026
2 checks passed
@queso
queso deleted the chore/nitpick-review branch August 21, 2026 14:06
@github-actions

Copy link
Copy Markdown

🎉 This PR is included in version 0.2.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant