diff --git a/CHANGELOG.md b/CHANGELOG.md index 26ba69f9..590d269f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,12 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), ## [Unreleased] +## [1.5.0] - 2026-11-21 + +### Added + +- With `approve` on and the new `approver-private-key` input, the approval is submitted as the NCode Review Approver, an app installed with Contents: write on the repositories that approve, with a token minted after the agents exit that can only write pull requests, so the approval meets a ruleset's required approval; the NCode Review app's own earlier approvals are dismissed, and without the key NCode Review approves as its own app, which meets no required approval. `approver-client-id` names the app. `tools/Enable-CodeReview.ps1 -ApproverPrivateKeyPath` stores the key in each repository's `code-review` environment and adds the repository to the approver's installation ([#232](https://github.com/NCodeGroup/code-review/issues/232), [ADR-1011](docs/decisions/1011-the-approval-is-submitted-by-an-approver-app-that-can-write.md)). + ### Changed - The template's `node_lint` answer is on, matching the project's Biome lint: template updates render CI with its `npm run lint` step instead of treating it as the project's own, VS Code recommends the Biome extension, and `docs/best-practices-criteria.md` counts `coding_standards_enforced`, `warnings` and `warnings_fixed` as met ([#219](https://github.com/NCodeGroup/code-review/issues/219)). @@ -287,7 +293,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), - The first release: a GitHub composite action that reviews a pull request with reviewers on the Claude Agent SDK and posts the findings as the "Code review" check run, a summary comment and inline threads. -[unreleased]: https://github.com/NCodeGroup/code-review/compare/v1.4.2...HEAD +[unreleased]: https://github.com/NCodeGroup/code-review/compare/v1.5.0...HEAD +[1.5.0]: https://github.com/NCodeGroup/code-review/compare/v1.4.2...v1.5.0 [1.4.2]: https://github.com/NCodeGroup/code-review/compare/v1.4.1...v1.4.2 [1.4.1]: https://github.com/NCodeGroup/code-review/compare/v1.4.0...v1.4.1 [1.4.0]: https://github.com/NCodeGroup/code-review/compare/v1.3.1...v1.4.0 diff --git a/README.md b/README.md index a52e634c..f2dc66e4 100644 --- a/README.md +++ b/README.md @@ -205,6 +205,8 @@ before 1.0.0 changed it in minor releases; their changelog sections say how. | `service-account-id` | no | `svac_012yav3w3rfhH5i9bAgoWLHe` | The Anthropic service account the federation rule maps to. | | `workspace-id` | no | `wrkspc_019f5RLk5SG4XWE8XKwWFHZh` | The Anthropic workspace that pays for the review. | | `approve` | no | `false` | Submits a review whose verdict is Approve as an approving review, so a ruleset can require one approval; every other review stays a comment and dismisses the bot's earlier approval. | +| `approver-private-key` | no | | The NCode Review Approver app's private key, from the repository's code-review environment; with `approve` on, the approval is submitted as the approver, which meets a required approval. | +| `approver-client-id` | no | `Iv23liDPFX1K6NjGIr9i` | The NCode Review Approver app's client ID. | The action has no outputs. A caller needs: @@ -363,13 +365,24 @@ old approval standing. Drafts are approved like any pull request; fork pull requ The summary comment has an `Approval:` line that says "Approved" or "Not approved" with the reason. When GitHub refuses an approval, the run warns, dismisses the earlier approval too, and the summary says the approval couldn't be submitted. +GitHub counts an approval toward a ruleset's required approval only from a reviewer that can write to the repository, +whatever its `author_association`. The NCode Review app can't, so its approval counts for Scorecard's Code-Review check +and meets no required approval. With `approver-private-key`, the NCode Review Approver submits the approval instead: an +app with Contents: write and Pull requests: write, installed only on the repositories that approve, whose token is +minted after the agents exit with Pull requests: write alone, so it can't push. The NCode Review app's own earlier +approvals are dismissed, so the pull request carries one approval from NCode Review. +`tools/Enable-CodeReview.ps1 -ApproverPrivateKeyPath` stores the key in each repository's `code-review` environment, +which only the default branch can deploy to, so a pull request's workflow never reads it. + A push removes the approval when the ruleset dismisses stale reviews and requires approval of the last push, and `/review` approves again. `/run-feedback` and `/run-issues` comment it after each push; after a hand push, comment -`/review` yourself, because NCode Review doesn't review on `synchronize`. Require one approval in the ruleset. +`/review` yourself, because NCode Review doesn't review on `synchronize`. With the approver, require one approval, of +the last push, in the ruleset; without it, require none where one person can write. -The trade-offs: Scorecard's Code-Review check describes human review, so the app's approval may not score as a -person's does, and an NCode Review outage or budget stop blocks merges until it is fixed or the ruleset changes. See -[ADR-1009](docs/decisions/1009-a-clean-review-approves-the-pull-request.md). +The trade-offs: an NCode Review outage or budget stop blocks merges where the approval is required until it is fixed or +the ruleset changes, and a leaked approver key can mint a Contents: write token for the repositories the approver is +installed on. See [ADR-1009](docs/decisions/1009-a-clean-review-approves-the-pull-request.md) and +[ADR-1011](docs/decisions/1011-the-approval-is-submitted-by-an-approver-app-that-can-write.md). ### Outcomes diff --git a/action.yml b/action.yml index 03be7b99..58b64b7f 100644 --- a/action.yml +++ b/action.yml @@ -48,6 +48,17 @@ inputs: comment. required: false default: 'false' + approver-private-key: + description: >- + The NCode Review Approver app's private key (the NCODE_REVIEW_APPROVER_PRIVATE_KEY secret of the repository's + code-review environment). With approve on, the approval is submitted as the approver, whose approval meets a + ruleset's required approval; without it, NCode Review approves as its own app, whose approval doesn't. + required: false + default: '' + approver-client-id: + description: The NCode Review Approver app's client ID. + required: false + default: Iv23liDPFX1K6NjGIr9i runs: using: composite @@ -215,6 +226,21 @@ runs: repositories: ${{ steps.review.outputs.guidance-repository }} permission-issues: write + # The approver's token, minted after the agents exit, for this repository only and able to write only pull requests: + # the publish step uses it to approve and to dismiss the approver's own approvals. The approver app's write access + # to the repository is what makes GitHub count its approval toward a required approval, while this token can't push. + # It may fail: the publish step then warns and approves with the NCode Review app. + - name: Get the approver's token + id: approver-token + if: steps.review.outputs.reviewed == 'true' && inputs.approve == 'true' && inputs.approver-private-key != '' + continue-on-error: true + uses: actions/create-github-app-token@64ab729a6294000f5aa4a4bc0b73d1f32bc45510 # v3.3.0 + with: + client-id: ${{ inputs.approver-client-id }} + private-key: ${{ inputs.approver-private-key }} + repositories: ${{ github.event.repository.name }} + permission-pull-requests: write + - name: Publish id: publish if: >- @@ -229,6 +255,9 @@ runs: CODE_REVIEW_CHECK_RUN_ID: ${{ steps.ack.outputs.check-run-id }} CODE_REVIEW_CHECK_RUN_SHA: ${{ steps.ack.outputs.head-sha }} CODE_REVIEW_APPROVE: ${{ inputs.approve }} + CODE_REVIEW_APPROVER: ${{ inputs.approver-private-key != '' }} + CODE_REVIEW_APPROVER_TOKEN: ${{ steps.approver-token.outputs.token }} + CODE_REVIEW_APPROVER_LOGIN: ${{ steps.approver-token.outputs.app-slug && format('{0}[bot]', steps.approver-token.outputs.app-slug) || '' }} run: node "$GITHUB_ACTION_PATH/src/publish.ts" # Completes the check run when the publish step didn't, so it never stays in progress: neutral "Review budget used diff --git a/docs/decisions/1009-a-clean-review-approves-the-pull-request.md b/docs/decisions/1009-a-clean-review-approves-the-pull-request.md index 9779e05f..fbeb583a 100644 --- a/docs/decisions/1009-a-clean-review-approves-the-pull-request.md +++ b/docs/decisions/1009-a-clean-review-approves-the-pull-request.md @@ -1,6 +1,6 @@ # 1009. A clean review approves the pull request -- **Status:** Accepted +- **Status:** Accepted (amended by [1011](1011-the-approval-is-submitted-by-an-approver-app-that-can-write.md)) - **Date:** 2026-11-15 - **Deciders:** Marcin Polewski diff --git a/docs/decisions/1011-the-approval-is-submitted-by-an-approver-app-that-can-write.md b/docs/decisions/1011-the-approval-is-submitted-by-an-approver-app-that-can-write.md new file mode 100644 index 00000000..ae46223f --- /dev/null +++ b/docs/decisions/1011-the-approval-is-submitted-by-an-approver-app-that-can-write.md @@ -0,0 +1,66 @@ +# 1011. The approval is submitted by an approver app that can write + +- **Status:** Accepted +- **Date:** 2026-11-21 +- **Deciders:** Marcin Polewski +- **Amends:** 1009 + +## Context + +GitHub counts an approval toward a ruleset's required approval only from a reviewer that can write to the repository, +whatever its `author_association`. The NCode Review app can't write to a repository, so its approval counts for +Scorecard's Code-Review check and meets no required approval +([#232](https://github.com/NCodeGroup/code-review/issues/232)). + +[NCodeGroup/project-template#460](https://github.com/NCodeGroup/project-template/issues/460) tested five legs: + +- An app with no Contents access: its approval isn't counted. +- An app installed with Contents: write: its approval is counted, even when the token that submitted it has Pull + requests: write alone and can't create a branch. +- GitHub checks write access when the review is read, not when it is submitted. +- The `author_association` of an app's review is `NONE` throughout, so it doesn't decide the count. +- While "Allow GitHub Actions to create and approve pull requests" is on, an approval by the workflow's `GITHUB_TOKEN` + with Pull requests: write counts too, and a pull request that adds such a workflow approves itself. + +## Decision + +The action takes a second app, the NCode Review Approver, which has Contents: write and Pull requests: write and is +installed only on the repositories that approve. + +- Its token is minted in a step after the review step, for this repository only, with `permission-pull-requests: write` + alone, so no token it is given can push. The token reaches only the Publish step, which uses it to approve and to + dismiss its own approvals. +- Its key is a secret of each repository's `code-review` environment, which only the default branch can deploy to, + never an organization secret: an organization secret reaches a workflow on any branch, whose author could then + approve their own pull request. +- Without the key, the NCode Review app approves as it does without an approver. With the key but a token step that + failed, the publish step warns and approves with the NCode Review app. +- When the approver approves, the NCode Review app's own earlier approvals are dismissed too, so the pull request + carries one approval from NCode Review. A review that doesn't approve dismisses both apps' approvals. +- The approver's login comes from the token step's `app-slug` output as `[bot]`, so no input names it. +- `approver-client-id` defaults to the NCode Review Approver's client ID. + +## Options considered + +1. Give the NCode Review app Contents: write. Rejected: the review app is installed wherever NCode Review runs. +2. The workflow's `GITHUB_TOKEN`. Rejected: it needs "Allow GitHub Actions to create and approve pull requests", which + lets any writer's workflow approve its own pull request. +3. An organization secret for the approver's key. Rejected: any branch's workflow can read it. +4. A dedicated approver app. Chosen: it is installed only on repositories that approve, and its key is used in one step. + +## Consequences + +- A leaked approver key can mint a Contents: write token for the repositories it is installed on. +- An NCode Review outage or budget stop blocks merges where the approval is required, until it is fixed or the ruleset + changes. +- A repository that approves installs the approver and stores its key with `tools/Enable-CodeReview.ps1 + -ApproverPrivateKeyPath`. + +## References + +- `src/comments.ts` (`applyApproval`, `dismissApprovals`) +- `src/publish.ts` +- `action.yml` +- `tools/Enable-CodeReview.ps1` +- [#232](https://github.com/NCodeGroup/code-review/issues/232) +- [NCodeGroup/project-template#460](https://github.com/NCodeGroup/project-template/issues/460) diff --git a/docs/decisions/README.md b/docs/decisions/README.md index b9ae2f8c..916a75e3 100644 --- a/docs/decisions/README.md +++ b/docs/decisions/README.md @@ -74,5 +74,6 @@ records are indexed in [project-template/](project-template/README.md). | [1006](1006-a-plan-that-reaches-consumers-releases-when-its-pull-request-merges.md) | A plan that reaches consumers releases when its pull request merges, and this repository's own pin is exempt from the Dependabot cooldown | Accepted | | [1007](1007-a-re-review-larger-than-one-part-is-a-full-review.md) | A re-review whose changed hunks are more than one part of a first review holds is a full review, in parts and under the run's cost ceiling | Accepted | | [1008](1008-one-problem-gets-one-thread.md) | One problem gets one thread: synthesis merges it across files with an "Also at" list, and a re-review lists a finding its reviewer says repeats an earlier one as already raised | Accepted | -| [1009](1009-a-clean-review-approves-the-pull-request.md) | A clean review approves the pull request when the caller sets `approve: true`, and any other review dismisses the bot's approval | Accepted | +| [1009](1009-a-clean-review-approves-the-pull-request.md) | A clean review approves the pull request when the caller sets `approve: true`, and any other review dismisses the bot's approval | Amended by 1011 | | [1010](1010-biome-formats-and-lints-the-typescript.md) | Biome formats and lints the TypeScript as biome.json sets, and CI's npm run lint fails on any difference | Accepted | +| [1011](1011-the-approval-is-submitted-by-an-approver-app-that-can-write.md) | With an approver key, the approval is submitted by the NCode Review Approver, an app that can write, with a token that can only write pull requests, so it meets a required approval; amends 1009 | Accepted | diff --git a/src/comments.ts b/src/comments.ts index 260122fa..2b5a5dcd 100644 --- a/src/comments.ts +++ b/src/comments.ts @@ -172,12 +172,13 @@ export async function submitApproval( } } -// Dismisses the bot's own approving reviews of the pull request, so an approval of an earlier head doesn't stand after a -// review that doesn't approve. A failure only warns. +// Dismisses the approving reviews of the pull request that `login` (the NCode Review app by default) submitted, so an +// approval of an earlier head doesn't stand after a review that doesn't approve. A failure only warns. export async function dismissApprovals( api: Api, results: Pick, log: (line: string) => void = writeLog, + login: string = botLogin, ): Promise { const reviews = `/repos/${results.repository}/pulls/${results.pullRequest}/reviews`; type ListedReview = { id: number; state?: string; user?: { login?: string } }; @@ -195,7 +196,7 @@ export async function dismissApprovals( return; } for (const review of listed) { - if (review.user?.login !== botLogin || review.state !== 'APPROVED') continue; + if (review.user?.login !== login || review.state !== 'APPROVED') continue; try { await api('PUT', `${reviews}/${review.id}/dismissals`, { message: `Dismissed: the review at ${results.headSha.slice(0, 7)} doesn't approve.`, @@ -207,21 +208,58 @@ export async function dismissApprovals( } } +// The approver app's API client and its login, which approves and dismisses its own approvals in place of the NCode +// Review app. +export interface Approver { + api: Api; + login: string; +} + +// The approver the publish step's env names: its API and login when the token step minted a token, so the approval +// meets a required approval, and the token, a secret wherever the app's is. `missing` says the caller gave the +// approver's key (CODE_REVIEW_APPROVER is 'true') but no approver came, so NCode Review approves as its own app. +export function approverFromEnv( + env: NodeJS.ProcessEnv, + makeApi: (token: string) => Api, +): { approver?: Approver; token?: string; missing: boolean } { + const token = env['CODE_REVIEW_APPROVER_TOKEN'] || undefined; + const login = env['CODE_REVIEW_APPROVER_LOGIN'] || undefined; + const approver = token && login ? { api: makeApi(token), login } : undefined; + return { + ...(approver ? { approver } : {}), + ...(token ? { token } : {}), + missing: env['CODE_REVIEW_APPROVER'] === 'true' && !approver, + }; +} + +// The warning for a run whose caller gave the approver's key but got no approver. +export const approverMissingWarning = + "::warning::The approver's token couldn't be minted, so NCode Review approves with its own app, whose approval meets no required approval."; + // Approves when the review qualifies; otherwise, and when the approval is refused, dismisses the bot's earlier approval, -// so a review that doesn't approve never leaves an old approval standing. +// so a review that doesn't approve never leaves an old approval standing. With an approver, it submits and dismisses +// the approver's approvals, and dismisses the NCode Review app's own too, so one approval from NCode Review stands. export async function applyApproval( api: Api, results: Pick, approve: boolean, secrets: Iterable, log: (line: string) => void = writeLog, + approver?: Approver, ): Promise<'approved' | 'not-approved' | 'refused'> { - if (!approve) { + const dismissAll = async () => { await dismissApprovals(api, results, log); + if (approver) await dismissApprovals(approver.api, results, log, approver.login); + }; + if (!approve) { + await dismissAll(); return 'not-approved'; } - if (await submitApproval(api, results, secrets, log)) return 'approved'; - await dismissApprovals(api, results, log); + if (await submitApproval(approver?.api ?? api, results, secrets, log)) { + if (approver) await dismissApprovals(api, results, log); + return 'approved'; + } + await dismissAll(); return 'refused'; } diff --git a/src/publish.ts b/src/publish.ts index 69a3064a..76c4839b 100644 --- a/src/publish.ts +++ b/src/publish.ts @@ -18,7 +18,15 @@ import path from 'node:path'; import { acknowledgedFrom, publishCheck } from './checks.ts'; import { assertNoSecrets } from './outputs.ts'; import { approvalFor, approvalLine, approveOn } from './approval.ts'; -import { applyApproval, postSummary, postThreads, errorText, readPostedThreads } from './comments.ts'; +import { + applyApproval, + approverFromEnv, + approverMissingWarning, + postSummary, + postThreads, + errorText, + readPostedThreads, +} from './comments.ts'; import { publishAnswer } from './explain.ts'; import { dismissedThreads, followUpThreads, planFollowUps } from './follow-up.ts'; import { githubApi, githubGraphql } from './github.ts'; @@ -56,11 +64,15 @@ async function main(): Promise { const outDir = process.env['CODE_REVIEW_OUT']; if (!token || !outDir) throw new Error('CODE_REVIEW_APP_TOKEN and CODE_REVIEW_OUT must be set.'); const api = githubApi(token); + // The approver's token is minted only for a run that approves. Its API submits the approval, and its token is a secret + // wherever the app's is. + const { approver, token: approverToken, missing: approverMissing } = approverFromEnv(process.env, githubApi); + const secrets = approverToken ? [token, approverToken] : [token]; // A closed pull request's run records outcomes and has no review to publish, nor a check run to complete. It never // pauses, so it comes first. const report = readOutcomes(outDir); if (report) { - await publishOutcomes(api, report, [token]); + await publishOutcomes(api, report, secrets); return; } // A paused run posts only the notice. It leaves an acknowledged check run to the finish step, which completes it with @@ -80,7 +92,7 @@ async function main(): Promise { return; } // A `/review explain` run leaves one comment to post instead of a review. - const answerUrl = await publishAnswer(outDir, api, [token]); + const answerUrl = await publishAnswer(outDir, api, secrets); if (answerUrl !== undefined) { writeLog(`Posted ${answerUrl}`); return; @@ -132,7 +144,7 @@ async function main(): Promise { annotations: annotations(results), }; const check = { head_sha: results.headSha, conclusion: 'neutral' as const, output }; - assertNoSecrets(JSON.stringify(check), [token], 'The check run'); + assertNoSecrets(JSON.stringify(check), secrets, 'The check run'); const completed = await publishCheck(api, results.repository, check, acknowledged); writeLog(`Posted ${completed.html_url}`); // Tells the finish step the check run is complete, so it leaves it alone. @@ -142,17 +154,20 @@ async function main(): Promise { await postThreads( api, results, - [token], + secrets, writeLog, dismissed.map((d) => d.posted), posted, ); if (threads) { const actions = await planFollowUps(threads, results, isWriter, new Set(dismissed.map((d) => d.thread.id))); - await followUpThreads((query, variables) => githubGraphql(api, query, variables), actions, [token]); + await followUpThreads((query, variables) => githubGraphql(api, query, variables), actions, secrets); } - const state = approving ? await applyApproval(api, results, approval.approve, [token]) : 'not-approved'; + if (approving && approverMissing) writeLog(approverMissingWarning); + const state = approving + ? await applyApproval(api, results, approval.approve, secrets, writeLog, approver) + : 'not-approved'; const body = summaryComment( results, @@ -160,7 +175,7 @@ async function main(): Promise { interactUrl, approving ? approvalLine(approval, state) : undefined, ); - assertNoSecrets(body, [token], 'The summary comment'); + assertNoSecrets(body, secrets, 'The summary comment'); await postSummary(api, results, body); } diff --git a/test/acknowledge.test.ts b/test/acknowledge.test.ts index 60c9533d..bb234a48 100644 --- a/test/acknowledge.test.ts +++ b/test/acknowledge.test.ts @@ -1071,6 +1071,32 @@ test('action.yml gives the app key only to the token steps, and each token only assert.deepEqual(users('steps.app-token.outputs.token'), ['publish', 'Finish the check run']); }); +test("action.yml gives the approver's key only to its token step, and its token only to the publish step", () => { + const steps = actionSteps(); + const holders = steps.filter((s) => s.text.includes('inputs.approver-private-key') && s.id !== 'publish'); + assert.deepEqual( + holders.map((s) => s.id), + ['approver-token'], + ); + const minted = holders[0]; + assert.match(minted?.uses ?? '', /^actions\/create-github-app-token@[0-9a-f]{40} /); + assert.deepEqual(Object.keys(minted?.with ?? {}).sort(), [ + 'client-id', + 'permission-pull-requests', + 'private-key', + 'repositories', + ]); + assert.equal(minted?.with['permission-pull-requests'], 'write'); + assert.equal(minted?.with['private-key'], '${{ inputs.approver-private-key }}'); + assert.equal( + minted?.if, + "steps.review.outputs.reviewed == 'true' && inputs.approve == 'true' && inputs.approver-private-key != ''", + ); + const users = steps.filter((s) => s.text.includes('steps.approver-token.outputs.token')).map((s) => s.id || s.name); + assert.deepEqual(users, ['publish']); + assert.doesNotMatch(steps.find((s) => s.id === 'review')?.text ?? '', /approver/); +}); + test('action.yml acknowledges before setup-node and npm ci, and never lets the acknowledgement stop the review', () => { const steps = actionSteps(); const index = (id: string) => steps.findIndex((s) => s.id === id); diff --git a/test/approval.test.ts b/test/approval.test.ts index b893d8fd..8fdb1ae0 100644 --- a/test/approval.test.ts +++ b/test/approval.test.ts @@ -4,7 +4,13 @@ import assert from 'node:assert/strict'; import { test } from 'node:test'; import { approvalFor, approvalLine, approveOn } from '../src/approval.ts'; -import { applyApproval, dismissApprovals, submitApproval } from '../src/comments.ts'; +import { + applyApproval, + approverFromEnv, + approverMissingWarning, + dismissApprovals, + submitApproval, +} from '../src/comments.ts'; import type { ThreadNode } from '../src/rereview.ts'; import { summaryComment, type Results } from '../src/results.ts'; import { threadBody } from '../src/threads.ts'; @@ -319,6 +325,68 @@ test('dismissApprovals dismisses only the bot approvals', async () => { ); }); +const approverApproval = [{ id: 5, state: 'APPROVED', user: { login: 'ncode-review-approver[bot]' } }]; + +function approverPair(options: { postThrows?: boolean } = {}) { + const review = fakeApi((call) => { + if (call === 'GET /repos/o/r/pulls/7') return { head: { sha: target.headSha } }; + return call.startsWith('GET') ? [...botApproval, ...approverApproval] : {}; + }); + const approver = fakeApi((call) => { + if (call === 'GET /repos/o/r/pulls/7') return { head: { sha: target.headSha } }; + if (call.startsWith('POST') && options.postThrows) throw new Error('HTTP 422'); + return call.startsWith('GET') ? [...botApproval, ...approverApproval] : {}; + }); + return { review, approver, approverArg: { api: approver.api, login: 'ncode-review-approver[bot]' } }; +} + +test('applyApproval with an approver approves through the approver and dismisses the review app approvals', async () => { + const { review, approver, approverArg } = approverPair(); + assert.equal(await applyApproval(review.api, target, true, [], () => {}, approverArg), 'approved'); + assert.ok(approver.calls.includes('POST /repos/o/r/pulls/7/reviews')); + assert.equal(review.calls.filter((c) => c.startsWith('POST')).length, 0); + assert.deepEqual( + review.calls.filter((c) => c.startsWith('PUT')), + ['PUT /repos/o/r/pulls/7/reviews/1/dismissals'], + ); + assert.equal(approver.calls.filter((c) => c.startsWith('PUT')).length, 0); +}); + +test('applyApproval with an approver dismisses both apps approvals when approve is false', async () => { + const { review, approver, approverArg } = approverPair(); + assert.equal(await applyApproval(review.api, target, false, [], () => {}, approverArg), 'not-approved'); + assert.deepEqual( + review.calls.filter((c) => c.startsWith('PUT')), + ['PUT /repos/o/r/pulls/7/reviews/1/dismissals'], + ); + assert.deepEqual( + approver.calls.filter((c) => c.startsWith('PUT')), + ['PUT /repos/o/r/pulls/7/reviews/5/dismissals'], + ); +}); + +test('applyApproval with an approver dismisses both and says refused when the approver POST throws', async () => { + const { review, approver, approverArg } = approverPair({ postThrows: true }); + assert.equal(await applyApproval(review.api, target, true, [], () => {}, approverArg), 'refused'); + assert.deepEqual( + review.calls.filter((c) => c.startsWith('PUT')), + ['PUT /repos/o/r/pulls/7/reviews/1/dismissals'], + ); + assert.deepEqual( + approver.calls.filter((c) => c.startsWith('PUT')), + ['PUT /repos/o/r/pulls/7/reviews/5/dismissals'], + ); +}); + +test('dismissApprovals with a login dismisses only that login approvals', async () => { + const fake = fakeApi((call) => (call.startsWith('GET') ? [...botApproval, ...approverApproval] : {})); + await dismissApprovals(fake.api, target, () => {}, 'ncode-review-approver[bot]'); + assert.deepEqual( + fake.calls.filter((c) => c.startsWith('PUT')), + ['PUT /repos/o/r/pulls/7/reviews/5/dismissals'], + ); +}); + const summaryInput: Results = { repository: 'o/r', pullRequest: 1, @@ -355,3 +423,56 @@ test('summaryComment puts the approval line right after the verdict line', () => assert.equal(lines[at + 3], ''); assert.ok(lines[at + 4]!.startsWith('**0 reviewers run**'), String(lines[at + 4])); }); + +// approverFromEnv: how the publish step builds the approver from the env the token step fills. +const recordApi = () => { + const made: string[] = []; + const makeApi = (token: string) => { + made.push(token); + return async () => undefined; + }; + return { made, makeApi }; +}; + +test('approverFromEnv builds the approver from its token and login', () => { + const { made, makeApi } = recordApi(); + const got = approverFromEnv( + { + CODE_REVIEW_APPROVER: 'true', + CODE_REVIEW_APPROVER_TOKEN: 'approver-token', + CODE_REVIEW_APPROVER_LOGIN: 'ncode-review-approver[bot]', + }, + makeApi, + ); + assert.equal(got.approver?.login, 'ncode-review-approver[bot]'); + assert.equal(got.token, 'approver-token'); + assert.deepEqual(made, ['approver-token']); + assert.equal(got.missing, false); +}); + +test('approverFromEnv reports the approver missing when the key was given but no token came', () => { + const { made, makeApi } = recordApi(); + const got = approverFromEnv( + { CODE_REVIEW_APPROVER: 'true', CODE_REVIEW_APPROVER_TOKEN: '', CODE_REVIEW_APPROVER_LOGIN: '' }, + makeApi, + ); + assert.equal(got.approver, undefined); + assert.equal(got.token, undefined); + assert.equal(got.missing, true); + assert.deepEqual(made, []); + assert.match(approverMissingWarning, /^::warning::The approver's token couldn't be minted/); +}); + +test('approverFromEnv keeps a token without a login as a secret, with no approver', () => { + const { makeApi } = recordApi(); + const got = approverFromEnv({ CODE_REVIEW_APPROVER: 'true', CODE_REVIEW_APPROVER_TOKEN: 'approver-token' }, makeApi); + assert.equal(got.approver, undefined); + assert.equal(got.token, 'approver-token'); + assert.equal(got.missing, true); +}); + +test('approverFromEnv is quiet when the caller gave no approver key', () => { + const { makeApi } = recordApi(); + const got = approverFromEnv({ CODE_REVIEW_APPROVER: 'false' }, makeApi); + assert.deepEqual(got, { missing: false }); +}); diff --git a/test/enable-code-review.test.ts b/test/enable-code-review.test.ts index 340704e5..3227d470 100644 --- a/test/enable-code-review.test.ts +++ b/test/enable-code-review.test.ts @@ -14,6 +14,11 @@ import { after, describe, test } from 'node:test'; const script = path.resolve(import.meta.dirname, '..', 'tools', 'Enable-CodeReview.ps1'); +// The approver tests' key, built from parts so the source holds nothing shaped like a private key, which NCode +// Review's trace check refuses to publish when a pull request's diff carries it. +const pemDashes = '-'.repeat(5); +const fakeApproverKey = `${pemDashes}BEGIN RSA PRIVATE KEY${pemDashes}\nnot-a-real-key\n${pemDashes}END RSA PRIVATE KEY${pemDashes}\n`; + function findPwsh(): string | undefined { try { return execFileSync('pwsh', ['-NoProfile', '-NonInteractive', '-Command', '(Get-Process -Id $PID).Path'], { @@ -52,6 +57,7 @@ type Call = { args: string[]; stdin: boolean }; const org = 'NCodeGroup'; const appKey = 'NCODE_REVIEW_APP_PRIVATE_KEY'; const bedrock = 'AWS_BEARER_TOKEN_BEDROCK'; +const approverKey = 'NCODE_REVIEW_APPROVER_PRIVATE_KEY'; // An organization with the app installed on all repositories, the repositories alpha (id 101) and beta (id 102), and // the app key limited to code-review. The Bedrock key is limited to code-review, visible to all repositories, or @@ -64,7 +70,12 @@ function scenario(bedrockKey: 'selected' | 'all' | 'missing' = 'selected'): Answ return [ { match: `^api orgs/${org}/installations`, - stdout: JSON.stringify({ installations: [{ id: 5, app_slug: 'ncode-review', repository_selection: 'all' }] }), + stdout: JSON.stringify({ + installations: [ + { id: 5, app_slug: 'ncode-review', repository_selection: 'all' }, + { id: 6, app_slug: 'ncode-review-approver', repository_selection: 'selected' }, + ], + }), }, { match: `^api repos/${org}/alpha$`, @@ -281,4 +292,72 @@ describe('tools/Enable-CodeReview.ps1', { concurrency: true }, () => { assert.ok(set?.stdin, lines.join('\n')); assert.ok(!lines.some((line) => line.includes(bedrock) && line.includes('-X PUT')), lines.join('\n')); }); + + test("-ApproverPrivateKeyPath sets the environment secret of each repository and adds it to the approver's installation", { + skip, + }, async () => { + const pem = path.join(tempDir(), 'approver.pem'); + writeFileSync(pem, fakeApproverKey); + const installed: Answer = { match: '^api user/installations/6/repositories --paginate', stdout: `${org}/beta` }; + const { status, output, calls } = await run( + ['-Repo', 'alpha,beta', '-ApproverPrivateKeyPath', pem, '-SkipBedrock'], + [installed, ...scenario()], + ); + assert.equal(status, 0, output); + const lines = joined(calls); + assert.ok(lines.includes('api -X PUT user/installations/6/repositories/101 --silent'), lines.join('\n')); + assert.ok(!lines.includes('api -X PUT user/installations/6/repositories/102 --silent'), lines.join('\n')); + for (const name of ['alpha', 'beta']) { + const set = calls.find( + (call) => call.args.join(' ') === `secret set ${approverKey} --env code-review --repo ${org}/${name}`, + ); + assert.ok(set?.stdin, lines.join('\n')); + assert.match(output, new RegExp(`${org}/${name}: set the code-review environment secret ${approverKey}\\.`)); + } + assert.ok( + !lines.some((line) => line.startsWith('secret set') && line.includes('--org') && line.includes(approverKey)), + ); + assert.doesNotMatch(output, /not-a-real-key/); + }); + + test('-ApproverPrivateKeyPath without -Repo stops before any change', { skip }, async () => { + const pem = path.join(tempDir(), 'approver.pem'); + writeFileSync(pem, fakeApproverKey); + const { status, output, calls } = await run(['-ApproverPrivateKeyPath', pem], scenario()); + assert.notEqual(status, 0, output); + assert.match(output, /-ApproverPrivateKeyPath needs -Repo/); + assert.deepEqual(calls, []); + }); + + test("-ApproverPrivateKeyPath stops on a file that isn't a PEM private key, or is empty, before setting a secret", { + skip, + }, async () => { + for (const text of ['not a key\n', '']) { + const pem = path.join(tempDir(), 'approver.pem'); + writeFileSync(pem, text); + const { status, output, calls } = await run( + ['-Repo', 'alpha', '-ApproverPrivateKeyPath', pem, '-SkipBedrock'], + scenario(), + ); + assert.notEqual(status, 0, output); + assert.match(output, /isn't a PEM private key/); + assert.ok(!joined(calls).some((line) => line.startsWith('secret set')), joined(calls).join('\n')); + } + }); + + test("-ApproverPrivateKeyPath stops when the approver app isn't installed", { skip }, async () => { + const pem = path.join(tempDir(), 'approver.pem'); + writeFileSync(pem, fakeApproverKey); + const withoutApprover: Answer = { + match: `^api orgs/${org}/installations`, + stdout: JSON.stringify({ installations: [{ id: 5, app_slug: 'ncode-review', repository_selection: 'all' }] }), + }; + const { status, output, calls } = await run( + ['-Repo', 'alpha', '-ApproverPrivateKeyPath', pem, '-SkipBedrock'], + [withoutApprover, ...scenario()], + ); + assert.notEqual(status, 0, output); + assert.match(output, /The NCode Review Approver app isn't installed on NCodeGroup/); + assert.ok(!joined(calls).some((line) => line.startsWith('secret set')), joined(calls).join('\n')); + }); }); diff --git a/tools/Enable-CodeReview.ps1 b/tools/Enable-CodeReview.ps1 index f5c4c2ea..035c5430 100644 --- a/tools/Enable-CodeReview.ps1 +++ b/tools/Enable-CodeReview.ps1 @@ -3,8 +3,8 @@ <# .SYNOPSIS - Opt repositories into NCode Review, and set or replace the review's two keys, which are stored once as organization - secrets. + Opt repositories into NCode Review, set or replace the review's two keys, which are stored once as organization + secrets, and store the approver's key in each repository's environment. .DESCRIPTION The review uses two organization secrets, each visible only to the repositories that opted in: @@ -24,6 +24,11 @@ takes precedence and would hide a replaced organization secret; - adds it to both organization secrets' repository lists. + -ApproverPrivateKeyPath stores the NCode Review Approver app's private key as the secret + NCODE_REVIEW_APPROVER_PRIVATE_KEY of each -Repo repository's `code-review` environment, and adds each repository to + the approver's installation when the app is installed on selected repositories only. With the key, the review's + approval is submitted as the approver, whose approval meets a ruleset's required approval. + -AppPrivateKeyPath sets or replaces the app key. -UpdateBedrockKey replaces the Bedrock key; it's also set whenever the organization doesn't have it yet, unless -SkipBedrock is given. -SkipBedrock skips only the Bedrock key's value: it never prompts for it or sets it, and the repositories in -Repo still get access to the existing secret. @@ -37,6 +42,11 @@ `pwsh -File` passes one value for a parameter, so there a list has to arrive as one string, -Repo a,b or -Repo 'a, b', never -Repo a, b. +.PARAMETER ApproverPrivateKeyPath + The NCode Review Approver app's private key (.pem), stored as the secret NCODE_REVIEW_APPROVER_PRIVATE_KEY of each + -Repo repository's `code-review` environment, never as an organization secret: only jobs on the default branch can + deploy to that environment, so only they can read the key. Needs -Repo. + .PARAMETER SkipBedrock Don't prompt for or set the Bedrock key, even when the organization doesn't have it yet. The repositories in -Repo are still given access to the existing AWS_BEARER_TOKEN_BEDROCK secret. @@ -53,6 +63,7 @@ param( [string[]]$Repo = @(), [string]$AppPrivateKeyPath, + [string]$ApproverPrivateKeyPath, [switch]$UpdateBedrockKey, [switch]$SkipBedrock, [string]$Org = 'NCodeGroup' @@ -60,10 +71,13 @@ param( $ErrorActionPreference = 'Stop' $Repo = @($Repo | ForEach-Object { $_ -split ',' } | ForEach-Object { $_.Trim() } | Where-Object { $_ }) if ($UpdateBedrockKey -and $SkipBedrock) { throw 'Pass -UpdateBedrockKey or -SkipBedrock, not both.' } +if ($ApproverPrivateKeyPath -and -not $Repo) { throw '-ApproverPrivateKeyPath needs -Repo: the key is stored in each repository''s code-review environment.' } $environment = 'code-review' $appSlug = 'ncode-review' +$approverSlug = 'ncode-review-approver' $appKeySecret = 'NCODE_REVIEW_APP_PRIVATE_KEY' +$approverKeySecret = 'NCODE_REVIEW_APPROVER_PRIVATE_KEY' $bedrockSecret = 'AWS_BEARER_TOKEN_BEDROCK' $adminOrgHint = 'gh auth refresh -h github.com -s admin:org' @@ -110,12 +124,24 @@ gh auth status *> $null if ($LASTEXITCODE) { throw 'gh isn''t signed in. Run: gh auth login' } gh secret list --org $Org *> $null if ($LASTEXITCODE) { throw "gh can't read $Org's organization secrets. Grant the scope with: $adminOrgHint" } -if (-not $Repo -and -not $AppPrivateKeyPath -and -not $UpdateBedrockKey) { throw 'Nothing to do. Pass -Repo, -AppPrivateKeyPath or -UpdateBedrockKey.' } +if (-not $Repo -and -not $AppPrivateKeyPath -and -not $ApproverPrivateKeyPath -and -not $UpdateBedrockKey) { throw 'Nothing to do. Pass -Repo, -AppPrivateKeyPath, -ApproverPrivateKeyPath or -UpdateBedrockKey.' } -$installation = (Invoke-Gh api "orgs/$Org/installations?per_page=100" | ConvertFrom-Json).installations | - Where-Object app_slug -eq $appSlug | Select-Object -First 1 +# Read once: both apps' installations come from this list. +$installations = @((Invoke-Gh api "orgs/$Org/installations?per_page=100" | ConvertFrom-Json).installations) +$installation = $installations | Where-Object app_slug -eq $appSlug | Select-Object -First 1 if (-not $installation) { throw "The NCode Review app isn't installed on $Org. Install it from the app's settings page first." } +$approverValue = $null +$approverInstallation = $null +if ($ApproverPrivateKeyPath) { + $approverKeyPath = (Resolve-Path -LiteralPath $ApproverPrivateKeyPath).Path + # An empty file reads as $null, so the string keeps .Trim() from failing before the PEM check says why. + $approverValue = "$(Get-Content -LiteralPath $approverKeyPath -Raw)".Trim() + if ($approverValue -notmatch '^-----BEGIN (RSA )?PRIVATE KEY-----') { throw "$approverKeyPath isn't a PEM private key." } + $approverInstallation = $installations | Where-Object app_slug -eq $approverSlug | Select-Object -First 1 + if (-not $approverInstallation) { throw "The NCode Review Approver app isn't installed on $Org. Install it from the app's settings page first." } +} + $repoInfos = foreach ($name in $Repo) { if ($name -notmatch '/') { $name = "$Org/$name" } if ($name.Split('/')[0] -ne $Org) { throw "$name isn't in $Org." } @@ -135,6 +161,15 @@ foreach ($info in $repoInfos) { } } + # The approver app. + if ($approverInstallation -and $approverInstallation.repository_selection -ne 'all') { + $approverInstalled = Invoke-Gh api "user/installations/$($approverInstallation.id)/repositories" --paginate --jq '.repositories[].full_name' + if ($approverInstalled -notcontains $full -and $PSCmdlet.ShouldProcess($full, 'Add the repository to the NCode Review Approver installation')) { + Invoke-Gh api -X PUT "user/installations/$($approverInstallation.id)/repositories/$($info.id)" --silent + "Added $full to the NCode Review Approver installation." + } + } + # The environment: only the default branch may deploy to it. if ($PSCmdlet.ShouldProcess("$full environment $environment", "Limit deployments to $defaultBranch")) { Invoke-Gh api -X PUT "repos/$full/environments/$environment" --silent ` @@ -151,6 +186,13 @@ foreach ($info in $repoInfos) { "${full}: environment $environment, only $defaultBranch can deploy." } + # The approver's key is an environment secret, never an organization secret: only default-branch jobs can read it. + if ($approverValue -and $PSCmdlet.ShouldProcess("$full environment $environment", "Set the secret $approverKeySecret")) { + $approverValue | gh secret set $approverKeySecret --env $environment --repo $full + if ($LASTEXITCODE) { throw "Couldn't set ${approverKeySecret} for ${full}: gh secret set exited with code $LASTEXITCODE." } + "${full}: set the code-review environment secret $approverKeySecret." + } + # Environment secrets would hide the organization secrets. $shadowing = @(gh secret list --env $environment --repo $full --json name --jq '.[].name' 2>$null) | Where-Object { $_ -in $appKeySecret, $bedrockSecret } @@ -171,7 +213,7 @@ foreach ($name in $appKeySecret, $bedrockSecret) { $value = $null if ($name -eq $appKeySecret -and $AppPrivateKeyPath) { $keyPath = (Resolve-Path -LiteralPath $AppPrivateKeyPath).Path - $value = (Get-Content -LiteralPath $keyPath -Raw).Trim() + $value = "$(Get-Content -LiteralPath $keyPath -Raw)".Trim() if ($value -notmatch '^-----BEGIN (RSA )?PRIVATE KEY-----') { throw "$keyPath isn't a PEM private key." } } elseif ($name -eq $appKeySecret -and -not $current) {