Skip to content

feat(release-notes): agent-generated release notes action - #4

Merged
feng-shiplight merged 2 commits into
mainfrom
feat/release-notes-action
Jul 28, 2026
Merged

feng-shiplight merged 2 commits into
mainfrom
feat/release-notes-action

Conversation

@feng-shiplight

Copy link
Copy Markdown
Contributor

Summary

A composite action that writes user-facing release notes for a commit range using a headless agent, returning them as a file for gh release create --notes-file. Consumer repos keep a short step plus their own notes-focus, mirroring how claude-review is wired.

Motivation: gh release create --generate-notes produces a list of merged PR titles, and a script like monots' gen-changelog.mjs can only reformat commit subjects. Neither can say what a release means for someone using the software. That is a writing task.

Design

The shape follows ci-triage/scripts/run-triage-agent.sh, the proven in-org pattern for running an agent non-interactively in CI:

  • Model fallback chain, best first — each Claude model, then each Codex model, both overridable. A model the CI token cannot access fails fast and the next is tried, so a missing entitlement degrades instead of dead-ending.
  • Success gated on a non-empty artifact, not an exit code, which also covers service-unavailable and unanticipated errors. A too-short result counts as no result, so a model refusal cannot become the release notes.
  • Never fails the job. Notes are not a gate — a caller that gets nothing falls back to --generate-notes rather than blocking a release on a model outage. generated is exposed as an output so the caller can branch.

Two deliberate departures from run-triage-agent.sh, both because this task is write a paragraph rather than investigate and fix:

  • Output is captured from stdout rather than written by the agent, so the agent needs no tools at all. Commit messages are author-controlled input, and this runs in a release job holding the caller's token; an agent with no write access cannot be steered by instructions embedded in a commit message. The prompt carries the whole range inline and instructs it not to use tools.
  • No actions/checkout. This runs late in a release job, after the artifact is built and packaged — a fresh checkout would wipe those untracked outputs. The caller's checkout is reused and only deepened, since release jobs check out at depth 1 and git log from..to needs history.

Verification

Run end to end against a real range — screen-recorder v0.1.0..main, 10 commits, a 16.5 kB prompt — with claude --print:

  • Grouped changes under themes a user would recognise (plan limits, recording reliability, sign-in and upload)
  • Described behaviour rather than files
  • Folded CI and refactor work into a single closing line instead of padding
  • Invented nothing — every claim checks out against the commits, including one whose effect is limited to unpacked development installs, which it correctly qualified

Also: shellcheck clean on the runner, and both files parse.

Notes for reviewers

  • v1 is not moved by this PR. The first consumer (screen-recorder) will pin to this PR's merge commit SHA, so the five repos on claude-review@v1 are untouched. Folding this into v1 later is a separate decision.
  • The Codex leg installs only when openai_api_key is supplied, so repos without it simply run the Claude chain.
  • Model defaults are pinned in the runner for the same reason ci-triage pins them: a runner otherwise inherits a model the CI token may not be entitled to.

feng-shiplight and others added 2 commits July 28, 2026 16:06
Adds a composite action that writes user-facing release notes for a commit
range with a headless agent, for `gh release create --notes-file`. Consumer
repos keep a short step plus their own `notes-focus`, mirroring how
claude-review is wired.

Shape follows ci-triage's scripts/run-triage-agent.sh, which is the proven
in-org pattern for running an agent non-interactively in CI:

- A model fallback chain, best first: each Claude model, then each Codex model,
  overridable via inputs. A model the CI token cannot access fails fast and the
  next is tried, so a missing entitlement degrades rather than dead-ends.
- Success is gated on a non-empty artifact, not an exit code, which also covers
  service-unavailable and unanticipated errors. A too-short result is treated as
  no result, so a refusal does not become the release notes.
- The script never fails the job. Notes are not a gate: a caller that gets
  nothing is expected to fall back to `--generate-notes` rather than block a
  release on a model outage. `generated` is returned as an output so the caller
  can branch.

Two deliberate differences from run-triage-agent.sh, both because this task is
write-a-paragraph rather than investigate-and-fix:

- Output is captured from stdout instead of the agent writing a file, so the
  agent needs no tools at all. Commit messages are author-controlled input and
  this runs in a release job holding the caller's token; an agent with no write
  access cannot be steered by instructions embedded in a commit message. The
  prompt therefore carries the whole range inline and says not to use tools.
- No actions/checkout. This runs late in a release job, after the artifact is
  built and packaged, and a fresh checkout would wipe those untracked outputs.
  The caller's checkout is reused and only deepened, since release jobs check
  out at depth 1 and `git log from..to` needs history.

Verified end to end against a real range (screen-recorder v0.1.0..main, 10
commits, 16.5 kB prompt): `claude --print` returned usable notes that grouped
changes by theme, described behaviour rather than files, folded CI and refactor
work into a closing line, and invented nothing — each claim checks out against
the commits, including one whose effect is limited to unpacked installs.
shellcheck is clean and both files parse.
Install .github/workflows/claude-code-review.yml so internal-tools gets the
same severity-based PR review it ships to consumer repos.

Unlike consumers (which pin ShiplightAI/internal-tools/claude-review@v1), this
caller uses the local ./claude-review, so a PR that edits the action is
reviewed by its own version instead of the older tagged copy. That needs an
explicit actions/checkout first, since a local action must be on disk before
`uses:` can resolve it.

The review-focus targets this repo's actual risk surface: SHA-pinning and
prompt injection in the composite action, the caller/action split invariant,
permissions creep, and agent-skills/** prompts that execute verbatim in other
repos.

Also fix the copy-in template, which pointed at the nonexistent
ShiplightAI/internal-agent-skills org path; the repo is ShiplightAI/
internal-tools, and shipyard's copy had already been corrected by hand.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@feng-shiplight
feng-shiplight merged commit 4077519 into main Jul 28, 2026
1 check passed
@feng-shiplight
feng-shiplight deleted the feat/release-notes-action branch September 25, 2026 15:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant