ci: publish releases through npm trusted publishing

[feng-shiplight]

Summary

Two commits: the release bump that has been sitting on this branch, and the workflow that can actually ship it.

chore: release — quality-core 0.3.1, quality-tools 0.3.3, quality-ui 0.1.1, each a patch off the version published on npm. quality-map stays at 0.2.0. approvedIncrease.version in packages/quality-tools/package-size.json moves with the bump, or the gate rejects the recorded approval.

ci: publish packages through npm trusted publishing — publishing here was manual, needing either a long-lived npm token or an interactive 2FA prompt in a maintainer's terminal. npm is retiring the former. The new .github/workflows/publish.yml is dispatched by hand from the Actions tab against main and authenticates by OIDC: this repository holds no npm credential, and pnpm@11.1.2 trades the job's GitHub id-token for a short-lived npm one at publish time. That is why the job declares id-token: write and asserts the token endpoint exists before spending time on the gates — the alternative is a bare ENEEDAUTH after the whole suite has run.

It does not decide versions, unlike the equivalent workflows in shiplight-cli, which compute published + 1 and push the bump themselves. packages/quality-tools/package-size.json pins a version-specific approvedIncrease that only a human maintainer may write; a workflow that bumped the version on its own would move that approval away from the number a human approved, on every run. So this one publishes exactly what package.json says on main, and skips any package already at that version on npm. Dropping the auto-bump also drops the GitHub App token, so the release path introduces no new secret.

Packages publish in dependency order — quality-map, quality-core, then the two that consume it. pnpm rewrites the workspace: ranges while packing, so publishing a consumer first would leave a tarball on npm that nobody can install until the next step finishes.

Test plan

• Workflow YAML parses
• The publish loop, run against the live registry, skips quality-map@0.2.0 and selects exactly quality-core@0.3.1, quality-tools@0.3.3, quality-ui@0.1.1
• pnpm@11.1.2 — the version packageManager pins — really does the OIDC exchange: releasing/commands/lib/publish/oidc/idToken.js requests a token with audience npm:registry.npmjs.org. The homebrew pnpm 10.27 on the maintainer's machine has no such path, which is why the manual attempt hit 2FA
• CI green on this PR
• After merge: one dry_run: true run to prove the credential path before a real release

Required before the workflow can run

1. A trusted publisher on npmjs.com for each package — organization ShiplightAI, repository quality, workflow filename publish.yml, environment blank. Without it npm rejects the OIDC token even from a correct workflow. The maintainer is configuring this.
2. This PR merged. workflow_dispatch only appears in the Actions tab once the file is on the default branch, so the workflow cannot cut its own first release from a branch.

Renaming publish.yml later breaks publishing until the npm side is updated to match; AGENTS.md now says so.

🤖 Generated with Claude Code

https://claude.ai/code/session_01NhxpEed9pbcjT7MLtnDbiH

[claude]

Review: ci: publish releases through npm trusted publishing

This PR adds a well-reasoned OIDC-based publish workflow, updates AGENTS.md with clear version-bump and publishing guidance, and bumps three package versions. The overall design is correct — no NPM_TOKEN, no auto-bump, no scope creep into version decisions. One HIGH issue blocks merge; one MEDIUM issue should be addressed.

---

CRITICAL — None

---

HIGH

1. Dead documentation reference in the workflow comment

.github/workflows/publish.yml:7 says:

# docs/how-to/publish-a-release.md.

That file does not exist in the repo and was not added in this PR. A maintainer following the comment to find step-by-step instructions gets a 404. Per the project's own review criteria, docs must match reality; a broken cross-reference in a release-critical workflow file is a real defect, not a nit.

Fix: either create docs/how-to/publish-a-release.md with the release procedure, or replace the reference with the AGENTS.md section that was actually added (## Publishing in AGENTS.md).

---

MEDIUM

2. Publish workflow uses tag-pinned actions, not SHA-pinned

publish.yml uses actions/checkout@v7, pnpm/action-setup@v6, and actions/setup-node@v7 without commit-SHA pinning. ci.yml does the same, but the publish job carries id-token: write — the OIDC credential that can push packages to npm. A tag can be force-moved by a compromised maintainer at the upstream action's org; SHA-pinning (actions/checkout@<sha>) makes the workflow immutable to that vector. The npm trusted publisher configuration is the primary guard, but defence-in-depth suggests pinning here.

This matches the existing ci.yml pattern, so it is not a regression introduced by this PR, but the stakes are higher for a workflow that can publish.

---

LOW

3. Step summary shows empty output when publish step fails early

.github/workflows/publish.yml:139-145:

- name: Step summary
  if: always()
  run: |
    ...
    - Published: `${{ steps.publish.outputs.published }}`
    - Already on npm, skipped: `${{ steps.publish.outputs.skipped }}`

if: always() means this step fires even if the publish step exits before writing to $GITHUB_OUTPUT. The summary then shows literal empty backtick spans (\`). This is cosmetically broken and can make a failed run harder to diagnose. A guard like if: always() && steps.publish.conclusion != 'skipped'plus a fallback string (e.g.${{ steps.publish.outputs.published || '(none)' }}`) would improve legibility.

4. prepublishOnly re-runs tests and size-gate inside pnpm publish

quality-tools/package.json has prepublishOnly: pnpm -w test && pnpm build && pnpm check:size and quality-ui/package.json has prepublishOnly: pnpm -w test && .... When the publish loop calls pnpm publish, these run again — after the explicit Test, Build, and check:size gates already passed. The redundancy is intentional belt-and-suspenders; just note it adds several minutes to each run.

---

What looks correct

• Dependency publish order (quality-map → core → quality-tools → ui) matches the workspace graph and ensures workspace: ranges are resolved before consumers land on npm.
• Skip-not-fail logic for already-published versions enables safe re-runs after a mid-release failure.
• ref: main checkout in the workflow ensures every dispatch publishes the reviewed commit, not an arbitrary branch.
• The OIDC pre-flight check (ACTIONS_ID_TOKEN_REQUEST_URL) surfaces the commonest misconfiguration early rather than after a full test run.
• approvedIncrease.version moves from 0.3.2 → 0.3.3 in lockstep with quality-tools/package.json, satisfying the size-gate contract.
• approvedBy: "Feng Qian" was written by a human author (feng-shiplight), not a bot — consistent with the AGENTS.md rule that agents must not claim this approval.
• No schema_version changes; no score-computation paths touched; no agent-skill guidance altered.

[claude]

Review: ci: publish releases through npm trusted publishing

This PR ships two things: a chore: release version bump and the .github/workflows/publish.yml that makes future releases self-service. Both are well-considered. No CRITICAL, HIGH, or MEDIUM issues found.

---

What the PR does

• Adds publish.yml: a manually dispatched workflow that authenticates to npm via GitHub OIDC (no stored token), runs the existing CI gates, then publishes packages in dependency order, skipping any version already on npm.
• Adds "Version bumps" and "Publishing" sections to AGENTS.md (mirroring what is already in CLAUDE.md).
• Bumps patch versions: quality-core 0.3.0 → 0.3.1, quality-tools 0.3.2 → 0.3.3, quality-ui 0.1.0 → 0.1.1.
• Updates package-size.json to version: "0.3.3" / packedBytes: 46291 (1-byte drift from the version string change in the packed package.json).

---

Security

The workflow is correctly constructed for a credential-holding job:

• SHA-pinned actions (publish.yml lines 42, 45, 47) rather than the floating tags used in ci.yml. The PR explains why: "a tag can be force-moved at the upstream org, and this is the only job in the repository whose credential can push code to every consumer." Correct call.
• id-token: write scoped to the job, with contents: read at the top level. No NPM_TOKEN anywhere in the repo.
• ref: main checkout ensures the workflow always publishes a reviewed commit, never a branch head — the safest possible checkout for a publishing job.
• Preflight check (Assert the OIDC id-token is available) catches the commonest misconfiguration before the full gate suite runs.
• Shell hygiene: set -euo pipefail throughout the publish script. $dir values are hardcoded in the for-loop so there is no shell injection path from package.json fields.
• Step summary values: sha is a hex git SHA; dry_run is a boolean; published / skipped come from hardcoded $dir entries. No untrusted data flows into GitHub expression context.

---

Correctness

• Publish order (quality-map → quality-core → quality-tools → quality-ui) matches the dependency graph. pnpm rewrites workspace: ranges during pack, so a consumer published before its dep would be broken until the dep lands. The order is correct.
• Skip-if-already-published (npm view "$name@$version" version) lets a re-run after a mid-release failure resume cleanly. Correct design.
• All-skipped guard: if [ ${#published[@]} -eq 0 ] → exit 1 with a GitHub error annotation catches the "forgot to land the release commit" error before it looks like a silent success.
• --no-git-checks rationale is sound: actions/checkout leaves a detached HEAD that pnpm's branch safety check rejects. Compensated by ref: main.
• Version bookkeeping: approvedIncrease.version in package-size.json moves to 0.3.3 alongside package.json. The gate would reject a mismatch; it's correct here.

---

LOW findings

[LOW] No concurrency group on the publish workflow (publish.yml, global scope missing).
If two maintainers trigger workflow_dispatch simultaneously, both jobs can read the same "not yet on npm" state before either publishes and then race to publish. One will fail with 403 You cannot publish over the previously published versions. The skip logic is a TOCTOU mitigation, not a lock. Adding concurrency: group: publish-npm (no cancel-in-progress, since cancellation mid-publish would be worse than the race) would prevent the double-trigger.

[LOW] No GitHub environment gate (publish.yml, jobs.publish).
Adding environment: npm and configuring protection rules on npmjs.com would enforce a required-reviewer approval before any credential is used. Without it, any org member with workflow_dispatch access can publish. This is a design choice (the PR doc says "dispatched by hand from the Actions tab"), but an environment gate makes the human-in-the-loop mandatory rather than conventional.

---

Invariant check

Invariant	Status
Independence — engine computes scores, no agent writes .quality/	✅ Not touched
Human-gated fields — approvedIncrease authored by "Feng Qian", not an agent	✅ Correct
Determinism — no Date.now(), Math.random(), or order-sensitive paths	✅ Clear
Dependency direction — publish.yml is CI infra, not a package dep	✅ Clear
Explorer read-only — not involved	✅ N/A
Saved-artifact compatibility — no schema version touched	✅ Clear
Published surface — patch bumps only, no API removed or renamed	✅ Clear
Agent skills — not modified	✅ N/A

---

Summary

The publish workflow is well-structured, correctly scoped, and hardened where it matters (SHA pins, no stored token, ref: main, gates before publish). The two LOW findings are hardening suggestions, not correctness issues. The version bumps and package-size.json update are internally consistent. Approving.

[claude]

Review: ci: publish releases through npm trusted publishing

Overview

Two commits: version bumps for quality-core (0.3.0→0.3.1), quality-tools (0.3.2→0.3.3), and quality-ui (0.1.0→0.1.1), plus a new .github/workflows/publish.yml that replaces manual npm publishing with OIDC trusted publishing. The design is solid: SHA-pinned actions, hard-coded ref: main, no long-lived npm token, concurrency.group prevents simultaneous publish races, and the workflow correctly refuses to bump versions on its own (preserving the human-approved approvedIncrease invariant documented in CLAUDE.md).

---

Findings

MEDIUM — Missing --provenance flag

.github/workflows/publish.yml, lines 132 and 135:

(cd "$dir" && pnpm publish --dry-run --no-git-checks --access public)
(cd "$dir" && pnpm publish --no-git-checks --access public)

This workflow already holds id-token: write and uses OIDC trusted publishing — exactly the prerequisites npm requires to attach a SLSA provenance attestation. Adding --provenance gives every consumer a verifiable, cryptographically-signed chain from commit → workflow run → published tarball; without it, a future account compromise could publish a look-alike tarball with no audit trail distinguishing it from a legitimate release.

For a quality oracle whose credibility depends on artifacts nobody can fake, supply-chain integrity of the tooling itself is load-bearing. pnpm 11.x supports the flag natively; it is a one-line addition.

Fix:

pnpm publish --dry-run --no-git-checks --access public --provenance
pnpm publish --no-git-checks --access public --provenance

---

LOW — Dry run fails when all versions are current

.github/workflows/publish.yml, lines 140–143:

if [ ${#published[@]} -eq 0 ]; then
  printf '::%s::%s\n' error "Every package version on main is already on npm…"
  exit 1
fi

The PR description calls for a dry_run: true run after merge "to prove the credential path." This works if the release versions are not yet on npm. But if a maintainer ever wants to re-verify the OIDC credential without landing a release commit first, this gate blocks them. Consider skipping or softening the "nothing to publish" error when DRY_RUN=true.

---

LOW — No GitHub Actions environment on the release job

The release job holds id-token: write but is not assigned to a named GitHub environment (e.g., environment: release). Without environment protection rules, any org member who can dispatch workflows can publish. The npm trusted publisher config intentionally says "environment blank," so npm won't enforce it; but a GitHub-side required-reviewers rule on a release environment would add one more ratification gate before the OIDC token is exchanged. Worth revisiting as the team grows.

---

LOW — Inline ${{ }} expressions in the step-summary shell step

.github/workflows/publish.yml, lines 154–158: ${{ steps.publish.outputs.published }} is expanded server-side and substituted verbatim into the bash script. Package names from the @shiplightai scope cannot contain shell metacharacters, so this is benign today. If the loop were extended to third-party package directories, an attacker-controlled package name could escape the echo. An env-var read eliminates the class:

env:
  PUBLISHED: ${{ steps.publish.outputs.published }}
run: echo "- Published: \`$PUBLISHED\`" >> "$GITHUB_STEP_SUMMARY"

---

What looks correct

• Version bumps follow CLAUDE.md exactly: patch-only off the published base, approvedIncrease.version tracks the new quality-tools version, quality-map is correctly unchanged, and approvedBy is attributed to the human committer — not an agent.
• SHA pins on all actions in the publish job — the rationale is sound; this is the only job whose credential reaches every consumer.
• ref: main hard-coded in checkout — the workflow can only be dispatched once the file is on the default branch, and always publishes main regardless of dispatch origin.
• Dependency-ordered publish loop — quality-map → core → quality-tools → ui ensures workspace: ranges are resolved before consumers reach npm.
• Idempotent re-run — the npm view skip-if-published check lets a mid-release retry finish safely.
• OIDC preflight assertion — catches misconfigured id-token before spending time on build/test/size gates.
• concurrency.group: publish-npm without cancel-in-progress — correct; aborting a publish mid-flight is worse than queueing.
• AGENTS.md additions match the workflow design precisely.

[feng-shiplight]

On the MEDIUM (--provenance): declining, with evidence. pnpm's OIDC path already attaches a provenance attestation on its own, and the flag would only make failures worse.

In pnpm@11.1.2 — the version packageManager pins — the publish path consults options.provenance only if a flag was passed; otherwise it calls determineProvenance, which reads the id-token's repository_visibility, asks the registry whether the package is public, and returns true when both are. This repository is public and both packages publish with access: public, so the attestation is signed either way.

What the flag does change is the failure mode. Without it, a ProvenanceError is caught and downgraded to Skipped setting provenance and the release still goes out. With it, the request reaches libnpmpublish, which throws Can't generate provenance for new or private package rather than warning — so an explicit flag converts a degraded release into a failed one, and buys no attestation we do not already get.

That reasoning is now recorded in the workflow next to the publish call, so the flag does not get added later.

Took both other LOWs: a dry run no longer fails when every version is already on npm (it exists to prove the credential path, which is unrelated to whether a release commit has landed), and the step summary now reads its values from the environment rather than having them substituted into the script text.

Not taking the environment gate. It is a policy decision about who may cut a release rather than a defect, and adding environment: without configuring protection rules in repo settings does nothing — GitHub auto-creates the environment with no rules. If it is wanted, the npm trusted publisher has to name the same environment, and that is a maintainer decision.

[claude]

Review summary

This PR introduces an OIDC-based npm trusted-publishing workflow and patches three package versions (quality-core 0.3.1, quality-tools 0.3.3, quality-ui 0.1.1). The design is sound and the security posture is strong. No CRITICAL, HIGH, or MEDIUM issues found.

---

Findings

LOW — cancel-in-progress not explicitly set

.github/workflows/publish.yml lines 27-29:

concurrency:
  group: publish-npm

The comment (line 24) correctly explains why cancel-in-progress must be false: interrupting a mid-flight publish is worse than queueing a second dispatch. The default happens to be false, so the behavior is correct. But the intent is security-load-bearing enough that it should be stated explicitly: cancel-in-progress: false. This mirrors how ci.yml makes cancel-in-progress: true explicit.

---

LOW — npm view cannot distinguish "not found" from a transient network failure

.github/workflows/publish.yml lines 99-104:

if npm view "$name@$version" version > /dev/null 2>&1; then
  skipped+=("$name@$version")
  continue
fi

A network hiccup causes npm view to exit non-zero, the skip is skipped, and the script attempts pnpm publish. Because npm does not allow re-publishing a version, pnpm publish will fail with 403, set -euo pipefail aborts the run, and the workflow turns red — so the failure mode is safe and visible. But it could confuse a maintainer debugging a network-flaky runner. A --json flag and an explicit exit-code check for "code":404 / "code":"E404" would make the distinction explicit, though this is strictly a DX nit.

---

LOW — No GitHub environment gate on the publish job

The job has id-token: write, which lets it publish to every consumer of these packages. GitHub Environments with required-reviewers would add a second human approval before the credential is exercised, defence-in-depth for a workflow that only needs to run a few times a year. Not a requirement, but worth considering as the package surface grows.

---

LOW — SHA annotation comments cannot be verified at review time

.github/workflows/publish.yml lines 45, 47, 50:

actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6.0.10
actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0

SHA-pinning is the right call here (unlike ci.yml, which uses mutable tags). The annotation comments are informational; the SHAs are the actual security boundary. It is worth confirming the SHAs resolve to the claimed tags before merge, as a drifted comment is a future footgun.

---

What is clearly right

• OIDC only — no NPM_TOKEN in repository secrets; the job mints a short-lived credential via the OIDC exchange. This is the correct threat model.
• ref: main hard-coded — even a workflow_dispatch triggered via the API against another branch will always publish from main. The version the workflow publishes is always the human-reviewed release commit, not something an attacker could wedge in via a branch.
• All actions SHA-pinned — the only job in this repo whose credential can affect every downstream consumer is also the only job where every action is pinned to a commit hash. This is the right risk-proportionate decision.
• Pre-flight OIDC check — the job fails fast with a clear message if id-token: write is missing, rather than letting a bare ENEEDAUTH surface after the gates have run.
• Gates re-run on main — build, typecheck, test, and size checks run again at publish time, not just at PR merge time. This is the right design: main can have moved since the last green CI run.
• Publish order — quality-map → core → quality-tools → ui matches the dependency graph. pnpm rewrites workspace: ranges during pack, so out-of-order publishing would put an uninstallable tarball on npm.
• Skip-not-fail for already-published versions — safe for re-runs after a mid-release failure.
• approvedIncrease.version moves with the bump — package-size.json is updated to 0.3.3 in the same commit. The gate will accept it.
• AGENTS.md / CLAUDE.md in sync — the version-bump and publishing rules are now documented in both agent-facing files and match actual workflow behaviour.
• No workflow-computed version bump — the workflow cannot invalidate the human-authored approvedIncrease by bumping the version itself. That constraint is correctly encoded in both the workflow and the documentation.

---

All four findings are LOW. The workflow is conservative by design, publishes from a fixed branch, carries no stored secrets, and cannot touch scoring, evidence, or quality-map schema — the invariants that matter most to this project.