Skip to content

feat(digest): surface standards-check warnings in the digest - #182

Merged
twistedmelonman merged 1 commit into
mainfrom
claude/feat-digest-standards-warnings-179
Oct 2, 2026
Merged

twistedmelonman merged 1 commit into
mainfrom
claude/feat-digest-standards-warnings-179

Conversation

@twistedmelonman

Copy link
Copy Markdown
Member

Summary

Adds a Standards warnings section to the daily Dependabot digest issue. For every repository each owner's app installation can read, the new scripts/dependabot-digest/standards.sh finds the latest standards-check run that vetted the default branch and lists its warning- and notice-level annotations (repo, file, message). Repos that could not be checked are listed by name with the reason, so "no warnings" can never stand in for "not checked".

Precondition from the issue: the first scheduled App-auth digest run (2026-10-02 18:49 UTC, run 37050060143) completed with success.

How the latest default-branch run is found

Every fleet caller triggers standards-check on pull_request only (standards/caller-stub.yml in github-workflows; confirmed on dotfiles, claude-config, dev-env, tensegrity). A squash merge writes a new commit on the default branch, and that commit has no run. So per repo:

  1. GET installation/repositories (--paginate --slurp) gives the repo list, default branch and archived flag. This is exactly the set the token can read.
  2. GET repos/{o}/{r}/commits/{default_branch} gives the head SHA.
  3. GET repos/{o}/{r}/commits/{head}/check-runs. If a run exists here (a push-triggered caller), use it.
  4. Otherwise GET repos/{o}/{r}/commits/{head}/pulls and select the PR whose merge_commit_sha == head and merged_at != null: the PR whose merge produced that commit.
  5. GET repos/{o}/{r}/commits/{pr.head.sha}/check-runs. The run is the one from app.slug == github-actions whose name is run-standards-check or ends with / run-standards-check. Callers can name their job differently. If re-runs leave several runs, the highest id wins.
  6. GET repos/{o}/{r}/check-runs/{id}/annotations (--paginate --slurp). Keep warning and notice. Drop failure, because a failure already makes the check red.

This uses only the permissions the app already holds: Checks, Contents, Pull requests (plus Metadata). It does not need Actions. I verified steps 3 to 6 live on this repo with the shell token, because dev-env is public: commits/9bc08fb/pulls returns #181 with merge_commit_sha equal to the main head. The check run name is standards-check / run-standards-check, and its 4 notice annotations came back. Budget: about 5 calls per repo × about 41 repos, well within installation rate limits.

Scope caveat, stated in the section itself: a PR run lints only the files that PR changed. Node-floor always checks the whole repo. So "no warnings" means the run was clean, not that the whole repo is clean.

Per-repo states and how each renders

State Meaning Renders as
ok run read, annotation count verified rows in the warnings table, or counted as clean
no-run no standards-check run on the head or on the producing PR's head (e.g. a repo with no caller) "Not checked" row: no standards-check run: …
no-pr head has no run and no merged PR produced it (direct push) "Not checked" row: no run, no merged PR: …
in-progress latest run not completed; annotations would be partial "Not checked" row: run in progress: …
unreadable gh error (stderr reason kept), check-run list nulled or short (total_count > entries, or an entry with no id/name), or fewer annotations than the run's output.annotations_count "Not checked" row: UNREADABLE: <reason>
archived archived repo named in one "Archived, not surveyed: …" line
unlisted (repo null) owner's installation repos cannot be listed, or standards.sh crashed "Not checked" row: all of <owner> | OWNER NOT LISTED: <reason>

Section-level guards:

  • Zero repos checked: "No repository could be checked" is shown, never "No warnings".
  • Empty survey output: "Not checked." is shown.
  • Malformed survey output: the whole file is parsed first, because jq stops at the first bad line. The section then shows "Not checked. … output is malformed".
  • A jq render failure: each table is captured before it is printed. If capture fails, a loud line is printed instead of a header over an empty table.
  • No section without --standards: existing render callers are unchanged. run-digest.sh always passes --standards.

A standards failure never blocks the Dependabot queue report: it renders as "not checked". Collection failures still abort the digest exactly as before.

Sample (from tests/fixtures/standards)

## Standards warnings

Warning- and notice-level annotations from each repository's latest `standards-check` run that vetted its default branch. Fleet callers run only on pull requests, so that is usually the run on the head commit of the PR that produced the default-branch head. A PR run lints only the files that PR changed (node-floor always checks the whole repo), so "no warnings" means that run was clean, not that the whole repository is.

**3 repositories checked**: 2 with warnings or notices, 1 with none. **6 not checked.**

| Repo | Level | File | Message |
| --- | --- | --- | --- |
| [o/warned](https://github.com/x/runs/11) | warning | `.github/workflows/ci.yml:12` | node-version 18 is below the floor 22 |
| [o/warned](https://github.com/x/runs/11) | notice | — | no shell files |
| [o/warned](https://github.com/x/runs/11) | notice | `.github/workflows/build.yml:7` | node-version is an expression (${{ inputs.node }}); not checked |
| [p/pwarned](https://github.com/x/runs/70) | notice | — | The ubuntu-latest label will migrate |

### Not checked

These repositories have no readable standards-check result. Their warnings, if any, are unknown — not absent.

| Repo | Why |
| --- | --- |
| o/direct | no run, no merged PR: main head fffffff has no standards-check run and was not produced by a merged pull request |
| o/forbidden | UNREADABLE: cannot read the head of main: gh: Resource not accessible by integration (HTTP 403)  |
| o/norun | no standards-check run: no standards-check run on main head ccccccc or on the head of the PR that produced it |
| o/nulled | UNREADABLE: cannot read check runs on main head ddddddd: check-run list for ddddddd is incomplete or nulled |
| o/short | UNREADABLE: annotations are incomplete or malformed (run reports 5) |
| all of q | OWNER NOT LISTED: cannot list the repositories this token can read: gh: Resource not accessible by integration (HTTP 403)  |

Archived, not surveyed: o/old.

GitHub files annotations that name no file under the path .github, with the log line as the line number, so these render as —.

Expected noise (follows from the locked design)

  • Every repo with no standards-check caller appears under "Not checked" as no standards-check run. Most of these are twistedmelonman forks. The design forbids dropping them.
  • no shell files / no YAML files / no workflows notices from run-standards.sh, and runner notices such as the ubuntu-latest migration notice, appear on most runs. The design is to list every notice. Filtering any of them would be a design change.

Tests

  • New tests/test-standards.sh covers fixture-driven gh stubs. Cases: warnings across two owners and two levels, failure-level excluded, latest re-run chosen, push-triggered run on the head, no run, no merged PR, nulled check-run list, short annotation pages, HTTP 403 with its reason kept, an archived repo, an unlistable owner, an all-clean survey with explicit "No warnings", clean next to unreadable, all unreadable, empty output and malformed output.
  • tests/test-run-digest.sh asserts the section appears end to end, and that an owner whose listing fails renders as not checked.
  • Fail-before: against an origin/main copy, test-run-digest.sh fails its 2 new assertions and test-standards.sh fails 32.
  • bash scripts/dependabot-digest/tests/run-tests.sh passes all five suites. shellcheck -S info is clean on every script.

Not verified here

  • installation/repositories and annotations reads under the real App installation token. Local probes used the shell token on a public repo. The first scheduled run after merge will show this. A refused read renders loudly as not checked.
  • gh api --paginate --slurp on the ubuntu-latest runner needs gh ≥ 2.48. An older gh would fail the call, which renders as not checked rather than clean.
  • The | escaping in table cells is not exercised by a test.

No workflow or permission change. The runbook is unchanged: no stated fact changed, and the same four read permissions cover the new calls.

Closes #179

Add a "Standards warnings" section to the daily digest issue. For each
repository the owner's app installation can read, standards.sh finds the
latest standards-check run that vetted the default branch and lists its
warning- and notice-level annotations (repo, file, message).

Fleet callers trigger standards-check on pull_request only, so a squash
merge leaves the default-branch head with no run. standards.sh checks the
head commit first, then follows commits/{sha}/pulls to the merged PR that
produced it and reads the run on that PR's head commit. Only the Checks,
Contents and Pull requests permissions the app already holds are used.

Every repository yields one record. A refused read, a nulled check-run
list, an annotation count short of the run's own count, a repo with no
run, a head with no merged PR, and an unlistable owner each render by
name under "Not checked", so "no warnings" can never stand in for "could
not check". Standards failures do not block the queue report.

Closes #179
@twistedmelonman
twistedmelonman merged commit 5848c36 into main Oct 2, 2026
3 checks passed
@twistedmelonman
twistedmelonman deleted the claude/feat-digest-standards-warnings-179 branch October 2, 2026 19:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Surface standards-check warnings in the daily digest

1 participant