Skip to content

Make NightVision findings read well in GitHub Code Scanning - #8

Merged
ryankinkead merged 4 commits into
mainfrom
improve-code-scanning-reporting
Sep 23, 2026
Merged

ryankinkead merged 4 commits into
mainfrom
improve-code-scanning-reporting

Conversation

@ryankinkead

Copy link
Copy Markdown
Owner

Why

Code Scanning made the demo look worse than the scan. On the June run every rule ID embedded a per-scan UUID, so each run closed all alerts and opened new ones (alert numbers are at 756 for 26 findings). The current CLI fixes the IDs, but GitHub still renders the export poorly:

  • rule help panels say "see the individual result"
  • result messages are Markdown, which GitHub shows as raw ** and emoji
  • everything is warning/note, so SQL injection looks like a missing header
  • untraced findings point at a file named /
  • SQL injection and Spring4Shell come through with no description at all

What changed

  • scripts/nightvision-sarif-enrich.py (stdlib Python): plain-text messages (SQL Injection - PostgreSQL on POST /search. + proof link), rule help and remediation, CWE tags, error for critical/high, rule-level security-severity and precision, a real file for app-wide findings, and dropping of informational and duplicate results. It also writes a severity table to the job summary. Schema-validated against SARIF 2.1.0; on the June scan it takes 35 raw results to 19 alerts (4 critical, 2 high, 13 low).
  • nightvision.yml: scans PRs too, waits for the app to respond instead of sleep 10, adds a manual scan-duration input, reads the scan ID from stdout only, still exports findings when a scan ends non-zero, excludes findings triaged as false positive/won't fix in NightVision, uploads artifacts, and bumps actions to Node 24 majors. The job id stays test, so the 26 stale UUID-rule alerts close on the first run.
  • Slack / Teams / email workflows: these were broken (retired upload-sarif@v2, docker-compose v1, removed -a flag, --lang spring, and invalid Slack YAML that meant it never ran). Fixed, Slack made manual-only so pushes don't start a second scan, and all four share one Code Scanning category.
  • Removed the never-run improved workflow file (no .yml extension), and wrote a real README.

Testing

  • Enricher run against the June scan's export from the current CLI, and the output validated against the SARIF 2.1.0 schema with 0 errors
  • actionlint is clean on nightvision.yml; all workflow YAML parses
  • This PR's own run is the end-to-end check: live scan, upload, and PR annotations

🤖 Generated with Claude Code

ryankinkead and others added 4 commits September 23, 2026 11:40
The export's rule help is boilerplate, result messages are Markdown that
GitHub shows as raw text, every finding is warning or note, and untraced
findings point at a file named "/". Add a stdlib-only enrichment step that
fixes all four, adds CWE tags and a run summary, and drops informational
and duplicate results.

Workflow: add PR scans, a readiness check instead of sleep 10, a manual
scan-duration input, scan-ID capture from stdout only, export of findings
from failed scans, and exclusion of findings triaged in NightVision. Bump
actions to Node 24 majors. The job id stays "test" so the stale alerts
from the old per-scan rule IDs close on the next run.

Repair the Slack, Teams and email workflows (retired upload-sarif v2,
docker-compose v1, removed -a flag, --lang spring, invalid Slack YAML),
make Slack manual-only, and point all four at one Code Scanning category.
Remove the never-run "improved workflow" file and write a real README.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The CI target is https://localhost:9000, but the July demo-baseline
commit turned SSL off and renamed the keystore password to one the
keystore does not use, so every scan since then fails the connectivity
check. Add docker-compose.ci.yml, which turns SSL on with the keystore's
real password for CI only; local demos keep plain HTTP.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A scan that dies partway (a relay timeout did this on the last PR run)
exports a subset of findings, and uploading it marks everything it
missed as fixed. Keep partial findings in the artifacts instead.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@ryankinkead
ryankinkead merged commit 9a331f3 into main Sep 23, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant