Skip to content

ci(#17): code coverage measurement + ratchet gate (baseline 57.31%) - #64

Merged
Cadacious merged 2 commits into
mainfrom
ci/17-coverage-gate
Jun 26, 2026
Merged

Cadacious merged 2 commits into
mainfrom
ci/17-coverage-gate

Conversation

@Cadacious

@Cadacious Cadacious commented Jun 25, 2026 •

Copy link
Copy Markdown
Contributor

Implements #17 — code coverage measurement + a hardened ratchet gate in build.ps1.

Baseline

57.31% overall (1199 tests), measured from the real suite via Pester JaCoCo — up from 33.2%. coverage-baseline.json holds overall + a 16-module per-module map.

How it works (Gate 3, opt-in -Coverage / -CodeCoverage)

  • Overall ratchet: fails if measured overall drops more than a 0.30pt tolerance below baseline (~15x the measured ~0.02pt jitter).
  • Per-module floors (enforced, not decorative): every module in the baseline is checked on every run; a module dropping >0.30pt below its own baseline fails and is named — even if overall stays within tolerance (so one module's gain can't mask another's collapse).
  • Anti-tamper absolute floor (50%): a measured overall or a committed baseline.overall below 50% fails as gutted/tampered, so the baseline can't be edited down to defeat the gate.
  • Desync guards: a new module absent from the baseline is WARNed (with the value to add); a baseline module that vanished fails with guidance.
  • On a >1pt rise, prints a ratchet: baseline can be raised hint (never auto-raises). Reuses the single Invoke-Pester run (+~14s for instrumentation). CI uploads coverage.xml; CONTRIBUTING documents the per-module raise process.

Proof

Per-module gate: raised VSSManager's baseline → build failed naming it while overall passed. Absolute floor: baseline.overall=5 → failed "implausible/tampered". Plain ./build.ps1 stays BUILD OK with the gate skipped.

Closes #17.

🤖 Generated with Claude Code

…ine + CI artifact

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@Cadacious
Cadacious requested a review from Copilot June 25, 2026 23:35
@coderabbitai

coderabbitai Bot commented Jun 25, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

@Cadacious, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 9 minutes. Learn how PR review limits work.

Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.

🚦 How do rate limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 9fec46c6-09c9-4719-aa90-3d7363821b1c

📥 Commits

Reviewing files that changed from the base of the PR and between 73ebdb8 and 67eb241.

📒 Files selected for processing (5)
  • .github/CONTRIBUTING.md
  • .github/workflows/ci.yml
  • .gitignore
  • build.ps1
  • coverage-baseline.json
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch ci/17-coverage-gate

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds opt-in code coverage measurement and a “ratchet” gate to the PowerShell build entrypoint so CI enforces that overall coverage does not regress beyond a small tolerance, while also publishing the JaCoCo coverage report as a CI artifact.

Changes:

  • Introduces a committed coverage-baseline.json (overall + per-module baseline map) used by the coverage ratchet gate.
  • Extends build.ps1 with a -Coverage switch to enable Pester JaCoCo collection and enforce the baseline/tolerance gate.
  • Updates CI to run coverage mode and upload coverage.xml, and documents the workflow in CONTRIBUTING.

Reviewed changes

Copilot reviewed 4 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
coverage-baseline.json Adds the committed baseline (overall + per-module) used by the coverage ratchet gate.
build.ps1 Adds opt-in Pester JaCoCo coverage collection and a baseline/tolerance “ratchet” failure gate.
.gitignore Ensures coverage outputs stay ignored while explicitly tracking coverage-baseline.json.
.github/workflows/ci.yml Runs build.ps1 with coverage enabled and uploads coverage.xml as an artifact.
.github/CONTRIBUTING.md Documents how the coverage gate works and how to raise the baseline deliberately.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread build.ps1
Comment on lines 45 to +48
param(
[switch]$InstallDeps,
[string[]]$ExcludeTag = @('E2E', 'Integration')
[string[]]$ExcludeTag = @('E2E', 'Integration'),
[switch]$Coverage
… tolerance; -CodeCoverage alias; doc fixes (8-lens)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 5 changed files in this pull request and generated 1 comment.

Comment thread build.ps1
# per-module floor. Measured run-to-run jitter is ~0.02pt, so 0.3pt is ~15x the noise -- wide
# enough to ride out that jitter, tight enough that real erosion (a function/test removed) trips
# it promptly. Named constant: change it here, in one place, for both gates.
$coverageTolerance = 0.30
@Cadacious
Cadacious merged commit acfa8ff into main Jun 26, 2026
3 checks passed
@Cadacious
Cadacious deleted the ci/17-coverage-gate branch June 26, 2026 00:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Code coverage measurement, gate, and ratchet

2 participants