Skip to content

fix(repo): pass a mutate set that matches no file with an empty report - #115

Merged
ryanleecode merged 8 commits into
mainfrom
gh-114
Sep 25, 2026
Merged

ryanleecode merged 8 commits into
mainfrom
gh-114

Conversation

@systemfsoftware-maker

@systemfsoftware-maker systemfsoftware-maker commented Sep 25, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

A stryker run whose mutate patterns match no file now performs the dry run, writes a mutation report with files: {}, and exits 0 whatever thresholds.break says. Before, it stopped at the instrument stage with exit 3 and printed InstrumentError: with an empty message, which failed sharded CI jobs whose shard owned no file.

Fixes #114

What changed for users

  • Empty mutate set: dry-runs and reports like any other run, matching upstream StrykerJS. An empty run has no mutation score, so it no longer trips the break threshold.
  • Printed errors: prepare, dry-run, and mutation-test stage errors carry their reason as the message, so a printed cause no longer shows up as a bare error name.
  • stryker merge-reports: a package whose report has no tested mutant shows n/a instead of 0.00.
  • Breaking, @systemfsoftware/stryker-js-plugin-interface (major): Report.Metrics#mutationScore and #mutationScoreBasedOnCoveredCode return Report.MutationScore (Scored { percentage } or Unscored) instead of a number that was NaN for an empty run. The changeset shows the Report.MutationScore.match migration. The machine-mode verdict envelope keeps score: number | null, and mutation.json never carried metrics, so neither wire format changes.

Why it takes this shape

The exit-code failure was not in the instrument stage alone. Once an empty set was admitted, Metrics.mutationScore returned NaN, and ClassifyExitCommand's schema rejected it (exit 1). Returning 0 would trip every configured break threshold, so the score became a union that cannot hold an invalid number. Every consumer (exit classifier, clear-text renderer, merge-reports, verdict envelope) now matches on the tag, and classifyExit is the single place that decides the exit code.

Verification

  • CLI on a fixture with mutate: ["src/a.ts", "!src/a.ts"] and thresholds.break: 100: exit 0, reports/mutation/mutation.json has "files": {}. Before the fix: exit 3.
  • New integration scenarios in packages/stryker-js/tests/empty-file-selection.integration.test.ts: an empty shard passes with an empty report, and an empty project fails at prepare with No input files found. as the stated cause.
  • Suites: stryker-js 634/634, plugin-interface 156/156, html-reporter 4/4. Workspace pnpm typecheck 58/58. oxlint, tsgo, dprint, and api:check are clean.
  • Property laws were audited for vacuity. Each new law was run against a deliberately broken implementation (< changed to <=, fileCount forced to at least 1, an unscored run failing a break threshold, the merge scoring an empty module), and all four broken versions failed.

Review findings (all applied)

Finding Resolution
P1: merge-reports computed its score apart from Report.Metrics (merge-report-parts.workflow.ts:215) The make-body-purity gate forbids calling Report.Metrics from the workflow's decide body, so the dead clamp that made the two differ was removed (2d132aa), and ∀cs_Modules_≡RowScoreIsTheReportMetricsScore now requires every merged row's score to equal the Report.Metrics score (191f494). A deliberately drifted merge formula fails that law.
P1: consumers outside this repo that read mutationScore as a number break (Metrics.schema.ts:53) Intended breaking change, declared major; the changeset now shows the before/after migration (670998e).
P2: the score was rendered in two places (mutation-reporting.service.ts, clear-text renderer) Only a scored run can break a threshold, so the exit-code logs take a plain percentage and the service's own n/a renderer is gone (86d68c3).
P3: an unscored run with no break threshold logged "No breaking threshold configured" Unscored runs with no threshold log nothing (f7e37a0).

Review run 20260925-153149-c8aaef73: correctness, project-standards, testing, maintainability, api-contract, and an in-process adversarial lens; no cross-model peer was installed. The validator rejected three further findings with code evidence (the reviewers expected mutation.json or other packages to read the score; they do not).


Compound Engineering

…error reasons

An empty mutate set was refused at the instrument stage (exit 3), and an
empty run's NaN score was fed to the exit classifier (exit 1). Plan the
empty set like any other and treat a non-finite score as no score, so the
run writes an empty report and passes whatever thresholds.break says

Prepare, dry-run, and mutation-test stage errors now derive their message
from their reason instead of printing an empty one

Closes #114
…d of NaN

Metrics.mutationScore returned NaN when no mutant counted toward the score,
and every consumer had to remember to check for it. The score is now
Report.MutationScore, a tagged union of Scored { percentage } and Unscored,
so classify-exit, clear-text, merge-reports and the verdict envelope match
on the tag. InstrumentCommand counts are non-negative integers. Property
laws that pinned constants or echoed fields are replaced by laws over the
full score domain

BREAKING CHANGE: Report.Metrics#mutationScore and
mutationScoreBasedOnCoveredCode return Report.MutationScore, not number

Refs #114
determineExitCode checked the unscored case before the classifier, so the rule that an unscored run passes lived in two places. Every score now goes through classifyExit and only the log line depends on the score's tag
detected never exceeds counted for either score, so the clamp was dead and was the only rule on which Metrics and the merge-reports score differed
The unscored-row law derived its oracle from a hand-kept status table; it now asks Report.Metrics, so the merge score cannot drift from the report score unnoticed
@systemfsoftware-maker systemfsoftware-maker changed the title gh 114 fix(repo): pass a mutate set that matches no file with an empty report Sep 25, 2026
Only a scored run can break a threshold, so determineExitCode now dispatches on the score tag and hands the logs a plain percentage; the service no longer renders n/a
The merge workflow cannot call Report.Metrics from its decide body, so the law now requires each merged row's score to equal the Report.Metrics score; a drift in either formula fails the suite
@ryanleecode
ryanleecode merged commit 8738bcb into main Sep 25, 2026
7 of 9 checks passed
@ryanleecode
ryanleecode deleted the gh-114 branch September 25, 2026 16:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Stryker fails when the mutate patterns match no files, and the error it prints has an empty message

2 participants