fix(repo): make mutation CI fail truthfully and actually mutate - #117
Merged
Merged
Conversation
- Port the systemfsoftware mutation lane: timed plan, matrix, timings, report; no continue-on-error, and a package that leaves no report fails its job - Resolve runner/checker plugins from node_modules so a package never loads its own unbuilt dist; dry runs use unit/property suites only - Piped stdout gets human output; JSONL only with --json or STRYKER_MODE=machine, and the stream file is written in both modes - merge-reports rebuilds partial reports from the stream's _tag records - TypeScript checker checks mutated files and their dependents instead of the whole program per mutant (0/157 in 240s, now 157/157 in 35s)
- Share one stdout-or-drain sink between the file and stdout drains - Decide the stderr echo with a boolean match instead of an orElse fallback
- Decode part meta, reports, JOB/JOBS, timings and workspace inputs with effect Schema instead of casting parsed JSON - Clear earlier reports before a budget-skipped package so it fails the gate - Name shardMutate and the path when package.json is missing or unparsable - Cover shardMutate, installedPlugin, buildSummary, the incomplete-part rebuild, writeEncoded modes and the --json argv probe - Accept --json=true like the CLI parser; rename the stream rebuild module to report-from-stream.steps.ts
macOS links /tmp to /private/tmp, and module resolution reports the real path, so the installedPlugin and package.json diagnostics tests failed on the macOS runner
…graph Dependents that import the mutated file through a `compilerOptions.paths` alias were missing from the affected-file set, so a mutant that the whole-program check reported as a compile error was handed to the test runner instead Every tsconfig the checker walks now contributes its alias patterns, and a non-relative specifier resolves through them to the same candidate files the compiler resolves it to, alongside the existing relative and re-export edges
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Mutation CI now fails when a package produces no mutation report, and every package really gets mutated. Before this, run 36160660790 showed green while all three jobs had failed: the Mutation step had
continue-on-error, the report check accepted the stream file every run writes, andmerge-reportsexited 0 after merging 0 of 3 reports.What changes
continue-on-error. A package that leaves nomutation-report.jsonfails its job with a::errorannotation, while a low score alone does not. Outputs from an earlier run are deleted before each run, including when a package is skipped because its job budget ran out. Oversized packages are split withshardMutate. It runs on pushes tomainand manual dispatch only.node_modules. Each package'sstryker.config.tscalledimport.meta.resolvewith its own package name. Node answered through the package's ownexports, so the config loaded the package's unbuiltdist/(exit 4).installedPluginnow resolves the publishedcatalog:strykercopy.stryker-jsandstryker-vm-harnessuse avitest.mutation.config.tslimited to unit and property tests. The integration and differential suites had pushed the dry run past its 5-minute limit (exit 3).--jsonorSTRYKER_MODE=machine; in that mode stdout carries nothing else, and stderr carries no echo of each event.reports/mutation-stream.jsonlis written in both modes. Anything that parsed NDJSON from a pipedstryker runwithout asking for it must add--json.stryker merge-reportsnow reads the stream's_tag:"mutant"records. It had been decodingkind:"mutant", so a stream never rebuilt anything.On a 4-mutant sample, the new checker returned the same statuses as the published 7.1.0.
Review notes
A multi-reviewer pass found 10 actionable findings, and all of them are fixed on this branch. The fixes include decoding every script input with effect Schema instead of casting it, and adding tests for
shardMutate,installedPlugin,buildSummary, the incomplete-part rebuild, and the--jsonargv probe. One finding claimed that themerge-reportsflags changed. They did not: the base commit already declares--parts/--out(Cli.parts.ts), and this branch only corrects a stale README example.Validation
pnpm format:check,pnpm typecheck,pnpm check:ci,pnpm testandpnpm test:scriptspass on this head.mutation-jobrun on the checker package, using the published 7.1.0 checker, printedMutation produced no reportand exited 1. A stubbed run that crashed and left an old report behind also failed.catalog:stryker.Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.