refactor(repo): run testRunner vm on the vitest runner and delete the vm harness - #111
Merged
Merged
Conversation
systemfsoftware-maker
force-pushed
the
refactor/vm-runner-vitest-delegation
branch
from
September 25, 2026 12:19
f5f8117 to
0eba4d7
Compare
systemfsoftware-maker
added this pull request to stack #112
September 25, 2026 14:15
… vm harness The vm runner re-implemented Vitest behaviour and reported 144 of 319 enterprise-fixture mutant statuses differently from Vitest. testRunner vm now runs the vitest runner on Vitest's isolated threads pool, so its verdicts are Vitest's own, and @systemfsoftware/stryker-vm-harness is removed Refs #100
…test runner A test file that fails as a whole (a suite registering no tests, a load-time throw, a beforeAll or afterAll failure) now reaches the mutant decision as a failing test, so the mutant is reported Killed instead of Surviving, and a dry run loading such a file fails naming it Skipped tests that never start are collected as skipped instead of being dropped from the run results A related-only start whose filter matches no test file is recognised from the rendered error code, so the run continues with no tests to kill the mutant instead of stopping with a runner crash Refs #100
A Vitest test declared with test.todo carries mode todo and no result, which the run-status mapping read as failed; it is now reported as skipped, as Vitest itself reports it Refs #100
… harness The harness compared a killed mutant's status reason against the deleted vm harness's literal; it now uses TestRunner.HitLimitReasonPrefix so a runner hit-limit kill is recognised as the runaway verdict Refs #100
Also regenerates the stryker-js API report for the new Plugin exports Refs #100
The vm test runner config is a pure rewrite, not a blueprint, so it moves out of the blueprint kind file and the blueprint-kind lint gate stops applying; the parity candidate child runner is built with a piped Option so the effect-tsgo pipeability diagnostic is gone Refs #100
…mutated code The vm runner now runs Vitest, which narrows the dry run to tests related to the mutated files; the fixture's test imported nothing, so the dry run found no tests. The workspaces also run the real platform ports now that vm spawns a worker Refs #100
… a real worker Replaces the scenarios that pinned the resolved runner config and the runner kind with a dry run of a browser-mode project, and moves the pool characterization out of the runner's src tests, which the test-discipline rules reject Refs #100
The vm runner narrows the dry run to tests related to the mutated files, and the workspace's only test imported nothing, so the run stopped before writing its report Refs #100
… failure The bail setting was fixed when the Vitest instance was created, so it applied to the dry run too and a failed dry run named only its first failing test. The runner now sets each project's bail per run: never for the dry run, as configured for mutant runs. The dry-run-failure workspace's tests now import the module they cover, since the runner only runs related tests Refs #100
Real Vitest fails a file whose empty describe registers no tests, and the vitest runner reports such a file as one failed test named by its path. The reference arm now records the same outcome from Vitest's file result, under a key both sandboxes share Refs #100
ryanleecode
force-pushed
the
refactor/vm-runner-vitest-delegation
branch
from
September 25, 2026 14:19
0eba4d7 to
51793fe
Compare
After merging main, a mutate set whose files produce no mutants reached the dry run with those files as its related set. The Vitest runner found no related test and the run failed with No tests were executed, where main expects an empty report. The dry run now relates tests only to files that carry mutants, and the runner runs every test when that list is empty Refs #100
With AGENT set the shared config bailed after one failure. The cancelled run then waited forever on the vm parity differential's worker, so a single failing test turned pnpm test into a hang until the caller's timeout. Agent runs now finish and report every failure, as CI runs do Refs #100
…ne console
The capture layer ran every first string argument through printf-style substitution, so console.log('%%') recorded a single percent sign where Node prints the string unchanged. The merge-reports summary property found it on macOS when it drew a summary containing %%. A single string argument is now recorded as given, as Node does
Refs #100
Merged
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
testRunner: 'vm'now runs the Vitest runner on Vitest'sthreadspool, so its verdicts are real Vitest's.@systemfsoftware/stryker-vm-harnessis deleted: about 30,000 lines that re-implemented Vitest behaviour. On the enterprise fixture that copy reported 144 of 319 mutant statuses differently from real Vitest.vmis still the zero-plugin default.@systemfsoftware/stryker-jsdepends on the Vitest runner through the@systemfsoftware/stryker-js-vm-runneralias and resolves it from its own install, not the project's.pooloption. Withpool: 'threads', it refuses browser-mode projects and tells the user to usetestRunner: 'vitest'.vmran through the runner:describe, a load error, or a failingbeforeAll) is reported as a failed test.skipped.vm.Plan:
docs/plans/2026-09-25-0809-refactor-vm-runner-runs-vitest-plan.md. It supersedes2026-09-25-0624: the U2 benchmark picked path C, thethreadspool.Fixes #100
Layer 2 of stack #112, on top of #110. Merging this PR also merges #110. If #110 merges first, GitHub rebases this PR onto
mainand changes its base.Validation
Local, on
f5f8117dd(before rebasing onto #110's peer fix):pnpm format:check,pnpm typecheck,pnpm test,pnpm check:ci, and./scripts/check-changeset.tsall pass.tests/vm-parity.differential.test.tspasses 14/14: dry-run and mutant verdicts on every fixture, plus generated suites. CI passed on that commit, including e2e with the four enterprise journeys onvm.Final build, enterprise fixture (319 mutants; one warm-up, then the median of 3 alternating runs through the CLI):
vitestrunner (stock)vm(vitest runner,threads)U2's old harness took 13.9 s, but 144 of its 319 verdicts differed from real Vitest.
R3 (#105): I packed
stryker-jsand the vitest runner and installed them into a fresh pnpm project (default isolated linker) whose only direct dependencies werestryker-jsandvitest. There, avmdry run and a mutation run both exit 0, with 2/2 mutants killed. The alias resolves from Stryker's own install; the project cannot resolve the runner.Known, not changed here: the shared vitest config sets
bail: 1only whenAGENTis set. After a bail, the forks worker runningvm-parity.differentialdoes not terminate, so an agent-runpnpm testwith an unrelated failure hangs until its timeout. CI does not bail.