Repository navigation
feat(repo): judge every e2e journey by authored mutant annotations - #128
Merged
Merged
Conversation
… mutants as Pending
…y catalog names and add the e2e core oracles
…gh published contracts
…d remove the blessed baselines
…segments to a leading letter
…lly disabled witness journeys
…s journey to name it
…tics on the contract branch
# Conflicts: # .changeset/mutant-ids-are-decimal.md # .changeset/thresholds-are-one-schema.md
…for CI coverage runs
… stream document differentially
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Every e2e journey now checks each reported mutant against an outcome a person wrote beside the mutated code, instead of against numbers recorded from an earlier engine run. A misclassified status therefore fails its journey with the file, line and both outcomes, where before it was absorbed into a re-blessed baseline. The Stryker CLI and plugin interface now publish their wire formats, stream, report, span taxonomy and mutator catalog as versioned contracts, and the lane decodes only through them.
This is the follow-up to #127's review, delivered as one wave per
docs/plans/2026-09-27-0424-refactor-contract-first-mutation-oracles-plan.md. Releasing waits for your approval.What reviewers should check
stryker-js-plugin-interface. New@systemfsoftware/stryker-js-cli-contractowns the stream schema, span taxonomy and stock catalog, and commits each as a JSON document undercontract/. A guard fails a contract document change that has no matching version bump. Changesets:stryker-js,plugin-interfaceandinstrumentermajor;cli-contractandtypescript-checkerminor.perTestcoverage now reports an uncovered mutant asNoCoveragewithout running it. An interrupted run's checkpoint lists each unsettled planned mutant asPending.<namespace>/<Name>.optInMutationsandexcludedMutationsare validated against the loaded catalogs before instrumentation.// @stryker-expect <scope> <outcome>: <mutators>lives in fixture source.CompileError(TSnnnn)andRuntimeError(Type)name their cause. An in-process placement check confirms every authored compile code with the real checker, and status closure proves every status has a witness journey.Engine bugs the authored oracle found
Every one of these was hidden by the recorded baselines, and each is fixed here:
// Stryker disablethat ignored a prefix of a node's mutants shifted the remaining replacements onto the wrong mutant ids.CompileErrorreason dropped theTScode.The first two have solution docs under
docs/solutions/runtime-errors/.Seeded faults
pnpm --filter @systemfsoftware/stryker-e2e seed-status-fault <status>patches one misclassification into a scratch copy and runs the journey the witness registry assigns. A run counts asredonly when the failure names that misclassification. Every status shipped by the product was caught:enterprise-composite-checker.e2e.test.ts26688e163a88053919a7f36d68beffb9enterprise-mutation-lifecycle.e2e.test.tse109bc3699c8cbf0249ca2cb00846297mutation-run.e2e.test.ts785d36c8bcdd635597ee0f4e71766df2enterprise-mutation-lifecycle.e2e.test.tsfd97f9916f143c69e10f86fee39b0a64enterprise-runner-resilience.e2e.test.tsbe0fb03ed075b193cd457849e4797f3denterprise-mutation-lifecycle.e2e.test.ts047804c98e400b3ba05713d5c08c7ee5svelte-app.e2e.test.tsd18916e557590d25723197e71e6e17f0enterprise-runner-resilience.e2e.test.tsfd6f90bee6198a7bffb95a8a888ccaf4Verification
585ed775b: bothcheckjobs and all five e2e shards pass.585ed775b: 11 files / 16 tests passed in 297 s, under the 1200 s shard cap.pnpm testacross the workspace passes, and so do the format check, typecheck,check:ci,guard:projectsselftests andcheck-changeset.Found while getting CI green
SimpleSpanProcessor, one OTLP request per span (about 11k per run), and the exporter refuses any export past 30 in flight. On CI the lifecycle journey's rerun loststryker.run_requestand itswritespan, which cut every phase off fromstryker.cli.runin the trace contract (trace50a1eccb8432e61850b11c34e754192c). The CLI now uses aBatchSpanProcessor. A probe against a collector that answers in 300 ms received 30 of 400 spans before the change and 400 of 400 after.∀x_RunEventWireLine_=xbecause verdict lines reused report shapes whose rest records let the generator write escaped keys that V8'sJSON.parsemisreads (nodejs/node#63785). The verdict now uses the fixed-fieldVerdictThresholdsandVerdictLocationstructs.stream-document.differential.test.tscompares the committed document with the wire codec on lines generated from both: they must refuse the same lines and reproduce the same JSON. A scratch candidate that drops one verdict field turns it red.Dependencies and test names
@systemfsoftware/*dependency is on its latest release; thecatalog:strykerdogfood entries staylatest, now resolving tostryker-js13.1.0 and the 8.0.1 runner and checker. Thedifferential-spechost-bound patch is ported unchanged from 0.6.0 to 0.6.1, which still runs host-bound comparisons on the simulation kernel.run-event-wire-line.integration.test.ts, the report codec suite isreport-codec.integration.test.ts, and the document-versus-codec parity check is aDifferential.comparespec. The newtest-suffix-outside-srclint rule now refuses the*.contract.test.tssuffix.*.cell.tsand pure imports in*.schema.ts. Helpers moved out of 12 cell files into plain modules,Checker.cell.tssplit intocheck.cell.tsandgroup.cell.ts,validate-options.cell.tssplit fromload-config.cell.ts, and the generated report module is nowgenerated/report.schema.ts. No published API report changed.Residuals
api:checksometimes rendersstryker-js'sPlugin.Nodeexport without theNode_2 asalias. The committed report keeps the alias, which is what CI renders onmain.extendsslot can hit the same missing-parens hole as the ternary test. It is documented, not fixed.Session-settled decisions carried from planning: authored contracts that producers compile against (user-directed, over contracts generated from the producer); annotations with a coverage model (user-directed, over runner snapshots); no engine-output snapshot anywhere (user-approved); annotated causes for CompileError/RuntimeError (user-approved); one catalog model with the provider loader in this wave (user-directed); two contracts (user-approved); Effect core for schema codegen (user-approved); one wave (user-directed); NoCoverage and Pending producers (user-directed).
Related: #127
Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.