feat(repo): run the e2e lane as effect specs over traces it owns - #127
Merged
Merged
Conversation
stryker.cli.run takes an external parent decoded from TRACEPARENT and TRACESTATE through Trace.Traceparent; an absent or malformed carrier keeps the fresh root span
Adds the Effect harness layer, the StrykerRun stimulus that hands the CLI a TRACEPARENT per run, stream decoding, Tempo remote observation and a collector preflight, and converts the mutation-run journey into an effect-gherkin-spec feature
…ions Ports the eight derive-oracle properties to Metamorphic.on and Differential.compare with shrunk counterexamples; the oxc-vs-ts-morph comparison is host-bound because the oxc instrumenter needs a real Effect runtime
The five typescript-checker journeys become one feature: a scenario outline over the build-mode, include and preset configs, the broken checker scenario and the vm-run report scenario, each keeping its authored counts
blessed-baseline runs under NodeRuntime.runMain on the harness services and the node FileSystem, with the bake as a scoped layer; refusals of non-verdict runs, non-zero exits and the sabotage slice are unchanged
failing-run, vm-vitest, vitest-nested-describe and svelte-app become features on the Effect harness; persisted reports are derived states read from the run's own fork and every authored literal is kept
edge, composite-checker, resilience and sabotage become features on the Effect harness with their oracle literal blocks byte-identical, so reconcile-oracle and the drift check still splice them
A test-side trace-taxonomy declares the lifecycle spans the exporter writes; the lifecycle feature judges its one CLI run through that contract over Tempo, replacing the hand-rolled span-name checks and the OTEL_ENABLED gate
Every journey now runs on the Effect harness, so microvm-harness.ts and tempo.ts go; the terminal-verdict accessor the enterprise journeys shared moves into the stream fixture, and trace break dumps are ignored
The carrier decode changed the declaration graph, and api-extractor now renders the exported Node without the Node_2 alias on every forced run; the API itself is unchanged
…ctor The lane docs name the Tempo preflight, TEMPO_URL, the owned trace id per run, Break dumps, the targeted-run command, and the widened E2E-2 oracle-literal rule for shared verify helpers
reportedMutantsOf and runIdsIn move into the stream fixture; the journeys and the checker helper that carried identical copies import them
The harness fixture, global setup and the bless script build their layers from HarnessServicesLive and HarnessPlatformLive instead of three hand-copied compositions
The checker counts already lived in the fixture verify helper on main, so the lane rule stays as it was rather than widening a judgment surface
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The e2e lane now runs as Effect
effect-gherkin-specfeatures on one Effect harness. Each CLI run owns its trace: the harness hands the CLI aTRACEPARENT, and the lifecycle journey is judged by atrace-speccontract over that trace as Tempo returns it, instead of by span names found through search. To make that possible, thestrykerCLI now continues a caller's trace fromTRACEPARENT/TRACESTATE(minor bump for@systemfsoftware/stryker-js).What reviewers should check
stryker.cli.runtakes an external parent decoded through the existingTrace.Traceparentschema. An absent or malformed carrier keeps today's fresh root, as the OpenTelemetry environment-carrier spec asks.ORACLE-LITERALSblocks byte-identical (check:oracle-drift: no drift). gherkin-spec refuses a second assertion on one state, so each oldbddStep('And', …)became aWhenthat derives a new state followed by one compositeThen.typescript-checker-*journeys are one feature: a scenario outline over the build-mode, include and preset configs, plus the broken-checker and vm-run scenarios. It runs in 261 s, inside the rest shards' 1200 s cap, with one forked microVM at a time.pnpm lgtm:up; it setsOTEL_ENABLED=truewhen unset and refuses an explicitfalse.differential-specrelations. The eight properties becameMetamorphic.on/Differential.comparerelations with shrinking. The oxc-vs-ts-morph comparison ishostBoundrather thanrunBudget-only (a deviation from the plan): the oxc instrumenter needs a real Effect runtime, and on the simulation kernel it deadlocks on async finalizers.Plan:
docs/plans/2026-09-26-1939-refactor-e2e-effect-spec-libraries-plan.md.Verification
vitest run, all journeys)TRACEPARENTset:stryker.cli.runparented on the carried span id. Unset or malformed: fresh rootpnpm --filter @systemfsoftware/stryker-js testtest:oraclebless-oracle -- --verify edgeformat:check,typecheck,test,check:ci, changeset gateEach live journey runs its CLI twice:
@systemfsoftware/vitestre-runs a passing live test once on a fresh build as a leaked-state check. Expect two owned trace ids per journey in Tempo.New concepts
Owned-trace contracts (
trace-spec). A test that asserts on spans owns the trace it reads, rather than searching the collector for one.Stimulus.makemints a trace id per run and hands it to the system as a W3Ctraceparent; aContractof declared spans and relations (Rel.exists,Rel.descendant) is then judged against exactly that trace, read back throughRemoteObservationuntil it settles. Here it replaces a span-name search over Tempo that could match another run's trace. It fits whenever the system under test runs in another process and can accept a trace carrier. It does not fit when the system cannot be told its parent context: then there is nothing to own.Unapplied review findings
Applied in this branch: #1 (E2E-2 rule text reverted to
main; the checker counts already lived in the fixture helper there) and #2 (one sharedsrc/Harness/harness-layers.tsfor the fixture, global setup andblessed-baseline.ts).@systemfsoftware/stryker-js/stryker-js-plugin-interfaceas valuesPre-existing on
main(the journeys already decodedRunEvent.RunEventWireLinefrom the workspace package); this branch only moved the import into one fixture. Either amend E2E-3 to admit the published wire-line and report codecs or move the decode behindtestResources/, as a separate owner decision.Not reproduced: each
StrykerRunmints a fresh random 128-bit trace id and the contract reads only that id; four lifecycle runs each produced exactly one owned trace.Not reproduced on the pinned Tempo:
GET /api/v2/traces/<unknown id>answers 200{"trace":{},"metrics":{}}, which reads as zero spans and keeps polling. Would bite only on a Tempo that returns 404; the fix would be upstream in trace-spec'sTempoTraceStore.source.OtelTracer.makeExternalSpanSuggested:
makeExternalSpan({ traceId, spanId, traceFlags, traceState })instead of passing the decoded parts record.TRACEPARENTfalls back to a fresh root silentlyThis is the specified behaviour (plan KTD4, OpenTelemetry env-carrier spec); a warn log would be a product decision.
describeEach relation registers as a named vitest test (
test:oracle: 6 files, 39 tests; each selectable withvitest -t <relation>).OTEL_ENABLEDrefusal have no automated testBoth were exercised by hand this run (
TEMPO_URL=http://127.0.0.1:1fails namingpnpm lgtm:up;OTEL_ENABLED=falseis refused).TimedOut@systemfsoftware/effect-readiness0.4.1 types the verdict asSatisfied | TimedOut, so every non-ready verdict is covered.The plan keeps the contract to the claims the old span-name check made; an exclusivity relation would be a new claim.
Review run:
20260926-211536-c46880af(ce-code-review, mode:agent; reviewers: correctness, project-standards, testing, maintainability, learnings, reliability, adversarial — in-process, no cross-model peer CLI on the host). Artifacts:/tmp/compound-engineering-0/ce-code-review/20260926-211536-c46880af/.Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.