Skip to content

CI: run the e2e suite's three parts in parallel - #80

Merged
rzonyx merged 1 commit into
masterfrom
faster-ci
Nov 3, 2026
Merged

rzonyx merged 1 commit into
masterfrom
faster-ci

Conversation

@rzonyx

@rzonyx rzonyx commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

The e2e job was CI's slowest at about 9 minutes; every other job takes under 2. A release waits for CI on the tagged commit, so E2E set the release time: v0.3.11 took 9.5 minutes end to end.

scripts/e2e.sh now takes E2E_PART, and ci.yml runs its three parts as a matrix. Times are from the last run's log:

Part Sections Time
runs the fixture suite, plus the commands that read its job: jobs, traces, the viewer ~220s
isolation integrity, egress, secrets ~90s
cli offline commands, authoring, trials, datasets, sweeps, exec, analyze/check, start-env/cache, live sync ~220s

Each part keeps the leaked-container check and uploads its own job dirs (e2e-jobs-<part>) when it fails. Unset, the script runs all three as before. Sections are only wrapped in if part …; then … fi; their bodies are unchanged.

With the split, E2E should take about 4–5 minutes, and a release about 6, limited by the 5-minute release builds.

The macOS jobs are unchanged. Their runners queued on 2 of the last 14 CI runs: v0.3.9 waited up to 34 minutes, and v0.3.12 about 14. Building both macOS targets in one job would add about 5 minutes to every release, to save that wait only some of the time.

Checks

  • bash -n scripts/e2e.sh passes, and an unknown E2E_PART exits 2 with a message.
  • This PR's own CI is the real test of the split: all three E2E parts should pass, each at about a third of the old time.
  • Branch rules require no status checks, so the renamed E2E (<part>) jobs don't block merging.

The e2e job was the slowest in CI (about 9 minutes, against 2 for every other job), and a
release waits for CI on the tagged commit, so it set the release's time. E2E_PART now runs one
of three parts of scripts/e2e.sh, each about a third of the suite in the last run: runs (the
fixture suite and the commands that read its job: jobs, traces, the viewer; ~220s), isolation
(integrity, egress, secrets; ~90s) and cli (the rest of the CLI; ~220s). The ci.yml job is a
matrix over the three, each uploading its own job dirs on a failure. Unset, the script runs all
three as before.

The macOS jobs stay as they are. Their runners queued for minutes on two of the last 14 CI
runs, but building both macOS targets in one job would add about 5 minutes to every release to
save that wait only sometimes.
@rzonyx
rzonyx merged commit ac2a7f3 into master Nov 3, 2026
11 checks passed
@rzonyx rzonyx mentioned this pull request Nov 5, 2026
@luiyen luiyen mentioned this pull request Nov 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants