Run two verify:quint jobs at once - #382
Merged
Merged
Conversation
Main's verification step now takes 19 to 31 minutes of its 50-minute bound, and six open lanes each add a model or deepen one. On main at 4a89c83 the model checks took about 21.5 minutes and the replay run 6.6 minutes, all serial, on a 4-vCPU, 16 GB runner that the checks use one process at a time. verify:quint now treats each model's checks as one job and the replay run as another, and runs two jobs at once through runAtMost. Every check, bound, and verdict is unchanged. After a job fails no further job starts, the running jobs finish, and the earliest-listed failure is rethrown, so a failure still fails the step and Required. Two jobs hold at most two Apalache heaps of 4 GiB each. Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The
verificationjob is part ofRequiredand runs everything serially. On main thebun run verifystep now takes 19 to 31 minutes of its 50-minute bound (#379 raised the bound as slow-runner headroom). On main's run at 4a89c83 (job 30m41s) the Quint model checks took about 21.5 minutes (fence alone 7.7) and the replay run 6.6 minutes, one process at a time on a 4-vCPU, 16 GB runner. Six open lanes add or deepen models (#380 promotion.qnt alone adds about 7 minutes of checks and 3 of replay on CI), which would push the step past its bound.Change
verify:quinttreats each model's checks (typecheck, simulation, mutant simulation, Apalache, Apalache mutants) as one job and the replay run as a second kind of job, and runs two jobs at once through a newrunAtMosthelper (QUINT_CONCURRENCY = 2). Two jobs hold at most two 4 GiB Apalache heaps on the 16 GB runner.Unchanged: every command, bound, seed, timeout, verdict rule, log file, and artifact. The replay run still covers every model's replay test in one
bun testprocess.toolchain.jsonstill lists the models in manifest order; it is now written after the replay run too.Failure semantics: after a job fails no further job starts, the running jobs finish (each is bounded by its own checker timeout), and the failure of the earliest-listed failed job is rethrown. A failure still fails
verify:quint, the step, the job, andRequired.Tests
scripts/verification-tools.test.ts"concurrent checker jobs": ordered start and the concurrency cap with gated tasks; no start after a failure, waiting for running tasks, and the earliest-listed failure rethrown; invalid limits rejected before any task runs; and a property over random task plans and limits (assertAsyncProperty) checking once-only in-order starts, a peak of exactlymin(limit, tasks), no start after a failure, and the rethrown failure.runAtMosteach fail these tests: ignoring failures (2 tests fail), ignoring the limit (3), rethrowing the first-in-time failure (2), and returning before running tasks finish (3).bun run verify:quinton a manifest narrowed tolockandmessagingpassed in 1m22s with the replay run and model checks interleaved, and wrotetoolchain.jsonin manifest order; the manifest was restored.bun run typecheckandbun run verify:claims(130 pass).Review focus
This changes how the required
verificationjob runs its checks, so it needs independent review against the prior coverage (AGENTS.md). The job, step, commands, and bounds in.github/workflows/ci.ymlare untouched; this PR's own verification run measures the new duration.Generated with Devin