feat(orchestrator): dispatch builds per speculation path - #503
Open
behinddwalls wants to merge 1 commit into
Open
feat(orchestrator): dispatch builds per speculation path#503behinddwalls wants to merge 1 commit into
behinddwalls wants to merge 1 commit into
Conversation
behinddwalls
marked this pull request as ready for review
August 4, 2026 01:52
behinddwalls
force-pushed
the
preetam/speculation-build-dispatch
branch
from
August 4, 2026 01:58
9cd7353 to
7ff20d4
Compare
This was referenced Aug 4, 2026
## Summary ### Why? The build stage triggered one build per batch, on the batch's full dependency list. Speculation needs one build per *path*, on the subset of dependencies the path assumes will succeed — that subset is what lets a batch be verified before the batches ahead of it resolve. ### What? The dispatch controller now reads the head's path set and starts a build for every pending path, on that path's own base. It **only starts builds** — stopping them belongs to the poll loop (next commit) — and it **never writes the path set**, which stays the speculate run's single-writer state; what this stage knows is the build it started, recorded in per-build rows of its own. The write order is Trigger → Build record → write-once link (`entity.PathBuild`) → signal; each write makes the previous one reachable, and the link is the idempotency point for redeliveries. A lost link race publishes both builds' signals and acks — the poll loop keeps the one the link names. On redeliveries only, every live linked path gets its signal re-published, closing the crash window between link and signal without forking duplicate poll chains past the queue's dedup horizon. The build signal carries only the runner's build ID and partitions on it, so one slow build cannot block a head's other paths. Known gap: a crash between `Trigger` and the link orphans that build; the fix is an idempotency key on `BuildRunner.Trigger`, marked TODO. Also adds `submitqueue/core/publish` — the registry-lookup-and-send plumbing and the message-ID deduplication rule in one place. ## Test Plan ✅ `bazel test //submitqueue/orchestrator/controller/build/... //submitqueue/core/publish/...` — write order, the base holding only assumed-success dependencies, redelivery republish without rebuild, lost-race double signal, halted batch starting nothing, redelivery healing for live linked paths, and a pin that no path-set write of any kind happens. ✅ `make fmt`, `make gazelle`, `make mocks`
behinddwalls
force-pushed
the
preetam/speculation-build-dispatch
branch
from
August 4, 2026 05:11
7ff20d4 to
2e32d07
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Why?
The build stage triggered one build per batch, on the batch's full dependency list. Speculation needs one build per path, on the subset of dependencies the path assumes will succeed — that subset is what lets a batch be verified before the batches ahead of it resolve.
What?
The dispatch controller now reads the head's path set and starts a build for every pending path, on that path's own base. It only starts builds — stopping them belongs to the poll loop (next commit) — and it never writes the path set, which stays the speculate run's single-writer state; what this stage knows is the build it started, recorded in per-build rows of its own.
The write order is Trigger → Build record → write-once link (
entity.PathBuild) → signal; each write makes the previous one reachable, and the link is the idempotency point for redeliveries. A lost link race publishes both builds' signals and acks — the poll loop keeps the one the link names. On redeliveries only, every live linked path gets its signal re-published, closing the crash window between link and signal without forking duplicate poll chains past the queue's dedup horizon.The build signal carries only the runner's build ID and partitions on it, so one slow build cannot block a head's other paths. Known gap: a crash between
Triggerand the link orphans that build; the fix is an idempotency key onBuildRunner.Trigger, marked TODO.Also adds
submitqueue/core/publish— the registry-lookup-and-send plumbing and the message-ID deduplication rule in one place.Test Plan
✅
bazel test //submitqueue/orchestrator/controller/build/... //submitqueue/core/publish/...— write order, the base holding only assumed-success dependencies, redelivery republish without rebuild, lost-race double signal, halted batch starting nothing, redelivery healing for live linked paths, and a pin that no path-set write of any kind happens.✅
make fmt,make gazelle,make mocksStack