Skip to content

feat(speculation): allocator contract and sticky impl - #450

Open
behinddwalls wants to merge 1 commit into
preetam/speculation-generatorfrom
preetam/speculation-allocator
Open

feat(speculation): allocator contract and sticky impl#450
behinddwalls wants to merge 1 commit into
preetam/speculation-generatorfrom
preetam/speculation-allocator

Conversation

@behinddwalls

@behinddwalls behinddwalls commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

Summary

Why?

Ranked candidates need a budget policy: something must decide which paths get the queue's limited concurrent-build slots, without double-funding paths that are already running or finished.

What?

Adds submitqueue/extension/speculation/allocator — the budget-spending contract the standard Speculator hands its candidate stream to — plus the sticky implementation and mocks.

The Allocator owns terminal-path suppression: it reconciles the stream against the stored path sets by path ID, so an in-flight candidate keeps the slot it already holds and a terminal one is skipped. Budget accounting reads path status and nothing else — pending, building, and cancelling paths hold a slot until their builds actually stop — and the unit is deliberately coarse: one slot per concurrent build.

sticky fills only free slots and never preempts a running build; its policy counterpart is a preempting allocator. Output is all-or-nothing (a partial list would fund an arbitrary prefix of the ranking), and a stored path with no status fails fast rather than guessing whether it holds a slot.

Test Plan

bazel test //submitqueue/extension/speculation/allocator/... — terminal candidates skipped per status, in-flight candidates kept, budget exhaustion, context cancellation, corrupt-status failure.

make fmt, make gazelle, make mocks

Issues

Stack

  1. feat(speculation): generator contract and bestfirst impl #446
  2. @ feat(speculation): allocator contract and sticky impl #450
  3. feat(speculation): standard composed speculator #451
  4. feat(storage): speculation path set store #501
  5. feat(storage): path-build link store for per-path builds #502
  6. feat(orchestrator): dispatch builds per speculation path #503
  7. feat(orchestrator): poll builds and stop the ones nothing wants #504
  8. fix(orchestrator): mint distinct message IDs for cancel re-publishes #505
  9. feat(orchestrator): re-plan the queue from the Speculator each run #506
  10. feat(orchestrator)!: finalize batches from their speculation paths #507
  11. feat(orchestrator): compose per-queue speculators and turn speculation on #508

@sbalabanov sbalabanov left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical comment about failing fast

Comment thread submitqueue/extension/speculation/allocator/README.md Outdated
Comment thread submitqueue/extension/speculation/allocator/README.md Outdated
Comment thread submitqueue/extension/speculation/allocator/sticky/sticky.go Outdated
Comment thread submitqueue/extension/speculation/allocator/sticky/sticky.go Outdated
@behinddwalls
behinddwalls force-pushed the preetam/speculation-allocator branch 2 times, most recently from bf09ae6 to 6977b91 Compare July 29, 2026 18:24
@behinddwalls
behinddwalls force-pushed the preetam/speculation-allocator branch 2 times, most recently from d6f1dfe to af1efe4 Compare July 29, 2026 22:41
@behinddwalls
behinddwalls force-pushed the preetam/speculation-allocator branch 2 times, most recently from 2b3a122 to 32e21b9 Compare July 29, 2026 23:18
@behinddwalls
behinddwalls force-pushed the preetam/speculation-allocator branch from 32e21b9 to 00ca258 Compare July 29, 2026 23:18
@behinddwalls
behinddwalls force-pushed the preetam/speculation-allocator branch from 00ca258 to 80c246f Compare July 30, 2026 01:19
@behinddwalls
behinddwalls force-pushed the preetam/speculation-allocator branch from 80c246f to 0049b73 Compare July 30, 2026 07:31
@behinddwalls
behinddwalls requested a review from sbalabanov July 30, 2026 07:37
## Summary

### Why?

Ranked candidates need a budget policy: something must decide which paths get the queue's limited concurrent-build slots, without double-funding paths that are already running or finished.

### What?

Adds `submitqueue/extension/speculation/allocator` — the budget-spending contract the standard Speculator hands its candidate stream to — plus the `sticky` implementation and mocks.

The Allocator owns terminal-path suppression: it reconciles the stream against the stored path sets by path ID, so an in-flight candidate keeps the slot it already holds and a terminal one is skipped. Budget accounting reads path status and nothing else — pending, building, and cancelling paths hold a slot until their builds actually stop — and the unit is deliberately coarse: one slot per concurrent build.

`sticky` fills only free slots and never preempts a running build; its policy counterpart is a preempting allocator. Output is all-or-nothing (a partial list would fund an arbitrary prefix of the ranking), and a stored path with no status fails fast rather than guessing whether it holds a slot.

## Test Plan

✅ `bazel test //submitqueue/extension/speculation/allocator/...` — terminal candidates skipped per status, in-flight candidates kept, budget exhaustion, context cancellation, corrupt-status failure.

✅ `make fmt`, `make gazelle`, `make mocks`
@behinddwalls
behinddwalls force-pushed the preetam/speculation-allocator branch from 5a83bb7 to 23a30ad Compare August 4, 2026 05:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants