fix(provider): one warm instance that does not arrive no longer fails the pool; release .120 - #373
Merged
Conversation
… the pool The warm reconciler treated two ordinary events as fatal. An instance that never published its readiness evidence within the minute returned a timeout, and an instance whose delete had begun answered a file read with "Failed getting instance pool: Instance storage pool not found" rather than 404. Either aborted the whole pool's reconcile, so the depth was not restored during exactly the bursts that consume it: on 2026-09-02 that happened eleven times between 09:35Z and 11:00Z, each one also marking the services host unhealthy for a minute. An instance that never becomes ready is now deleted and recorded in the result as abandoned; the deficit refills on the next pass. An instance that vanished mid-read is skipped, the way one retired during create already was. A pool still fails on anything that is not one instance's own problem. Claude-Session: https://claude.ai/code/session_0128syXKxAGCfJGRDxUUNQXp
… instance Source 3a77ae2, built twice with CGO_ENABLED=0 go build -trimpath -buildvcs=false -ldflags "-buildid= -s -w -X main.version=v0.1.5-nddev.120 -X main.commit=<source>"; both builds agree on 3c07b99b8d11f1bcded6208147540522aaa37302bf2203da78937bdbd2c70368. Claude-Session: https://claude.ai/code/session_0128syXKxAGCfJGRDxUUNQXp
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The warm reconciler treated two ordinary events as fatal:
did not publish readiness evidence: timed out)Failed getting instance pool: Instance storage pool not foundrather than 404Either aborted the whole pool's reconcile, so the warm depth was not restored during exactly the bursts that consume it. Measured on 2026-09-02: eleven aborted reconciles between 09:35Z and 11:00Z, each also marking the services host unhealthy for a minute (visible only since #369 made the observer demand these units).
An instance that never becomes ready is deleted and recorded in the result as
abandoned_during_create; the deficit refills next pass. An instance that vanished mid-read is skipped, the way one retired during create already was. A pool still fails on anything that is not one instance's own problem.stateOperationTimeoutbecomes a var so the test can shorten it.Release v0.1.5-nddev.120 (two reproducible builds agree).
https://claude.ai/code/session_0128syXKxAGCfJGRDxUUNQXp