feat(proxy): coalesce concurrent misses for a key - #21
Merged
Merged
Conversation
jamiehdev
force-pushed
the
feat/request-coalescing
branch
from
September 27, 2026 16:46
ebe94af to
d518329
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When many requests miss on one cache key at once, the proxy now sends one request to the origin and the others wait for it. This stops a burst of origin traffic when a popular entry expires. Stale entries without
stale-while-revalidatecoalesce too, so the origin gets one conditional request.Changes
Varynames the cache knows, so known variants coalesce separately.GETandHEADrequests that the cache can serve take part, and only aGETleads. Requests withAuthorization,Cookie, a conditional header or requestno-cachego to the origin as before.Teestores the body or gives up the copy, when the origin fails, or when the client disconnects. Followers then read the cache and getX-Shadowstep-Cache: COALESCED./healthhas acoalescedcounter, andhit_ratiocounts it.Verification
All twelve tests in
tests/integration/coalescing.rsfail onmain. Some fail there only on the absentcoalescedcounter, so deliberate faults test them too. A guard that never drops makes seven tests time out, the disconnect test among them. Coalesced credentials fail the bypass test, and flights that ignoreVaryfail the known-variants test. The suite passed five runs in a row.Risks
Actix does not see a client disconnect while the origin sends nothing. In that case, followers wait for the full upstream timeout.