Repository navigation
Conversation
When the 50 s request deadline cancels a router run, the engine's decision_path is lost with it. The error carried an empty path and provider null, so nobody could tell which route had hung. The host now records each provider attempt in a request-scoped ContextVar. On a deadline the shim rebuilds decision_path from that record: the attempt still waiting is reported as an in-flight timeout. The error message names the deadline and summarizes the attempts, and the call ledger's routing_summary now lists them as well. Timeouts now return 503 instead of 504. Cloudflare in front of router.ygr.ai replaces an origin 502/504 body with a plain-text "error code: 504", so callers never saw the JSON error. error.code is still "timeout", so clients can tell it apart from other 503s.
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars.
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Callers sometimes get a 504 from
router.ygr.aiafter exactly 50 s. On devexp-dev on 2026-10-28 there were 25 of them, all from theROUTER_REQUEST_DEADLINE_MSdeadline. Two things made them impossible to diagnose:decision_pathis lost with it. The response haddecision_path: []andprovider: null. The auth-proxy log and the call ledger couldn't say which provider or model hung.router.ygr.aiis behind Cloudflare, which replaces an origin 502/504 body with its own plain-texterror code: 504. The pod's nginx logged a 274-byte JSON body. The caller (Deckard) received only the Cloudflare text.Change
llm_router_host.attempt_logis a request-scopedContextVar. When the shim sets a list,execute_asyncrecords each provider attempt in it: provider, family, start time, and the outcome once it returns. This adds no work when unset.decision_pathon deadline. The shim builds the path from that record. An attempt still waiting at the deadline appears aserror_kind: "timeout", in_flight: truewith its elapsed time.routing_summaryalready readsdecision_path, so the call ledger now records these attempts too.router: request deadline of 50000 ms exceeded — <provider>/<family>=timeout "no response before the 50000 ms request deadline". The warning log line names the route it was waiting on.timeout. This covers the deadline,exhausted: timeout, and flow-node timeouts.error.codestill carriestimeout, so clients can tell it apart fromno_candidatesand capacity 503s. OpenAI SDKs retry 503 the same way as 504.Notes for reviewers
error.code == "timeout"instead.502(e.g.exhausted: server_error, which carries the per-attempt summary) is still lost for public callers. That is left for a follow-up because it touches the general error mapping.Tests
test_request_deadline_names_the_route_it_was_waiting_on: the first attempt times out and the second hangs. The 503 response lists both attempts, marks the second as in flight, and the message names it.test_sources/test_wallet_keepercame from running with-p no:logging, which removes thecaplogfixture, not from this change. CI runs the Lua suite.