Repository navigation
fix(llm): honour the configured request timeout on generate - #240
Merged
Merged
Conversation
Generate handlers forced a 600-second fallback, so a provider's timeout setting and its env variable applied only to structured output. The client now resolves one default, and the declared default matches it.
skhaz
approved these changes
Nov 4, 2026
added 4 commits
November 4, 2026 16:58
Every driver accepts seconds or a duration string such as 20m through transport.timeout_seconds, the deadline cap compares numbers, and an invalid value is an invalid_request naming its source rather than a silent default.
An explicit empty setting stays empty, so an empty api_key cannot pick up a credential from the environment. Every HTTP request gets its timeout as a millisecond duration, because the runtime reads a number as whole seconds.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Generate handlers set
timeout = contract_args.timeout or 600, so a driver's configured timeout and its env variable (ANTHROPIC_TIMEOUT,BEDROCK_TIMEOUT,OPENAI_TIMEOUT,OPENAI_COMPAT_TIMEOUT) never applied to generate, only to structured output. Claude's manifest also declared 240 while the code used 600.Changes
options.timeout, routecontext.timeout, driver config or env, then 600 seconds.transport.timeout_secondsparses every timeout: seconds, a numeric string, or a duration such as20mthroughtime.parse_duration. All drivers use it, including Google and typesafe.deadline_mscompares numbers in the transport, so a duration string is capped correctly.Behaviour change
soonis now aninvalid_requestnaming its source, where it used to fall back silently.Tests
src/llm/test/timeout_test.luachecks each precedence level at the HTTP boundary for generate and streaming on every driver. llm 2658 pass, 9 skipped, lint clean. Tests for the removed fallback cap moved to the transport. No live provider call.