Stop calling a model API rejection a tool failure - #668
Open
elias-ba wants to merge 2 commits into
Open
Conversation
elias-ba
force-pushed
the
fix/name-model-api-errors
branch
2 times, most recently
from
September 4, 2026 23:18
b4d5cea to
d828ae3
Compare
The planner's tool loop caught every exception and re-raised it as a 500 "Tool execution error", so a rate limit, an overload or a bad request all reached Lightning looking like a tool that had broken. That makes the two useful questions unanswerable: is this our code or theirs, and is it worth retrying. Model API failures are caught separately now and given a status that says whose failure it is. A caller only shapes how much text they send, so those are the rejections that keep their upstream status. The model, the beta headers and the tool definitions are ours, so a rejection of any of those is a 502 rather than a verdict on the caller, which is what the auth README means by never a misleading 401. The message is written for whoever ends up reading it, because Lightning renders it straight into the chat. The upstream sentence was not: it says things like "invalid x-api-key", which describes our key. It goes to details.upstream_message and Sentry instead, masked. The kind is read from the response body rather than the SDK class. A failure that arrives mid-stream is always a bare APIStatusError carrying the 200 that opened the stream, because the SDK builds it from that response, so anything keyed on the class would be dead code on the path global chat uses.
elias-ba
force-pushed
the
fix/name-model-api-errors
branch
from
September 4, 2026 23:21
d828ae3 to
ebc979a
Compare
Only conflict was the util import line, where main added format_attachments and this branch added mask_secrets. Both kept.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Short Description
The planner's tool loop turned every exception into a 500 "Tool execution error", so a rate limit, an overload or a bad request all reached Lightning looking like a broken tool.
Implementation Details
Model API failures are caught separately now and get a real status and a message written for the person who ends up reading it, since Lightning renders it straight into the chat.
A caller only shapes how much text they send, so only the two size rejections keep their upstream status. The model, the beta headers and the tool definitions are ours, so those come back as a 502 instead of as a verdict on the caller. The upstream sentence goes to
details.upstream_messageand Sentry.One thing to know when reading it: the kind is read from the response body rather than the SDK class, because a failure that arrives mid-stream is always a bare
APIStatusErrorcarrying the 200 that opened the stream.Each kind was checked against the live API rather than reasoned about.
job_chatandworkflow_chatstill put the raw SDK string in front of users and still answer 401 when our own key is rejected. The router picks between the three, so which one a user gets depends on a routing choice. That is a follow-up, not this PR.AI Usage
Please disclose whether you've used AI in this work (it's cool, we just want to know!):
You can read more details in our Responsible AI Policy