Add model_discovery backend option to disable /v1/models probing - #59
Merged
Conversation
Allow backends to opt out of model discovery via a new model_discovery config flag (defaults to true). Backends such as GCP Vertex AI do not expose an OpenAI-compatible /v1/models endpoint, so the Models TUI tab now marks them as disabled instead of surfacing an error. Also report upstream HTTP failures with the standard status text instead of a truncated response body.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a per-backend
model_discoveryconfig option (defaults totrue) so backends that do not expose an OpenAI-compatible/v1/modelsendpoint — such as GCP Vertex AI — can opt out of model discovery.Changes
model_discoveryfield on backends with aModelDiscoveryEnabled()helper (nil/absent = enabled)DiscoverModelsskips disabled backends (no upstream request) and marks the resultSkipped; upstream HTTP failures now report the standard status text instead of a truncated response body