Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 0 additions & 4 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -37,10 +37,6 @@ TRUSTED_PROXIES=
# HTTPS_PROXY=http://proxy.example.com:8080
# NO_PROXY=localhost,127.0.0.1,.internal.example.com

# Ordinary-channel UA fallback; restart after changes. Empty keeps GPT-Load/<version>.
# Nonempty request-header rules or client UA take precedence; subscription UA is unchanged.
# UPSTREAM_USER_AGENT_FALLBACK=Go-http-client/1.1

# Logging
LOG_LEVEL=info
LOG_FORMAT=text
Expand Down
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -203,6 +203,8 @@ Installing a newer Setup stops the service gracefully before updating it. Window

### Environment configuration

System and group upstream request-header rules support **Set**, **Remove**, and **Default**. Default fills only missing, empty, or whitespace-only headers, preserving nonempty client values; credential probes and model discovery use the same rules. For example, default `User-Agent` to `Go-http-client/1.1`. The API accepts a `header_rules.default` object; a header may appear in only one of `set`, `remove`, or `default`, and a group override still replaces the whole ruleset. Without a custom default, ordinary channels retain the built-in `GPT-Load/<version>` UA.

At startup, the application reads `.env` in the current directory; existing process environment variables take precedence. Unless noted otherwise, changes require restarting the process or container; see [`.env.example`](.env.example) for the common configuration template.

<details>
Expand All @@ -229,7 +231,6 @@ At startup, the application reads `.env` in the current directory; existing proc
| `HTTP_PROXY` | Empty | Environment proxy for HTTP upstream requests. |
| `HTTPS_PROXY` | Empty | Environment proxy for HTTPS upstream requests. |
| `NO_PROXY` | Empty | Comma-separated hosts, domains, or IPs that bypass the environment proxy. |
| `UPSTREAM_USER_AGENT_FALLBACK` | Empty, uses `GPT-Load/<version>` | Fallback User-Agent for ordinary-channel API requests, credential probes, model discovery, and Responses WebSocket handshakes. Used only when the final UA is missing, empty, or whitespace-only after header rules; nonempty rule/client values take precedence. Example: `Go-http-client/1.1` (fixed, not protocol-dependent). Subscription adapter identities are unchanged. Restart after changes. |
| `LOG_LEVEL` | `info` | Supports `panic`, `fatal`, `error`, `warn`, `warning`, `info`, `debug`, and `trace`; invalid values warn and fall back to `info`. |
| `LOG_FORMAT` | `text` | Supports `text` and `json`; any other value fails startup. |
| `MODELS_DEV_AUTO_SYNC_ENABLED` | Unset, initial default `true` | When unset, uses the persisted management UI setting; when set, forces Models.dev auto-sync on or off and makes the same UI option read-only. |
Expand Down
3 changes: 2 additions & 1 deletion README_CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -203,6 +203,8 @@ Windows 普通用户可改为下载 `gpt-load-windows-setup.exe`。双击并确

### 环境配置

系统和分组的上游请求头规则支持“设置”“移除”和“兜底”。兜底仅在请求头缺失、为空或仅含空白时补值,保留客户端的非空值;凭据探测和模型发现同样生效。例如设置 `User-Agent` 的兜底值为 `Go-http-client/1.1`。API 使用 `header_rules.default` 对象,与 `set`、`remove` 一样,同一请求头不能重复配置多个操作;分组覆盖仍替换整套规则。未配置自定义兜底时,普通渠道的内置 UA 仍为 `GPT-Load/<version>`。

应用启动时读取当前目录的 `.env`,已有的进程环境变量优先。除特别说明外,修改后需要重启进程或容器;常用配置模板见 [`.env.example`](.env.example)。

<details>
Expand All @@ -229,7 +231,6 @@ Windows 普通用户可改为下载 `gpt-load-windows-setup.exe`。双击并确
| `HTTP_PROXY` | 空 | HTTP 上游请求的环境代理。 |
| `HTTPS_PROXY` | 空 | HTTPS 上游请求的环境代理。 |
| `NO_PROXY` | 空 | 逗号分隔的不经过环境代理的主机、域名或 IP。 |
| `UPSTREAM_USER_AGENT_FALLBACK` | 空,使用 `GPT-Load/<version>` | 普通渠道 API 请求、凭据探测、模型发现和 Responses WebSocket 握手的兜底 User-Agent。仅在请求头规则应用后最终 UA 缺失、为空或全是空白时生效,不覆盖规则或客户端提供的非空 UA。例如 `Go-http-client/1.1`(固定值,不随 HTTP 协议变化)。不改变订阅适配器身份。修改后重启。 |
| `LOG_LEVEL` | `info` | 支持 `panic`、`fatal`、`error`、`warn`、`warning`、`info`、`debug`、`trace`;无效值会告警并回退到 `info`。 |
| `LOG_FORMAT` | `text` | 支持 `text`、`json`;其他值会导致启动失败。 |
| `MODELS_DEV_AUTO_SYNC_ENABLED` | 未设置,初始默认 `true` | 未设置时使用管理界面的持久化设置;设置后强制开启或关闭 Models.dev 自动同步,并使管理界面中的同名选项变为只读。 |
Expand Down
3 changes: 2 additions & 1 deletion README_JP.md
Original file line number Diff line number Diff line change
Expand Up @@ -203,6 +203,8 @@ Windows の一般ユーザーは代わりに `gpt-load-windows-setup.exe` を利

### 環境設定

システムとグループの上流リクエストヘッダールールは「設定」「削除」「既定値」をサポートします。既定値はヘッダーが未指定、空、または空白のみの場合に適用され、クライアントの空でない値を保持します。認証情報の検証とモデル検出にも適用されます。例えば `User-Agent` の既定値を `Go-http-client/1.1` に設定できます。API は `header_rules.default` オブジェクトを使用し、同一ヘッダーを `set`、`remove`、`default` に重複指定できません。グループ設定は従来どおりルール全体を置き換えます。独自の既定値がない通常チャネルでは、組み込みの `GPT-Load/<version>` UA が維持されます。

アプリケーションは起動時にカレントディレクトリの `.env` を読み込み、既存のプロセス環境変数を優先します。特記がない限り、変更後はプロセスまたはコンテナを再起動してください。一般的な設定テンプレートは [`.env.example`](.env.example) を参照してください。

<details>
Expand All @@ -229,7 +231,6 @@ Windows の一般ユーザーは代わりに `gpt-load-windows-setup.exe` を利
| `HTTP_PROXY` | 空 | HTTP アップストリームリクエストの環境プロキシ。 |
| `HTTPS_PROXY` | 空 | HTTPS アップストリームリクエストの環境プロキシ。 |
| `NO_PROXY` | 空 | 環境プロキシをバイパスするホスト、ドメイン、IP のカンマ区切りリスト。 |
| `UPSTREAM_USER_AGENT_FALLBACK` | 空、`GPT-Load/<version>` を使用 | 通常チャネルの API リクエスト、認証情報テスト、モデル検出、Responses WebSocket ハンドシェイク用のフォールバック User-Agent。ヘッダールール適用後の UA が未指定・空・空白のみの場合に使用し、ルールやクライアントの空でない UA は維持します。例:`Go-http-client/1.1`(HTTP プロトコルに依存しない固定値)。サブスクリプションアダプターの識別情報は変更しません。変更後は再起動が必要です。 |
| `LOG_LEVEL` | `info` | `panic`、`fatal`、`error`、`warn`、`warning`、`info`、`debug`、`trace` に対応します。無効な値は警告を出して `info` に戻ります。 |
| `LOG_FORMAT` | `text` | `text` と `json` に対応します。それ以外の値では起動に失敗します。 |
| `MODELS_DEV_AUTO_SYNC_ENABLED` | 未設定、初期既定値 `true` | 未設定時は管理画面に保存された設定を使用します。設定時は Models.dev の自動同期を強制的に有効または無効にし、管理画面の同名オプションを読み取り専用にします。 |
Expand Down
4 changes: 2 additions & 2 deletions internal/container/container.go
Original file line number Diff line number Diff line change
Expand Up @@ -193,8 +193,8 @@ func BuildContainer(configure ...func(*dig.Container) error) (*dig.Container, er
audioSpeech, mistral,
)
},
func(registry *channel.Registry, cfg *config.Config) (*bifrostexecutor.RuntimeManager, error) {
return bifrostexecutor.NewManagedRuntime(registry, cfg.UpstreamUserAgentFallback)
func(registry *channel.Registry) (*bifrostexecutor.RuntimeManager, error) {
return bifrostexecutor.NewManagedRuntime(registry)
},
func(adapters *provideradapter.Registry, quotaRuntime *accessquota.Runtime, credentials *state.CredentialRegistry) *state.Manager {
manager := state.NewManager()
Expand Down
5 changes: 5 additions & 0 deletions internal/control/discover_executor.go
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,11 @@ func applyControlHeaderRules(rules state.HeaderRules, apiKey string) http.Header
for _, name := range rules.Remove {
headers.Del(name)
}
for name, value := range rules.Default {
if strings.TrimSpace(headers.Get(name)) == "" {
headers.Set(name, strings.ReplaceAll(value, "${API_KEY}", apiKey))
}
}
headers.Set("Accept-Encoding", "identity")
return headers
}
Expand Down
12 changes: 10 additions & 2 deletions internal/control/group_detail.go
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,13 @@ func effectiveGroupConfig(
return GroupEffectiveConfigResponse{}, err
}
set := make(map[string]string, len(resolved.HeaderRules.Set))
var defaults map[string]string
if len(resolved.HeaderRules.Default) > 0 {
defaults = make(map[string]string, len(resolved.HeaderRules.Default))
for name, value := range resolved.HeaderRules.Default {
defaults[name] = value
}
}
for name, value := range resolved.HeaderRules.Set {
set[name] = value
}
Expand All @@ -89,8 +96,9 @@ func effectiveGroupConfig(
RequestTimeout: durationSeconds(resolved.Timeouts.Request),
StreamIdleTimeout: durationSeconds(resolved.Timeouts.StreamIdle),
HeaderRules: HeaderRulesResponse{
Set: set,
Remove: append([]string{}, resolved.HeaderRules.Remove...),
Set: set,
Default: defaults,
Remove: append([]string{}, resolved.HeaderRules.Remove...),
},
BlacklistThreshold: resolved.BlacklistThreshold,
AffinityEnabled: resolved.AffinityEnabled,
Expand Down
43 changes: 43 additions & 0 deletions internal/control/header_defaults_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
package control

import (
"encoding/json"
"testing"

"gpt-load/internal/protocol"
"gpt-load/internal/state"
)

func TestControlHeaderDefaults(t *testing.T) {
rules := state.HeaderRules{Default: map[string]string{"User-Agent": "Go-http-client/1.1", "X-Template": "${API_KEY}"}}
headers := applyControlHeaderRules(rules, "fixture-token")
if headers.Get("User-Agent") != "Go-http-client/1.1" || headers.Get("X-Template") != "fixture-token" {
t.Fatal(headers)
}
}

func TestHeaderDefaultsSettingsRoundTrip(t *testing.T) {
fixture := newServiceFixture(t)
updated, err := fixture.service.UpdateSettings(t.Context(), SettingsUpdateRequest{Settings: map[string]json.RawMessage{
state.SettingHeaderRules: json.RawMessage(`{"default":{"user-agent":"Go-http-client/1.1"}}`),
}})
if err != nil {
t.Fatal(err)
}
if updated.Values.HeaderRules.Default["User-Agent"] != "Go-http-client/1.1" {
t.Fatal(updated.Values.HeaderRules)
}
read, err := fixture.service.GetSettings(t.Context())
if err != nil {
t.Fatal(err)
}
if read.Values.HeaderRules.Default["User-Agent"] != "Go-http-client/1.1" {
t.Fatal(read.Values.HeaderRules)
}
group := validationSignatureGroup()
before := computeGroupValidationSignature(group, protocol.OpenAICompletions, "fixture-model")
group.HeaderRules.Default = map[string]string{"User-Agent": "fallback/1.0"}
if after := computeGroupValidationSignature(group, protocol.OpenAICompletions, "fixture-model"); after == before {
t.Fatal("defaults missing from probe signature")
}
}
17 changes: 13 additions & 4 deletions internal/control/settings.go
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,9 @@ import (
)

type HeaderRulesResponse struct {
Set map[string]string `json:"set"`
Remove []string `json:"remove"`
Set map[string]string `json:"set"`
Remove []string `json:"remove"`
Default map[string]string `json:"default,omitempty"`
}

type CORSConfigResponse struct {
Expand Down Expand Up @@ -395,6 +396,13 @@ func mapSettingsResponse(
set[name] = value
}
remove := append([]string{}, settings.HeaderRules.Remove...)
var defaults map[string]string
if len(settings.HeaderRules.Default) > 0 {
defaults = make(map[string]string, len(settings.HeaderRules.Default))
for name, value := range settings.HeaderRules.Default {
defaults[name] = value
}
}
responseSet := make(map[string]string, len(settings.ResponseHeaderRules.Set))
for name, value := range settings.ResponseHeaderRules.Set {
responseSet[name] = value
Expand Down Expand Up @@ -456,8 +464,9 @@ func mapSettingsResponse(
RequestTimeout: durationSeconds(settings.RequestTimeout),
StreamIdleTimeout: durationSeconds(settings.StreamIdleTimeout),
HeaderRules: HeaderRulesResponse{
Set: set,
Remove: remove,
Set: set,
Default: defaults,
Remove: remove,
},
CORS: CORSConfigResponse{
Enabled: settings.CORS.Enabled,
Expand Down
12 changes: 12 additions & 0 deletions internal/control/validation.go
Original file line number Diff line number Diff line change
Expand Up @@ -445,6 +445,18 @@ func computeGroupValidationSignature(
}

var signature groupValidationSignature
defaultParts := make([]headerSetPart, 0, len(group.HeaderRules.Default))
for name, value := range group.HeaderRules.Default {
defaultParts = append(defaultParts, headerSetPart{name: normalizeValidationHeaderName(name), value: value})
}
sort.Slice(defaultParts, func(i, j int) bool { return defaultParts[i].name < defaultParts[j].name })
if len(defaultParts) > 0 {
writeValidationSignatureUint64(hasher, uint64(len(defaultParts)))
}
for _, part := range defaultParts {
writeValidationSignaturePart(hasher, []byte(part.name))
writeValidationSignaturePart(hasher, []byte(part.value))
}
copy(signature[:], hasher.Sum(nil))
return signature
}
Expand Down
4 changes: 2 additions & 2 deletions internal/execution/bifrost/executor.go
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ type streamSDKResult struct {

// Execute executes one non-streaming attempt.
func (r *Runtime) Execute(parent context.Context, spec execution.AttemptSpec) (result execution.AttemptResult) {
spec = withUserAgent(spec, r.userAgentFallback)
spec = withUserAgent(spec)
defer func() {
normalizeImagesAttemptResult(spec, &result)
normalizeEmbeddingsAttemptResult(spec, &result)
Expand Down Expand Up @@ -197,7 +197,7 @@ func (r *Runtime) ExecuteStream(
spec execution.AttemptSpec,
sink execution.StreamSink,
) (result execution.StreamResult) {
spec = withUserAgent(spec, r.userAgentFallback)
spec = withUserAgent(spec)
defer func() {
normalizeImagesStreamResult(spec, &result)
}()
Expand Down
7 changes: 1 addition & 6 deletions internal/execution/bifrost/newapi_runtime_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -505,11 +505,6 @@ func TestNewAPINativeStreamsKeepProtocolWireAndUsage(t *testing.T) {

func newNewAPIManagerForTest(t *testing.T, baseURL string) (*RuntimeManager, channel.ResolvedTarget) {
t.Helper()
return newNewAPIManagerWithOptionsForTest(t, baseURL, runtimeOptions{allowPrivateNetwork: true})
}

func newNewAPIManagerWithOptionsForTest(t *testing.T, baseURL string, options runtimeOptions) (*RuntimeManager, channel.ResolvedTarget) {
t.Helper()

registry := channel.NewRegistry()
resolved, err := registry.Resolve(
Expand All @@ -519,7 +514,7 @@ func newNewAPIManagerWithOptionsForTest(t *testing.T, baseURL string, options ru
if err != nil {
t.Fatal(err)
}
manager, err := newRuntimeManager(options, registry)
manager, err := newRuntimeManager(runtimeOptions{allowPrivateNetwork: true}, registry)
if err != nil {
t.Fatal(err)
}
Expand Down
7 changes: 2 additions & 5 deletions internal/execution/bifrost/runtime.go
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,6 @@ var errKeyPoolDisabled = errors.New("Bifrost key pool is disabled; DirectKey is

// Runtime owns one long-lived Bifrost Core and executes attempts with explicit credentials.
type Runtime struct {
userAgentFallback string
core *core.Bifrost
account *directAccount
registry *channel.Registry
Expand Down Expand Up @@ -72,7 +71,6 @@ func newConfiguredRuntime(
}

type runtimeOptions struct {
userAgentFallback string
allowPrivateNetwork bool
maxUnaryResponseBodyBytes int64
logger schemas.Logger
Expand Down Expand Up @@ -100,8 +98,8 @@ func NewRuntime(ctx context.Context, registry *channel.Registry) (*RuntimeManage

// NewManagedRuntime creates an unstarted production runtime manager. The
// application lifecycle must call Start before accepting requests.
func NewManagedRuntime(registry *channel.Registry, userAgentFallback string) (*RuntimeManager, error) {
return newRuntimeManager(runtimeOptions{allowPrivateNetwork: true, userAgentFallback: userAgentFallback}, registry)
func NewManagedRuntime(registry *channel.Registry) (*RuntimeManager, error) {
return newRuntimeManager(runtimeOptions{allowPrivateNetwork: true}, registry)
}

func newRuntimeShell(options runtimeOptions, registry *channel.Registry) (*Runtime, error) {
Expand All @@ -110,7 +108,6 @@ func newRuntimeShell(options runtimeOptions, registry *channel.Registry) (*Runti
return nil, fmt.Errorf("initialize execution runtime: channel registry is required")
}
return &Runtime{
userAgentFallback: options.userAgentFallback,
account: newDirectAccount(),
registry: registry,
allowPrivate: options.allowPrivateNetwork,
Expand Down
2 changes: 0 additions & 2 deletions internal/execution/bifrost/test_runtime_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@ import (
)

type testRuntimeOptions struct {
userAgentFallback string
allowPrivateNetwork bool
openAIBaseURL string
anthropicBaseURL string
Expand All @@ -28,7 +27,6 @@ func newRuntimeForTest(t *testing.T, options testRuntimeOptions) *testRuntime {
t.Helper()
registry := channel.NewRegistry()
manager, err := newRuntimeManager(runtimeOptions{
userAgentFallback: options.userAgentFallback,
allowPrivateNetwork: options.allowPrivateNetwork,
maxUnaryResponseBodyBytes: options.maxUnaryResponseBodyBytes,
}, registry)
Expand Down
9 changes: 3 additions & 6 deletions internal/execution/bifrost/user_agent.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import (
)

// withUserAgent 仅为普通渠道补齐出站身份;订阅渠道由各自适配器处理。
func withUserAgent(spec execution.AttemptSpec, fallback string) execution.AttemptSpec {
func withUserAgent(spec execution.AttemptSpec) execution.AttemptSpec {
// 系统或分组规则已先应用,保留其配置值或客户端原始身份。
if strings.TrimSpace(spec.Header.Get("User-Agent")) != "" {
return spec
Expand All @@ -18,10 +18,7 @@ func withUserAgent(spec execution.AttemptSpec, fallback string) execution.Attemp
if spec.Header == nil {
spec.Header = make(http.Header)
}
// 环境配置仅影响兜底;未配置时保留项目身份,包括规则明确删除或留空的情况。
if fallback == "" {
fallback = "GPT-Load/" + version.Version
}
spec.Header.Set("User-Agent", fallback)
// 没有有效 UA 时使用项目身份,包括规则明确删除或留空的情况。
spec.Header.Set("User-Agent", "GPT-Load/"+version.Version)
return spec
}
Loading