From d8fbc5fac961d58d0dcd03add18b0b6080d759f8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Patrick=20Brand=C3=A3o?= Date: Sat, 7 Nov 2026 18:06:58 -0300 Subject: [PATCH 1/2] =?UTF-8?q?docs(roteamento):=20ID=20concreto=20no=20Cl?= =?UTF-8?q?aude=20Code=20=C3=A9=20rota=20expl=C3=ADcita,=20sem=20fallback;?= =?UTF-8?q?=20virtual=20com=20o=20mesmo=20nome=20devolve=20a=20cascata?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Stress run 2026-11-07: Claude Code harnesses got the local cooldown 429 while Anthropic SDK harnesses on the same chain fell back. Cause: the SDK load sent model=auto, the Claude Code load a concrete ID, and a concrete ID resolves to an explicit route with no fallbacks (intended, ADR 0004). New spec proves the three cases (auto, concrete ID, shadowing virtual model) with Claude Code headers against a cooling-down primary. Co-Authored-By: Claude Opus 5.5 --- .../tutoriais/thinker-with-white-wizard-02.md | 13 +- docs/usuario/conectar-harnesses.md | 2 + docs/usuario/provedores-e-roteamento.md | 4 + ...proxy.service.claude-code-fallback.spec.ts | 225 ++++++++++++++++++ 4 files changed, 243 insertions(+), 1 deletion(-) create mode 100644 packages/backend/src/routing/proxy/__tests__/proxy.service.claude-code-fallback.spec.ts diff --git a/docs/tutoriais/thinker-with-white-wizard-02.md b/docs/tutoriais/thinker-with-white-wizard-02.md index 12d2bc3a..7e11699c 100644 --- a/docs/tutoriais/thinker-with-white-wizard-02.md +++ b/docs/tutoriais/thinker-with-white-wizard-02.md @@ -451,6 +451,16 @@ h-claude-code-1 anthropic claude-sonnet-5-5 h-claude-code-2 anthropic claude-haiku-5-5 EOF +# 4b - o gerador do Claude Code manda um ID concreto (claude-sonnet-5-5), e ID +# concreto e rota explicita, sem fallback. Um modelo virtual com esse nome, sem +# principal e com Continue on Default, herda a cadeia do tier default +# (falhas -> instavel -> perfeito) +for h in h-claude-code-1 h-claude-code-2; do + jq -n '{name: "claude-sonnet-5-5", continue_on_default: true}' \ + | api POST "/routing/$h/virtual-models" >/dev/null + echo "virtual $h claude-sonnet-5-5 -> cadeia do default"; +done + # 5 - chaves dos harnesses num arquivo 600 (lido pelo loop) umask 077 : > harnesses.keys @@ -788,6 +798,8 @@ Construir esse ambiente revelou três bugs no Thinker, já corrigidos: - **0.31.5**: com base URL própria, o OpenCode recebia o modelo **com o prefixo do catálogo** (`opencode-go/glm-5.3`) e respondia `Model is unavailable`; - **0.31.6**: a conexão Anthropic com base URL própria não recebia o **`anthropic-beta`** do cliente, e todo turno do Claude Code voltava 400 (`messages.1.output_config: Extra inputs are not permitted`). +E um achado que **não é bug**: com o principal em cooldown, os harnesses Claude Code recebiam o 429 local (`Provider route temporarily cooling down after an upstream 429`) e nenhum fallback, enquanto o Anthropic SDK, com a mesma rota, descia a cascata. A diferença era o `model` do corpo: o gerador do SDK manda `auto`, o do Claude Code manda `claude-sonnet-5-5`, e um ID concreto é uma rota explícita, **sem fallback** (veja [`model` explícito](../usuario/provedores-e-roteamento.md#model-explícito)). Os cabeçalhos do Claude Code e a afinidade de sessão não entram nisso. O passo 4b cria um modelo virtual com o nome do ID, que devolve a cascata; o teste `proxy.service.claude-code-fallback.spec.ts` prova os três casos (`auto`, ID concreto e virtual). + E as lacunas que continuam abertas (contorne como mostrado): - **Uma réplica = um núcleo**: ~125 req/s por processo; escala horizontal só com réplicas, e os guardrails `THINKER_*_MAX` valem por processo; @@ -798,7 +810,6 @@ E as lacunas que continuam abertas (contorne como mostrado): - **Sem Azure nativo**: provedor custom OpenAI; - **`provider connect` sem `--base-url`**: a primeira validação vai ao provedor real; - **Headers `x-wwz-*` não passam**: o comportamento do simulador fica todo no YAML; -- **Cooldown sem cascata no Claude Code** (aberta, sem correção): com o principal em cooldown, os harnesses Claude Code receberam o 429 local (`Provider route temporarily cooling down after an upstream 429`) em vez de seguir para os fallbacks, enquanto o Anthropic SDK, com a mesma rota, desceu a cascata. Isolar se é a plataforma, a forma da requisição ou a janela de indisponibilidade da **instavel** fica para uma investigação própria. ## 11 – Limpeza diff --git a/docs/usuario/conectar-harnesses.md b/docs/usuario/conectar-harnesses.md index e37b8252..f610f32a 100644 --- a/docs/usuario/conectar-harnesses.md +++ b/docs/usuario/conectar-harnesses.md @@ -33,6 +33,8 @@ A URL base **não** leva `/v1`: o cliente Anthropic acrescenta `/v1/messages`. D Por um gateway, o Claude Code assume contexto de 200k e compacta sessões longas cedo. Para trabalho com contexto longo, fixe um modelo de 1M em vez de `auto` (por exemplo `/model claude-opus-5-5[1m]`). +Um modelo fixado (no `/model`, em `ANTHROPIC_MODEL` ou nos subagentes, que pedem `claude-haiku-*`) é uma rota explícita e **não tem fallback**. Para manter a cascata num nome fixo, crie um modelo virtual com esse nome (veja [`model` explícito](provedores-e-roteamento.md#model-explícito)). + ## Codex O Codex usa a API Responses. Em `~/.codex/config.toml`: diff --git a/docs/usuario/provedores-e-roteamento.md b/docs/usuario/provedores-e-roteamento.md index 07f30836..8b8c4c51 100644 --- a/docs/usuario/provedores-e-roteamento.md +++ b/docs/usuario/provedores-e-roteamento.md @@ -73,6 +73,10 @@ Ligada, cada requisição é pontuada pelo conteúdo (palavras-chave, tamanho, f `auto` pede para o Thinker rotear. Qualquer outro valor é uma escolha explícita: o Thinker procura o modelo entre os descobertos do harness (use os IDs de `GET /v1/models`). Um ID qualificado pelo provedor (`openai/gpt-novo`) ainda pode ser roteado por esse provedor se a credencial dele estiver habilitada no harness, mesmo fora do catálogo. Se o modelo não está disponível para o harness, a resposta é `M302`. +Uma rota explícita **não tem fallback**: é só aquele modelo, na conexão fixada pelo tier (o rótulo da chave vem do tier, os fallbacks não). Se ela falha, ou está em espera depois de um 429 (`Provider route temporarily cooling down after an upstream 429`), o cliente recebe esse erro direto, sem cabeçalho `X-Thinker-Fallback-*`. A afinidade de sessão e os cabeçalhos do cliente não mudam isso: o que decide é o `model` do corpo. + +Isso pega o **Claude Code**, que manda sempre um ID concreto (`claude-sonnet-5-5`, `claude-haiku-5-5` para subagentes e tarefas leves) quando não está configurado com `auto`. Para dar cascata a esses nomes sem mexer no cliente, crie um [modelo virtual](#modelos-virtuais) com o **mesmo nome** do ID: o virtual vence o ID explícito e roteia pela própria cadeia (ou, com Continue on Default, pela do tier `default`). + ## Fallback Quando uma tentativa ao provedor falha (resposta HTTP 4xx ou 5xx, timeout, resposta em streaming que não começa dentro de `STREAM_WARMUP_MS`, silêncio maior que `STREAM_IDLE_TIMEOUT_MS`, resposta 200 sem conteúdo e com zero tokens, resposta 200 que só traz um erro, erro no início do streaming antes de qualquer conteúdo), o Thinker tenta o próximo modelo de fallback do tier, na ordem configurada. A primeira tentativa que der certo atende a requisição; o cliente vê um único resultado. diff --git a/packages/backend/src/routing/proxy/__tests__/proxy.service.claude-code-fallback.spec.ts b/packages/backend/src/routing/proxy/__tests__/proxy.service.claude-code-fallback.spec.ts new file mode 100644 index 00000000..e5c798c7 --- /dev/null +++ b/packages/backend/src/routing/proxy/__tests__/proxy.service.claude-code-fallback.spec.ts @@ -0,0 +1,225 @@ +import { ConfigService } from '@nestjs/config'; +import type { ModelRoute } from 'thinker-shared'; +import { ProxyService } from '../proxy.service'; +import type { ResolveService } from '../../resolve/resolve.service'; +import type { ProviderKeyService } from '../../routing-core/provider-key.service'; +import type { ProxyFallbackService } from '../proxy-fallback.service'; +import type { AgentModelParamsService } from '../../routing-core/agent-model-params.service'; +import type { ProviderParamSpecService } from '../../routing-core/provider-param-spec.service'; +import type { ModelDiscoveryService } from '../../../model-discovery/model-discovery.service'; +import type { DiscoveredModel } from '../../../model-discovery/model-fetcher'; + +/** + * White Wizard stress finding (2026-11-07): Claude Code harnesses got the + * local cooldown 429 while Anthropic SDK harnesses on the same chain fell + * back. The difference is the body's `model`, not the Claude Code headers: + * the SDK load sent `auto`, the Claude Code load a concrete ID, and a + * concrete ID is an explicit route without fallbacks. A virtual model with + * that same name gives the ID a chain. + */ +describe('ProxyService — Claude Code Messages request vs a cooling-down primary', () => { + const primary: ModelRoute = { + provider: 'anthropic', + authType: 'api_key', + model: 'claude-sonnet-5-5', + keyLabel: 'falhas', + }; + const fallbacks: ModelRoute[] = [ + { ...primary, keyLabel: 'instavel' }, + { ...primary, keyLabel: 'perfeito' }, + ]; + const chain = (reason: string) => ({ + tier: 'default' as const, + route: primary, + fallback_routes: fallbacks, + response_mode: 'buffered' as const, + confidence: 1, + score: 0, + reason, + }); + const cooldown429 = () => ({ + response: new Response( + JSON.stringify({ + error: { + message: + 'Provider route temporarily cooling down after an upstream 429: ' + + 'anthropic/claude-sonnet-5-5', + }, + }), + { status: 429, headers: { 'content-type': 'application/json', 'retry-after': '15' } }, + ), + isGoogle: false, + isAnthropic: false, + isChatGpt: false, + providerCallStarted: false, + }); + const claudeCodeHeaders = { + 'x-claude-code-session-id': 'cc-session-1', + 'anthropic-beta': 'claude-code-20250219,interleaved-thinking-2025-05-14', + 'anthropic-version': '2023-06-01', + 'user-agent': 'claude-cli/2.1.200 (external, cli)', + }; + + let resolveService: { + resolveLazy: jest.Mock; + resolveForTier: jest.Mock; + resolveVirtualModel: jest.Mock; + pinRouteKeyLabel: jest.Mock; + }; + let fallbackService: { tryForwardToProvider: jest.Mock; tryFallbacks: jest.Mock }; + let modelDiscovery: { getModelsForAgent: jest.Mock }; + let svc: ProxyService; + + beforeEach(() => { + resolveService = { + resolveLazy: jest.fn().mockResolvedValue(chain('scored')), + resolveForTier: jest.fn(), + resolveVirtualModel: jest.fn().mockResolvedValue(null), + // The tier's connection pin survives an explicit model (see explicitRouting). + pinRouteKeyLabel: jest.fn(async (_a, _t, route: ModelRoute) => ({ + ...route, + keyLabel: 'falhas', + })), + }; + fallbackService = { + tryForwardToProvider: jest.fn().mockResolvedValue(cooldown429()), + tryFallbacks: jest.fn().mockResolvedValue({ + success: { + forward: { + response: new Response('{"ok":true}', { status: 200 }), + isGoogle: false, + isAnthropic: true, + isChatGpt: false, + }, + model: 'claude-sonnet-5-5', + provider: 'anthropic', + authType: 'api_key', + keyLabel: 'instavel', + fallbackIndex: 0, + }, + failures: [], + }), + }; + const discovered: DiscoveredModel = { + id: 'claude-sonnet-5-5', + displayName: 'Claude Sonnet 5.5', + provider: 'anthropic', + contextWindow: 200000, + inputPricePerToken: null, + outputPricePerToken: null, + capabilityReasoning: true, + capabilityCode: true, + qualityScore: 5, + authType: 'api_key', + }; + modelDiscovery = { getModelsForAgent: jest.fn().mockResolvedValue([discovered]) }; + const providerKeyService = { + getProviderApiKey: jest.fn().mockResolvedValue('decrypted-key'), + getProviderRegion: jest.fn().mockResolvedValue(null), + getProviderKeyId: jest.fn().mockResolvedValue('up-falhas'), + selectProviderKey: jest.fn().mockResolvedValue({ + apiKey: 'decrypted-key', + id: 'up-falhas', + region: null, + label: 'falhas', + priority: 0, + }), + hasRouteCredentials: jest.fn().mockResolvedValue(true), + }; + const oauth = { unwrapToken: jest.fn().mockResolvedValue(null) }; + svc = new ProxyService( + resolveService as unknown as ResolveService, + modelDiscovery as unknown as ModelDiscoveryService, + providerKeyService as unknown as ProviderKeyService, + oauth as never, + oauth as never, + oauth as never, + oauth as never, + oauth as never, + oauth as never, + { + recordTier: jest.fn(), + recordCategory: jest.fn(), + getRecentTiers: jest.fn().mockReturnValue([]), + getRecentCategories: jest.fn().mockReturnValue([]), + } as never, + { checkLimits: jest.fn().mockResolvedValue(null) } as never, + fallbackService as unknown as ProxyFallbackService, + { get: jest.fn().mockReturnValue(undefined) } as unknown as ConfigService, + { retrieve: jest.fn().mockReturnValue(null) } as never, + { retrieve: jest.fn().mockReturnValue(null) } as never, + { + get: jest.fn().mockResolvedValue(null), + list: jest.fn().mockResolvedValue([]), + } as unknown as AgentModelParamsService, + { + getSpecs: jest.fn().mockResolvedValue([]), + list: jest.fn().mockResolvedValue([]), + } as unknown as ProviderParamSpecService, + ); + }); + + const claudeCodeRequest = (model: string) => + svc.proxyRequest({ + agentId: 'agent-cc', + userId: 'user-1', + tenantId: 'tenant-1', + agentName: 'h-claude-code-1', + apiMode: 'messages', + stream: false, + headers: claudeCodeHeaders, + sessionKey: 'cc-session-1', + sessionCacheKey: 'cc-session-1', + providerCacheKey: 'cc-session-1', + sessionMomentumKey: 'cc-session-1', + body: { + model, + max_tokens: 1024, + stream: false, + messages: [{ role: 'user', content: 'hi' }], + }, + } as Parameters[0]); + + it('walks both fallbacks when the body names auto, Claude Code headers and all', async () => { + const result = await claudeCodeRequest('auto'); + + expect(fallbackService.tryFallbacks).toHaveBeenCalledTimes(1); + const call = fallbackService.tryFallbacks.mock.calls[0]; + const [models, routes] = [call[2], call[14]]; + expect(models).toEqual(['claude-sonnet-5-5', 'claude-sonnet-5-5']); + expect(routes).toEqual(fallbacks); + expect(result.forward.response.status).toBe(200); + expect(result.meta).toMatchObject({ fallbackIndex: 0, provider_key_label: 'instavel' }); + }); + + it('walks the chain of a virtual model that shadows the concrete ID', async () => { + resolveService.resolveVirtualModel.mockResolvedValue({ + ...chain('virtual-model'), + virtual_model_id: 'vm-sonnet', + virtual_model_name: 'claude-sonnet-5-5', + }); + + const result = await claudeCodeRequest('claude-sonnet-5-5'); + + expect(fallbackService.tryFallbacks).toHaveBeenCalledTimes(1); + expect(result.forward.response.status).toBe(200); + expect(result.meta.fallbackIndex).toBe(0); + }); + + it('returns the cooldown 429 for a concrete ID: an explicit route has no fallbacks', async () => { + const result = await claudeCodeRequest('claude-sonnet-5-5'); + + expect(resolveService.resolveLazy).not.toHaveBeenCalled(); + expect(fallbackService.tryForwardToProvider).toHaveBeenCalledWith( + expect.objectContaining({ + provider: 'anthropic', + model: 'claude-sonnet-5-5', + hasNextRoute: false, + }), + ); + expect(fallbackService.tryFallbacks).not.toHaveBeenCalled(); + expect(result.forward.response.status).toBe(429); + expect(await result.forward.response.text()).toContain('temporarily cooling down'); + expect(result.meta).toMatchObject({ tier: 'direct', reason: 'direct' }); + }); +}); From f30aef05b4273fd4a16293e2177a3ddce73583e9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Patrick=20Brand=C3=A3o?= Date: Sat, 7 Nov 2026 18:07:12 -0300 Subject: [PATCH 2/2] chore(release): version 0.31.8 --- docs/releases/0.31.8.md | 9 +++++++++ package-lock.json | 2 +- packages/thinker/package.json | 2 +- 3 files changed, 11 insertions(+), 2 deletions(-) create mode 100644 docs/releases/0.31.8.md diff --git a/docs/releases/0.31.8.md b/docs/releases/0.31.8.md new file mode 100644 index 00000000..b13914bf --- /dev/null +++ b/docs/releases/0.31.8.md @@ -0,0 +1,9 @@ +# Thinker 0.31.8 + +```bash +docker pull tmsoftbrasil/thinker:0.31.8 +``` + +## Mudanças + +- docs(roteamento): ID concreto no Claude Code é rota explícita, sem fallback; virtual com o mesmo nome devolve a cascata diff --git a/package-lock.json b/package-lock.json index 3b1cb50f..08bc3662 100644 --- a/package-lock.json +++ b/package-lock.json @@ -14768,7 +14768,7 @@ } }, "packages/thinker": { - "version": "0.31.7" + "version": "0.31.8" } } } diff --git a/packages/thinker/package.json b/packages/thinker/package.json index 54fafce7..bf9fbd25 100644 --- a/packages/thinker/package.json +++ b/packages/thinker/package.json @@ -1,6 +1,6 @@ { "name": "thinker", - "version": "0.31.7", + "version": "0.31.8", "private": true, "description": "Canonical release version for the Thinker Docker image. This package holds no code — the real backend and frontend live in ../backend and ../frontend. See README.md for details." }