fix(agent): protocol-shape, image-delivery, quota-status and residue defects on both compatibility surfaces - #167
Closed
maxff77 wants to merge 51 commits into
Closed
fix(agent): protocol-shape, image-delivery, quota-status and residue defects on both compatibility surfaces#167maxff77 wants to merge 51 commits into
maxff77 wants to merge 51 commits into
Conversation
…text 500
Two independent defects kept Qwen from ever seeing an image sent by an agent
client, even though the model itself handles images fine.
1. Claude Code delivers images only inside Anthropic `tool_result` blocks
(verified by capturing a real session). `flattenAnthropicMessages` filtered
that block's content with `.filter(b => b?.type === 'text')`, destroying the
image before any upload was attempted. Images now ride a `media`
side-channel on the tool message — they cannot travel in `content`, which
must stay a string for `foldToolMessages` — and are re-attached to the
current turn after folding. `resultContent` is unchanged, byte for byte.
2. The upstream returned HTTP 500 whenever an image sat in `content[]` while
`files[]` also carried the externalized agent-context `.txt`. Probes
isolated it as a shape problem, not a size one: 107 KiB without an image
and 61 KiB with one both returned 200, while 108 KiB with one returned 500.
Images now travel in `files[]` as `{type:'image', url}` — the only entry
shape this repo has proven upstream, used by the `image_edit` path.
The `files[]` split is gated on chat_type: `t2i`, `t2v` and `image_edit` reach
a controller that reads `messages[0].content` directly and would otherwise
degrade to text-to-image, silently dropping the user's input image.
Only images are re-routed. Video has no upstream-proven `files[]` shape, so it
stays in `content[]` and behaves exactly as before.
Verified against the live upstream: the tool_result shape and the >92160-byte
image request both return 200 and the model reads the image; text-only bodies
are byte-identical to before. 623 unit tests pass, and each of the guards is
pinned by a test that fails when the guard is removed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude Code sends a pasted image as [{text},{image}] and then appends a
text-only meta message ("[Image: source: <local cache path>]"), so the
image is never the last message. parserMessages' multi-message branch
only uploads media from lastMessage, and extractTextFromContent erases it
from every earlier one, so the image died with no upload attempt and no
log. Proven from the real transcript plus staging logs: zero image/jpeg
uploads in the window, and the model's own reasoning complained it could
not reach a file path -- text that exists only in the meta message.
Extend the current-turn backward scan in buildInternalRequest, which
already collects the tool_result media side-channel, to also pull
image_url items out of the content[] of non-last messages in the turn and
strip them from the carrier. The existing re-attach then hands them to
parserMessages for upload. The turn boundary is unchanged, so images from
earlier turns stay unattached.
Also fixes the same defect's other trigger: an image block placed before
the prompt, which flatten emits as its own message.
Verified against a local instance (qwen3.8-max, 446-byte magenta PNG):
paste shape returns "Magenta", an image-only carrier is described, an
earlier-turn paste is still not re-attached, and both OpenAI-path shapes
already worked. Suite: 628 pass / 0 fail.
…nnot drop the image parserMessages only uploads media from the LAST message (chat-helpers.js:396); every earlier message goes through formatHistoryMessages -> extractTextFromContent, which drops non-text items with no log. So an image that is not last simply disappears — no upload attempt, no warning. Real clients put it exactly there. Captured live on 2026-09-08 with a transparent proxy in front of /v1/chat/completions (OpenClaw -> Qwen2API), the agent request that produced the answer looked like this: 1:user:text+image_url+image_url the user's screenshot ... 7:user:text+image_url "Attached image(s) from tool result:" 8:user:text OPENCLAW_INTERNAL_CONTEXT, text only The last message is text, so nothing was uploaded: 3/3 agent requests in that capture uploaded zero images, while a sibling request in the same minute whose last message WAS the image uploaded fine. The model then answered from whatever image was still in its context, which is what the user saw as a hallucination. Fix mirrors anthropic.js#buildInternalRequest, which already solved the same defect for Claude Code's paste shape: scan back to the turn boundary, lift the media items off the non-last carriers, and re-attach them to the last message after foldToolMessages so parserMessages uploads them. Scope guards kept from that twin: only the current turn is harvested (history images are still not re-attached), the last message is never touched, and a media-free request produces a byte-identical upstream body. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…t message
The first cut of the harvest broke on any assistant message. In a tool loop the
assistant speaks several times inside ONE user turn — every intermediate step
carries tool_calls — so from the second step onward the image fell back out of
the scan window and the follow-up request went out without it.
Observed live on 2026-09-08 18:54. The model had clearly seen the screenshot:
it called web_search with the exact video titles visible in it
("midudev GPT-6 es Salvaje", "GPT-6 Astra vs Fable 5.1"). That call was rejected
by schema validation (it sent `queries`, the schema wants `query`), which added a
second assistant step. The next request harvested nothing, and the final answer
was invented from data sitting in the ~39 KB system prompt.
Boundary is now the last assistant message WITHOUT tool_calls, i.e. a real final
answer. Intermediate tool-call steps are inside the turn.
Also dedupe harvested media by URL, seeded from the media already on the last
message: one image legitimately appears several times in a turn (the user's
message and OpenClaw's "Attached image(s) from tool result:" carrier), and
without this it would be sent upstream twice. Carriers are still stripped even
when they contribute nothing, so no base64 leaks into the externalized context.
anthropic.js#buildInternalRequest has the same any-assistant boundary and is
likely wrong the same way for multi-step Claude Code tool loops — untested,
left alone deliberately.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…e turn buildInternalRequest's backward scan broke on ANY assistant message. Inside one user turn the assistant speaks once per tool step, so the scan only ever saw messages after the last tool step: a user-pasted image died at the FIRST tool call, and a tool_result image died from the second assistant turn onward. Measured against the real upstream before this change (/v1/messages, qwen3.8-max, 446-byte magenta PNG, upload delta counted per request): image last, no tools uploads=1 model answered "magenta" image + 1 tool round-trip uploads=0 model answered "no image was provided" image + 2 tool round-trips uploads=0 same One tool call was enough, and Claude Code calls tools constantly — so the paste fix in a783f92 only ever covered turns where the model answered directly. Boundary is now the last assistant WITHOUT tool_calls (a real final answer); intermediate tool steps are inside the turn. This is the same rule 5019f04 applied to the OpenAI twin in chat-helpers.js#harvestCurrentTurnMedia. Dedupe by media URL ships in the same commit, never after it: widening the window makes "user pastes an image, then Read reads the same file" reach both the content[] copy and the tool_result media side-channel, which without dedupe becomes two files[] entries — two uploads and the image twice in the prompt. The seed is taken only from the last flat message's content[], never from its .media: in a normal Read turn that last message IS the tool message carrying the image, and seeding from it would suppress the only copy. The .media branch deliberately keeps no lastFlatIndex guard. The last tool message's content is a string, so parserMessages can pick up nothing from it; the single-step Read turn passes only because of that asymmetry. Tests: the four tool-loop cases fail on the previous code and pass on this one; the dedupe case fails if the boundary is widened without it. Regression guards cover same-turn parallel tool_use, the previous-turn boundary, two parallel Reads of different images, and the media side-channel never reaching the body. 653 tests, 0 fail (run with --test-concurrency=1; the parallel runner silently drops whole files and reports a lower count). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…cap the harvest Four defects found by an adversarial audit of e3bd245/5019f04, each reproduced by a runnable test before being accepted. 1. Dedupe seeded from the PRE-fold last message. foldToolMessages replaces a role=tool array body with a stringified one, so the copy that seeded the set was destroyed moments later while the surviving copy had already been suppressed. Harvest no longer dedupes; attachMediaToLastMessage does, seeded from the post-fold last message. It also registers as it filters, so two carriers of the same image collapse to one (the audit's version filtered only against the initial seed and would have let both through). 2. attachMediaToLastMessage silently no-oped when the last message's content was neither string nor array. {role:'assistant', content:null, tool_calls:[...]} is the canonical OpenAI shape and arrives verbatim whenever nothing folds (tool_choice:'none', non-t2t chat types, no tools) — the harvest had already stripped the image off its carrier, so it was dropped. Terminal else added. Same fix rescues a last role=tool message with array content: it is now inside the harvest window (willBeFolded), because folding would otherwise JSON stringify a few hundred KB of base64 into the [TOOL RESULT] text block with files[] left empty. 3. The harvest ran for every chat_type. For t2i/t2v it turned a plain-string prompt into an array, which breaks '@16:9' size sniffing and pays for an upload the image controller never reads. Now an allowlist (t2t, search, image_edit) rather than a denylist: routes/chat.js sends deep_research and every unknown type to the same controller, so a denylist would silently admit anything added later. image_edit stays in — the harvest is what puts its input image into files[]. 4. The scan was unbounded: with no final-answer assistant anywhere in the array the whole history was harvested and every past image re-uploaded. Capped at 4 items (not messages — a legitimate turn spans many messages) in both twins. Honest framing: the shape that needs this is not in any captured traffic. Also corrects two comments that justified the strip by claiming it keeps base64 out of the externalized context document. It never could: getMessageTextContent and extractTextFromContent are text-only, so media on a history carrier is invisible to that document. 5019f04's commit message repeated the same false claim. The real base64-as-prose leak is foldToolMessages, fixed by (2). 659 tests, 0 fail (--test-concurrency=1). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ting a tool call
Two silent-data-loss defects, each reproduced before being accepted.
flattenAnthropicMessages had no final else in its user-block dispatch. Any block
that is not text/image/tool_result — document (PDF), search_result,
server_tool_use — vanished, and the model answered using only the sentence
wrapped around it. An `image` block whose source shape we cannot forward
(source:{type:'file',file_id}, a documented Anthropic feature) hit the same
silent path via a bare `if (imageItem)`.
Worse, when EVERY block of a user message was unhandled the message itself
disappeared from the flattened array. With history that makes parserMessages
treat the previous ASSISTANT message as "# Current message" — the model answers
its own last reply. Alone it throws inside parserMessages, the throw is
swallowed, and the upstream prompt becomes the literal string
'聊天历史处理有误…'.
Unhandled blocks now leave a visible breadcrumb in the text and are collected
into one WARN per request. A user message that produced no output at all keeps
its slot as an empty user message. Deviation from the audit's recommendation: it
proposed a 400 for that case, but `content: []` is spec-legal, so preserving the
slot fixes both failure modes without changing the API contract. thinking and
redacted_thinking stay silently dropped — they carry no user intent and cannot
be replayed to Qwen.
consumeTrailingCloser only recognised a bare closer when the keyword was written
in full, so a stream that died mid-closer (`[END TOOL C` + EOF) released the
fragment as prose. That is not cosmetic: the leaked "\n[END " makes the next
[TOOL CALL] fail the "trigger must be the first content" gate, so a real tool
call is silently dropped. It now consumes a dangling closer prefix at end of
stream, reusing isDanglingCloserPrefix, which only accepts canonical spellings
filling the rest of the line — a lone '[' stays prose because it is genuinely
ambiguous, and prose that merely looks closer-ish ('[END]', '[NOTE] done') is
untouched. Not applied to consumeMandatoryBracketCloser: that gate is what stops
a quoted payload from synthesizing a call.
669 tests, 0 fail (--test-concurrency=1; verify the count matches expectations —
the runner under-reports silently, including in serial mode).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…n it was cut Two independent problems in how the live prompt is built after the context is externalized as a Qwen document. buildRecentAgentHistory hardcoded `entries.slice(length - 4, -1)` plus the latest entry: exactly five rounds, whatever the budget said. The weighted allocation above it was therefore decorative for the one section that can actually absorb history. Measured on a 60-round, 44 KB envelope with a 49152-byte cap: before 4559 bytes delivered 9.3% of the cap 5 of 60 rounds kept after 46017 bytes delivered 93.6% of the cap 60 of 60 rounds kept It now fills the budget backwards from the newest round, keeping whole JSONL lines — a half-truncated line is unparseable and more misleading than absent. When even the newest round overflows the cap it falls back to the old head-tail truncation, so the current round is always represented, and a compaction marker is emitted whenever anything was dropped so the model knows it is not seeing everything. The allocator itself also became two-pass: `remainingBytes -= budget` subtracted the ALLOCATED quota rather than the amount used, so a small section swallowed its own unused headroom, and whatever was left landed on the last section — the current message, which is short and inelastic — where it died. Now each section takes min(weighted quota, what it actually needs) and the surplus is handed out in elasticity order, recent history first. On its own this changed nothing measurable (the fixed five-round slice capped the demand), but leaving it one-pass would have re-introduced the ceiling the moment anything else grew. sendChatRequest now returns contextCompacted / contextExternalized / contextSerializedBytes, and both controllers emit X-Qwen2API-Context-Compacted: <original bytes> when the attachment failed and the context was reduced. That path returns a normal 200, so without the header a client cannot tell that the model saw a fraction of what it sent. Not done from the audit's item 6, and why: it proposed splitting the upload/parse retry in externalizeOversizedAgentContext's catch, but that catch contains no retry at all — it falls straight through to compaction, so the premise does not hold against this code. The AGENT_CONTEXT_PARSE_* knobs are also left out: they size a timeout whose real-world frequency has not been measured. 673 tests, 0 fail (--test-concurrency=1). The two budget tests fail on the previous code. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…per request parserMessages built a fresh CacheManager on every call, i.e. every HTTP request. A user turn is several requests — the agent tool loop — so the same image was uploaded whole each time. Measured on staging 2026-09-08: 6 uploads of a 114440-byte image inside 77 seconds, plus 3 of a 206032-byte one; ~830 KB and ~10 round trips wasted per turn, each burning an account-rotation slot. The cache is now a module-level singleton, bounded at 512 entries, with entries expiring 10 minutes after they are written. On why a time bound rather than a measurement: file_url is minted by the STS endpoint BEFORE any bytes are uploaded (upload.js:151-157), so we have no lower bound on how long one stays valid, and "shorter than any plausible OSS lifetime" is an unsupported claim. The bound that does hold is that this repo already ships the infinite-TTL version of the same bet as its recommended Docker deployment: CACHE_MODE=file with ./caches mounted (README.md:207, 222-234) writes the same file_url to disk and reuses it forever, across restarts. A 10-minute in-memory map is strictly more conservative than that. The honest cost, since it is a real regression in one dimension: today is self-healing — every request re-uploads, so a dead URL cannot persist. After this, a URL that dies inside the window is reused silently, because a cache hit skips the upload and the `return null` in normalizeMediaContentItem, the only image-drop detector on this path, never fires. The TTL is what bounds that at 10 minutes. That is why it is a constant: making it refreshing (LRU) or configurable removes the only cap on the age of a URL handed upstream. Entries are never refreshed, so Map insertion order is age order and plain FIFO already evicts the oldest. Deliberately not done: no account-scoped key (upload and chat accounts are already independent getAccount() calls that differ with 2+ accounts, so cross-account referencing happens on 100% of image requests today — if it were broken images would already be broken, and keying by account only lowers the hit rate); no file_id/evict-on-failure plumbing (multi-file change guarding an unmeasured failure the TTL already caps); no env var; and CACHE_MODE=file is left alone — it is broken only without the documented volume mount, which is a different one-line bug for a different commit. chat-helpers.js now holds a module reference to upload.js rather than a destructured binding, so the uploader can be replaced in a test without hitting the network. imgCacheManager is exported solely so tests can call clear(): node --test isolates per file, not per test, and a module-level cache otherwise leaks state between cases in the same file. 678 tests, 0 fail (--test-concurrency=1). All five new tests fail on the previous code. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…t and upload cache Nine commits, each with a test that fails without it. Everything below was measured against the real Qwen upstream on the staging container, not inferred from unit tests. WHAT WAS BROKEN parserMessages uploads media only from the LAST message; every earlier message goes through extractTextFromContent, which drops non-text items with no log and no upload attempt. Real clients never leave the image last — Claude Code appends a text-only meta message, OpenClaw appends a runtime-context block, and any tool round-trip pushes it further back. So the model was asked about screenshots it had never been sent, and answered from whatever image was still in its context. MEASURED, BEFORE -> AFTER (5 runs each, uploads counted per request) anthropic, image + 1 tool call 0/5 uploads, "no image was provided" -> 5/5, 4/5 correct anthropic, image + 2 tool calls 0/5 uploads -> 5/5, 5/5 correct openai, real OpenClaw shape 0 uploads (3/3 captured requests) -> 5/5 uploads agent context budget 4559 B, 9.3% of cap, 5 of 60 rounds -> 46017 B, 93.6%, 60 of 60 same image across 4 requests 4 uploads -> 1 The 4/5 is not a residual defect: the no-tool control scores the same, so that one miss is model non-determinism, not transport. ALSO FIXED - Unsupported Anthropic content blocks (document/PDF, unforwardable image sources) vanished silently; a message made only of them disappeared entirely, which made the previous ASSISTANT reply become "# Current message". - A stream dying mid-closer released "[END " as prose, which made the NEXT [TOOL CALL] fail its gate — a real tool call dropped in silence. - Context externalization failure returned 200 with a fraction of the context and no signal; now X-Qwen2API-Context-Compacted says so. KNOWN TRADE-OFF, ACCEPTED DELIBERATELY The upload cache is the one place correctness was traded for cost. Today every request re-uploads, so a dead URL cannot persist; from now on a URL that dies inside the 10-minute window is reused silently, because a cache hit skips the upload and the only image-drop detector never fires. The TTL is the sole bound on that, which is why it is a constant and must not become refreshing or configurable. Justified by the fact that CACHE_MODE=file — the recommended Docker deployment — already caches the same URLs on disk forever. DELIBERATELY NOT DONE - The agent-turn gate still rejects prose+wrapper and exhausts to a 429. It is a designed contract with its own test, and real OpenClaw traffic passes it. - CACHE_MODE=file never creates its caches/ directory. Real one-line bug, broken only without the documented volume mount. Separate commit. - The double-<agent_final> wrapper leak. Cosmetic. - Account-scoped cache keys, file_id eviction plumbing, and a TTL env var: each rejected with reasons in 264d562. NOT VERIFIED No human has run a real Claude Code or Nova session against the last three commits; the evidence there is probes and upload counts, not a live session. Production (7860) does not have any of this and is out of our hands.
The existing dev-probes all cover image delivery and context size. None covers the tool protocol, which is what Claude Code actually runs on. Cells A-F run against /v1/messages, G mirrors them against /v1/chat/completions. B is the one that matters: five Read calls with different paths in one synthetic session, then "what did the SECOND one return?" - it fails if the model re-reads instead of using the result, which is the 63.7% duplicate class. D, E and F are read off the responses A/B/C already paid for, so the probe spends four model calls per path and eight in total. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Medido sobre 192 sesiones reales de Claude Code que pasaron por este proxy
(15.337 bloques tool_use): 1.451 llamadas duplicadas entre turnos, y en 925
de ellas (63,7%) habia otra llamada a la MISMA herramienta con argumentos
distintos entre la original y la repeticion. foldToolMessages tiraba el
call id y etiquetaba cada resultado solo con el nombre, asi que un turno con
veinte Read producia veinte bloques identicos `[TOOL RESULT: Read]`: el
modelo no podia saber que resultado contestaba a que llamada, y volvia a leer.
La historia foldeada pasa a llevar un ordinal monotono por request:
`[TOOL CALL #n]` y `[TOOL RESULT #n: <name>]` comparten numero, con el
tool_call_id como enlace. Un resultado que no reclama ninguna llamada se queda
en la forma sin numero: inventarle uno lo haria apuntar a otra llamada.
El ordinal existe SOLO en la historia. El marcador vivo que el prompt pide
emitir sigue siendo `[TOOL CALL]` sin atributos, y el prompt lo refuerza
("Never write a number in a marker you emit"). Aun asi, si el modelo imita el
ordinal la llamada no se pierde: el trigger es un prefijo y el payload se
recupera igual (pinchado en tests).
Frontera de inyeccion: neutraliseResultMarkers exigia un ":" pegado a RESULT,
asi que no reconocia la forma numerada — un cuerpo de resultado (contenido no
confiable) podia falsificar `[TOOL RESULT Rfym21#3: X]` y suplantar la respuesta de
una llamada real. Ahora se desarma cualquier `[` seguido de TOOL RESULT.
npm test: 678 baseline + 8 nuevos = 686, 0 fail.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Medido sobre 192 sesiones reales de Claude Code (15.337 bloques tool_use):
1.451 llamadas duplicadas entre turnos, y 526 de ellas (36,3%) sin ninguna
colision de nombre — el modelo simplemente reemitio una llamada que ya habia
hecho. No era confusion de correlacion (eso lo arregla la numeracion de
foldToolMessages): era que ni buildToolSystemPrompt ni buildAgentTurnDirective
tenian una sola regla contra repetir, mientras que todas las que si tenian
empujan a emitir mas llamadas.
buildToolHistoryLedger(messages, {maxEntries, maxBytes}) construye el bloque
"# Already executed this task": una linea por par distinto nombre +
canonicalJson(args), con el ordinal de foldToolMessages y un digest del
resultado de <=120 caracteres. Nada se suprime — repetir es a veces correcto,
releer un archivo despues de editarlo es la conducta buena — solo se hace
visible y direccionable.
Acotado por los dos lados porque se inyecta en CADA request y compite contra
el umbral de externalizacion de 90 KiB: 40 entradas y 6 KB por defecto, las
mas recientes primero, con una nota explicita cuando la lista quedo recortada
(sin ella, "no esta en el ledger" se leeria como "no se llamo nunca", que es
justo la conclusion falsa que dispara el duplicado).
Argumentos y digests son contenido NO confiable que vuelve al prompt, asi que
el renglon entero pasa por neutraliseResultMarkers: canonicalJson escapa
comillas y saltos pero no los corchetes, de modo que un argumento con
`[TOOL RESULT Rfym21#2: Read]` llegaria literal y podria hacerse pasar por la
respuesta de otra llamada. Para que folding y ledger usen una unica regla,
neutraliseResultMarkers se muda a agent-turn.js (la hoja del grafo) y
tool-prompt.js la importa; el cuerpo no cambia.
Una regla nueva en buildToolSystemPrompt y una clausula nueva en
buildAgentTurnDirective, ambas con la excepcion en la misma linea
("unless a preceding action could have changed it") para no prohibir el
repetido legitimo. La numeracion del ledger y la de la historia foldeada
quedan clavadas juntas por test: si se desincronizan, el ledger dice Rfym21#3 y la
historia llama Rfym21#3 a otra llamada, que es peor que no numerar.
npm test: 700 tests / 0 fail (686 previos + 14 nuevos).
El ledger de la Task 2 no servia de nada mientras nadie lo llamara. Ahora se inyecta en los dos caminos con el mismo orden de ensamblado: toolPrompt -> ledger -> envelope (historia + mensaje actual) -> directive. Va pegado al protocolo de herramientas porque es parte de ese contrato: sin el prompt delante seria una lista de ordinales sueltos, y tiene que leerse antes de la historia que documenta. Y vive en el PREFIJO, que parseAgentEnvelope (utils/request.js) nunca externaliza — dentro del bloque de historia el contrapeso desapareceria justo en las conversaciones largas, que son exactamente las que repiten llamadas. Se arma sobre los mensajes PRE-FOLD en ambos lados (`flat` antes del fold en anthropic.js#buildInternalRequest, `messages` antes del suyo en chat-middleware.js#processRequestBody). Despues de foldToolMessages la llamada ya es texto dentro de un string (`[TOOL CALL Rfym21#1]`), sin tool_calls ni tool_call_id que recorrer: un ledger armado tarde sale vacio y el bloque desaparece sin que nada lo delate. Los tests clavan ese fallo silencioso comprobando que la entrada trae los argumentos y el digest reales. Gated en hasTools en los dos caminos: sin protocolo de herramientas el bloque no tiene contrato que lo explique. Cubierto para tools ausentes y para tool_choice "none", que es el otro modo en que hasTools cae a false. npm test: 704 tests / 94 suites / 0 fail (700 previos + 4 nuevos). Nota: la corrida en paralelo dio 703/93 una vez — `node --test --test-concurrency=1` reproduce 704/94/0 de forma estable y es lo que hay que mirar cuando el numero baja sin fallos. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Los tres createToolCallLedger() son por-intento: nada en el servidor comparo
jamas una llamada saliente contra los tool_use que ya venian en el array de
mensajes. Por eso los 1.451 duplicados entre turnos medidos sobre 192 sesiones
reales de Claude Code no dejaron una sola linea de log — el servidor no sabia
que esas llamadas ya habian corrido.
createToolCallLedger acepta ahora {seed} (extractHistoryToolCalls sobre los
mensajes PRE-fold, con los mismos ordinales que foldToolMessages escribe en
`[TOOL CALL #n]`) y expone wasInHistory(call).
La restriccion que manda: una entrada sembrada NO SUPRIME. Marca la llamada
como ya vista para poder registrarla; suprimir romperia la relectura legitima
despues de un edit, que es conducta correcta. La decision de emitir no cambia
ni un byte — el pin tests/openai-agent-turn-cutoff.test.js:600 pasa sin tocar.
Lo unico nuevo es un logger.warn etiquetado AGENT por repeticion historica, con
nombre y ordinal y jamas el payload (tests/tool-prompt.test.js:1503,1774).
Sembrado en las dos rutas: anthropic.js lo saca de buildInternalRequest y lo
pasa por ctx a las ramas de streaming y no-streaming; chat-middleware.js lo deja
en req.tool_history_calls y chat.js lo pasa al runtime OpenAI.
11 tests nuevos en tests/tool-repetition.test.js (704 -> 715, 0 fail, en serie).
…call was emitted mapAnthropicStopReason checked hasToolCalls before the length/max_tokens branch, so a turn the upstream cut off mid-emission still reported stop_reason "tool_use". The client reads that as "the model finished asking for the tool, run it" and executes a call whose arguments may be truncated. The native API reports max_tokens there: the turn did not end. This is precedence, not suppression — the tool_use blocks already emitted still ship on the wire, only the stop_reason framing them changes. Both call sites (streaming and non-streaming) go through the shared mapper, so they move together. content_filter/refusal with an emitted call deliberately keeps reporting tool_use; same family, but out of this change's scope and noted in the test. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
/v1/messages was emitting `call_<24 hex>` — the shape the shared constructor (tool-prompt.js createToolCallObject / buildEmitted) mints for the OpenAI wire. The native Anthropic API uses `toolu_`, and this controller already generated `toolu_` for the INBOUND direction (flattenAnthropicMessages, when a client tool_use block arrives without an id), so the two directions lived in different id namespaces inside the same file. The rewrite goes at this route's emission boundary, never in the shared constructor: /v1/chat/completions must keep emitting `call_`, and that shape is part of its contract. Both Anthropic emission sites are twins and changed together — the streaming `emitToolUse` content_block_start and the non-stream loop that assembles `content[]`. The relabel keeps the same 24 hex, so two distinct calls in one turn (fresh UUIDs) stay distinct; an id of any other shape cannot be relabelled without risking a collision, so it gets a freshly minted one instead. flattenAnthropicMessages deliberately does NOT go through the rewriter. There the id is the client's own (`toolu_01LhEfp5…`, base62, not 24 hex) and it is the key linking a tool_use to its tool_result — rewriting it would break that correlation. It only shares the minting helper. tests/anthropic-native-parity.test.js gains five tests: toolu_ on a single call, toolu_ plus uniqueness across two calls in one turn (stream and non-stream), no `call_` anywhere on the Anthropic wire, and the OpenAI twin still minting `call_` with unique ids — that last one is the guard that the change stayed local. The stale pin at anthropic-native-toolcall.test.js:381 asserted `call_` on the Anthropic wire and now asserts `toolu_`; its intent (fresh ids, never a platform function_id) is unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`stripToolCallResidue` tenía cuatro llamadores en anthropic.js y CERO en el camino OpenAI: openai-agent-runtime.js calculaba `residueSpans` dentro de settledTextRound y los tiraba al suelo — el objeto attempt no los exponía y ningún llamador los leía. Por eso un `[END TOOL CALL]` huérfano salía como texto visible del asistente (20 casos medidos sobre 192 sesiones reales de Claude Code). El attempt expone ahora `residueSpans`, ya rebasados a coordenadas de `visibleText`, y chat.js#prepareAgentOutput los pela por posición en la entrega — un único embudo para streaming y no-streaming. El rebase es lo que hace útil al resto: el gate sólo entrega prosa envuelta en <agent_final> (agentTurnAcceptBareFinal=false), así que el único residuo entregable pasa por el desenvuelto de parseAgentControlText y, sin rebasar, el pelado posicional no encontraría un solo span. Falla cerrado en las dos direcciones (tramo contiguo y no ambiguo, cada span revalidado contra el destino): ante la duda se entrega el residuo antes que morder la respuesta. La DETECCIÓN no se toca: `attempt.visibleText` sigue byte a byte como salió del parser, así que containsOrphanProtocolResidue enciende el reintento malformed_protocol igual que antes; lo que cambia es lo que se entrega en la segunda pasada "tal cual". Y como el pelado es posicional y nunca una búsqueda, un bloque encercado que cita el mismo marcador no registra spans y llega intacto. Límite conocido, no cubierto por esta capa: en streaming con la config por defecto el cuerpo del <agent_final> sale en vivo por on_content_delta mientras se genera, y ahí el residuo ya está en el cable — misma limitación que anthropic.js con sus text deltas inline. Los spans se filtran contra lo ya emitido para no reenviar el turno entero detrás. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…tools foldToolMessages was gated on hasTools. A request with no `tools` array (or `tool_choice: 'none'`) skipped folding, so an assistant message carrying only a `tool_use` block kept `content: ''`; formatSingleMessage (chat-helpers.js) drops any message whose text is empty, so the entire assistant turn vanished from the JSONL history while its `tool_result` survived as a line with the nonexistent role "tool" — the model saw a result with no call that asked for it. Claude Code's compaction and summarisation requests have exactly that shape. The history is now folded according to what it CONTAINS, not what this request declares. This is rendering, not protocol: the tool system prompt, the executed call ledger and the agent turn directive stay gated on hasTools, so a tools-off request recovers its readable history without learning to call tools. The predicate is chat-helpers.js#willBeFolded, now exported rather than rewritten: it already exists for the media sweep and its contract is literally "does foldToolMessages rewrite this message?", aligned with the fold's two branches. A third copy would drift. Tests: 4 in tests/anthropic-native-parity.test.js — both turns render in order with the right roles with no tools array and with tool_choice none; the protocol prompt, ledger and directive stay absent; a history with no tool blocks is unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Con extended thinking + tools, Claude Code reenvia el bloque `thinking` junto al `tool_use` que produjo. `flattenAnthropicMessages` lo tiraba, borrando el registro que el propio modelo dejo de POR QUE hizo esa llamada — justo lo que alimenta la clase de duplicado que ataca este plan. La rama `assistant` ni siquiera tenia clausula: el bloque se caia del if/else sin dejar rastro (ni siquiera una linea en droppedBlockTypes). Ahora el razonamiento se renderiza delimitado por [THINKING] / [END THINKING] delante del texto y, tras foldToolMessages, delante del bloque de llamada: se lee en orden cronologico penso -> dijo -> llamo. Tres cuidados, cada uno con su razon: - Neutralizacion: el texto es contenido que vuelve al prompt y puede citar marcadores. Pasa por neutraliseResultMarkers (la misma regla que el fold y el ledger) y ademas se defusa [END THINKING], porque un delimitador que el cuerpo puede escribir no delimita nada. - Tope de 1200 chars por mensaje, recortando por la CABECERA: la decision que produjo la llamada esta al final del razonamiento, asi que quedarse con el principio tiraria exactamente el porque que veniamos a rescatar. - redacted_thinking rinde un placeholder, nunca los bytes opacos; la signature no viaja. La rama `user` sigue tirando el bloque a proposito: segun la spec el razonamiento vuelve en turnos de assistant, en rol user no hay intencion de usuario que preservar, y esa decision ya estaba fijada por image-passthrough.test.js:387. La asimetria queda documentada en ambos sitios y con un test que guarda el limite. Efecto colateral correcto: un turno de assistant que solo llevaba thinking producia content:'' y desaparecia entero de la historia (misma clase de bug que la tarea 8); ahora sobrevive. Tests: 737 -> 743 (+6), 0 fail. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…teaches T1 numbers the folded call block (`[TOOL CALL #n]`), which the model reads on every round. The natural imitation mirrors the ordinal onto the closer, and the parser was blind to exactly that — because of the space T1 writes: "[END TOOL CALL]" orphan=true spans=["[END TOOL CALL]"] "[END TOOL CALL#7]" orphan=true spans=["[END TOOL CALL#7]"] "[END TOOL CALL Rfym21#7]" orphan=false spans=[] <- leaked TOOL_CALL_CLOSE_BRACKET_RE's decoration class `[^\s[\]]{0,16}` excludes whitespace, so it could never reach the '#'. Consequence on BOTH paths: `[END TOOL CALL Rfym21#3]` was delivered to the client as visible assistant text, stripToolCallResidue had no span to remove (there is deliberately no second delivery-layer scan), and containsOrphanProtocolResidue returned false, so the malformed_protocol retry at anthropic.js:1404,:2070 and openai-agent-runtime.js:600,612,640,756 never fired. That is a regression in the exact metric the plan measures (protocol text leaked to the user). The fix is one bounded ordinal arm, mirrored in all three places that spell the closer out: the anchored regex, the bare arm (a stream dying on `[END TOOL CALL Rfym21#3` requires "nothing left after the match", and `Rfym21#3` always remained), and isDanglingCloserPrefix's literal table (`[END TOOL CALL #` at EOF). Only a digit run is tolerated, never a word: `[END TOOL CALL Rfym21#3 and the answer is 42]` still parses as prose, keeping the rule at tool-prompt.js:103-105 (better to leak a closer than to eat the model's answer). Whole-text and streaming stay in lockstep. The synthetic-rescue boundary is unchanged: the write side (neutraliseResultMarkers) already breaks the head char of any `[...TOOL CALL` in a result body regardless of what follows. probe-agent-loop.js cell F scanned for literal markers, so it would have reported "clean" on the very form that leaked; it now matches the numbered family too and prints the exact leaked text, flagged ORDINAL-IMITATED. That cell is the live acceptance gate for whether Qwen imitates the ordinal at all. Tests: 3 new regression tests + 2 extended closer matrices fail without this fix. 2 further tests guard the widening itself (prose is never eaten; the injection boundary still holds) and the TOOL_CALL_CLOSE_MAX literal mirror. Suite: 678 baseline + 65 (T1-T9) = 743 before; +5 added here = 748 expected, 748 observed, 0 fail (stable over 6 consecutive runs).
…e a result Dos defectos alcanzables desde una request normal por la ruta ya cableada (chat-middleware.js / anthropic.js), ambos verificados extremo a extremo. 1. Una entrada forjada desde `arguments`. renderLine colapsaba el NOMBRE y el digest, pero a los argumentos solo les aplicaba truncateChars, y la rama de parseo deja `parsed` como el string CRUDO cuando los argumentos no son JSON — el sintoma medido que motiva el plan: el modelo emite argumentos malformados, vuelven como historia en el turno siguiente y se auto-forjan — o cuando el JSON decodifica a un string. Los saltos literales sobrevivian y cada uno abria otro renglon con la forma exacta de una entrada legitima, bajo una leyenda que le dice al modelo que esos resultados ya corrieron y los reuse. neutraliseResultMarkers no lo tapaba: reescribe `[` y `<`, nunca saltos. Se colapsa esa rama en construccion. Solo esa: colapsar tambien la salida de canonicalJson fundiria `echo hi` con `echo hi`. 2. Un digest viejo colgado de un ordinal sin contestar. En una repeticion, la entrada avanzaba al ordinal nuevo y se quedaba con el digest del viejo, asi que releer despues de editar — el escenario que JUSTIFICA no suprimir repeticiones — rendia `Rfym21#3 Read {a.txt} -> CONTENIDO PRE-EDICION` cuando en la historia foldeada no existe ningun [TOOL RESULT Rfym21#3]. La misma correlacion falsa que Task 1 elimina. Ahora byCallId guarda el ordinal de CADA llamada, el digest se adjudica a esa instancia (y solo avanza a una mas nueva, no al ultimo resultado procesado) y el renglon nombra la instancia contestada cuando difieren. 3. El presupuesto documentado era ASCII. Medido: ~215 B/entrada y ~26 entradas en ASCII, ~460 B y ~12 en CJK. Degrada sin mentir (la nota de omision se dispara igual), pero el comentario era 2x optimista para un producto bilingue con upstream chino. Tests: 7 nuevos en tests/tool-repetition.test.js. 4 fallan contra el codigo previo (las dos ramas de forja, la repeticion sin contestar y los resultados en desorden); 3 son guardas contra sobre-corregir (el JSON bien formado no se colapsa, la repeticion CONTESTADA se renderiza limpia, el tope de bytes aguanta CJK). Suite: 748 base + 7 = 755, 101 suites, 0 fail (paralelo y serial coinciden).
T3 wired the block correctly — order, position, gating and pre-fold construction all verified — but adversarial review found the block itself stating falsehoods, and stating DIFFERENT falsehoods on each path. The caption tells the model "these calls already ran and their results are above; reuse a result instead of repeating its call", so every false line in it is a direct push toward the duplicate the plan exists to remove. 1. A result carrying an image was reported as no result at all, and the two paths disagreed on how. Claude Code's Read puts the image inside tool_result.content; each path moves it somewhere different before the ledger sees it — Anthropic to the `media` bypass leaving content:'' (rendered `-> (empty)`), OpenAI as an item in the content array that harvestCurrentTurnMedia empties (rendered `-> []`). Read is the most repeated tool in the measurement (802 of 1,451). Fixed by summariseToolResultContent: text is text, non-text items are COUNTED and announced as `(1 image)` / `(N images)`, never serialized (the old JSON.stringify put the base64 data URI in the prompt, cut at 120 chars). Both paths now build the ledger before their own media step, so the two render the same line for the same logical call. 2. The legacy functions API listed every call as resultless. The call-side walk supports `assistant.function_call`, but the result side keyed only on tool_call_id, which `role:'function'` never carries — the branch was dead and its messages were dropped one line later, while foldToolMessages did write their `[TOOL RESULT: Read]` two lines below. Now an id-less call registers in a per-name FIFO and a result with no tool_call_id resolves through it. Only the ABSENCE of an id falls back to the name: a mismatched id is a mismatch, not an absence, and still adjudicates nothing. 3. The ledger put tool-derived text ahead of the envelope headers for the first time. parseAgentEnvelope splits on indexOf for the history header (first wins) and lastIndexOf for the current-message header (last wins), so a result containing either literal moved the cut and the poisoned tail was parsed as genuine JSONL. neutraliseResultMarkers now breaks the leading `#` of both, the same way it already breaks `[` and `<`, one ASCII byte for another so the byte caps stay exact. Ordinary markdown headings are untouched. 4. The Anthropic path JSON-escaped its whole prefix. ensureAgentCurrentEnvelope short-circuits when it already sees a history marker; a request whose history fits in one message has none, so applying the envelope AFTER the prefix wrapped the tool protocol and the ledger inside `# Current message` with literal \n — reachable with a one-message request, not a hypothetical. The envelope now runs before the prefix, mirroring the OpenAI twin. With history present the assembled content is byte-identical to before, verified on both paths, with and without `system`, with and without tools. Tests: 11 added to tests/tool-repetition.test.js. 7 fail against the previous source (both digest shapes, the base64 leak, the legacy link, the two header-smuggling cases and the escaped prefix); 4 are guards against over-correcting — a wrong id is still not rescued by name, markdown headings survive, and the two paths are compared to EACH OTHER for both a text result and an image result, which no earlier test did. A fixture with a non-empty `system` now pins the three-part Anthropic prefix order. Not fixed, stated deliberately: foldToolMessages still renders an image-only result as `null` (Anthropic) / `[]` (OpenAI) inside the folded history. That predates this branch, and a symmetric repair needs the media twin scans in chat-helpers.js to leave the count behind — those two scans must change in lockstep and belong to the image-delivery invariant, not here. The ledger, which is the block that asserts the results are usable, is now correct and identical on both paths. Also not changed: the 6 KB ledger cap. It measures 5,986 B on 30 Read calls (6.5% of the 90 KiB externalization threshold). That cost buys the counterweight in exactly the long conversations that produce duplicates, because the prefix is never externalized; the digest is a 120-char pointer to a result the model must still read above, not a copy of it. Retuning it belongs to Task 11, against live evidence. Suite: 755 before + 11 = 766 expected, 766 observed, 101 suites, 0 fail (identical in parallel and serial). eslint clean on all four files.
…livering nothing T7 ported the delivery-layer strip from anthropic.js:2278 but not the residue-only guard that immediately follows it (anthropic.js:2269), whose in-code comment says the order is load-bearing: strip first, then judge emptiness, and never ship an empty-content message. Porting half of it moved the OpenAI path from one frozen-matrix violation to the other: `<agent_final>[END TOOL CALL]</agent_final>` stripped to empty and went out as HTTP 200 with content "" and finish_reason "stop" — a silent dead turn for an agentic client, where /v1/messages returns 502 for the same bytes. Measured on the parent commit the same input delivered "[END TOOL CALL]", so the commit traded a raw-protocol leak for an empty success. - prepareAgentOutput now reports `residueOnly` — spans registered, no tool calls, body non-blank before the peel and blank after — and both handlers route it to a 502 `invalid_tool_call`, matching the twin's failure class and detail. On the stream path it fires only when nothing has gone out on the content channel; once text is live the gate's 422 already owns that case. - The peel now applies stripAgentTags after stripToolCallResidue, the full pair from anthropic.js:2278: a nested <agent_final> survives unwrapExactTag (anchored at the end, so it only consumes the outer wrapper) and was reaching clients raw — measured at 3 of 29,352 real turns. Kept inside the `spans.length > 0` guard like the twin, so a zero-residue round is still byte-for-byte what it is today; stripping tags unconditionally would break the stream discount when an unstripped nested tag already went out live and would re-send the whole turn behind it. - Both the delivered content and the discount comparison go through one peelDeliverableText, so they cannot drift apart and duplicate the answer. Opposite direction pinned too: prose plus a stray closer stays a 200 with the closer removed and never escalates to 502. Four of the seven new tests fail against the pre-fix source with the exact reported symptoms (200 vs 502, finish_reason "stop" vs an error event, and the literal "x <agent_final>y</agent_final>"); the other three are pass-both-ways controls. Serial suite: 766 baseline + 7 added = 773 tests, 101 + 2 = 103 suites, 0 fail.
Verificacion adversarial de la Tarea 8. El arreglo Anthropic era correcto pero
la tarea quedaba incompleta contra la restriccion global del plan ("ambos
caminos cambian juntos"), y abria una via nueva para falsificar un ordinal.
1) EL GEMELO (chat-middleware.js). `foldToolMessages` estaba dentro de
`if (hasTools)` y reproducia el defecto letra por letra en
/v1/chat/completions: sin `tools` (o con `tool_choice: 'none'`) el assistant
que solo lleva `tool_calls` tiene `content: null`, formatSingleMessage lo
descarta y EL TURNO ENTERO desaparecia, mientras su resultado sobrevivia
como una linea JSONL con el rol inexistente "tool". Medido en HEAD antes de
tocar nada. El fold pasa a la misma puerta que el gemelo Anthropic
(`hasTools || some(willBeFolded)`), entre la cosecha de medios y el
recolgado. El supuesto bloqueo (image-passthrough.test.js) no existia: la
asercion de ese pin sobrevive intacta, solo su comentario estaba caducado.
Verificado ademas que t2i / t2v / deep_research / image_edit / search dan
salida identica byte a byte antes y despues, salvo la historia reparada.
2) COLISION DE ORDINALES (tool-prompt.js). Desde que la historia se pliega
tambien sin tools, un resumen de compactacion puede citar los marcadores que
le enseñamos; el cliente lo reenvia como mensaje de texto plano en la
peticion siguiente, esta vez CON herramientas, y ese `[TOOL RESULT Rfym21#1: X]`
inventado convivia con el Rfym21#1 real — dos bloques reclamando la misma llamada,
uno falso. Es la colision que la Tarea 1 existe para eliminar. Ahora el fold
defusa los marcadores de todo texto que no escribio el mismo: mensajes de
paso y el texto libre del assistant que precede a sus propias llamadas (este
ultimo llevaba sin defusar desde la Tarea 1). Solo se toca texto; los items
de media pasan intactos y un mensaje limpio conserva su identidad.
La defensa va en la ENTRADA, no en la entrega: sin `hasTools` no se
construye parser (anthropic.js:1107), asi que no hay residueSpans que
recortar, y crearlos obligaria a correr el parser de herramientas sobre
peticiones que no declararon ninguna — riesgo mayor que la fuga. Defusar a
la vuelta cubre ademas cualquier otro origen (transcripcion pegada, fichero
citado). La decision queda pinchada con su test.
3) Un resultado vacio dice `(empty)`, no `null`: la herramienta corrio y no
devolvio nada, no devolvio JSON null. Solo se nota ahora que el mensaje ya
no desaparece. El pin de tool-prompt.test.js conserva su proposito (el
bloque sigue visible) con el literal actualizado.
Tests: 12 nuevos (5 gemelo OpenAI, 6 colision, 1 vacio-vs-null); 10 de los 12
fallan contra el fuente previo. 773 (HEAD ab89650, medido) + 12 = 785 tests /
106 suites / 0 fail. Lint limpio en los seis ficheros tocados.
…ops being per-message Tres agujeros del primer intento de la Tarea 9, medidos, no supuestos. 1. El delimitador era forjable desde tres de los cuatro canales. La defusa vivia dentro del cuerpo del `thinking`, asi que el texto HERMANO del mismo mensaje y el cuerpo de un `tool_result` (ficheros, paginas, salida de comandos: el canal MENOS confiable) escribian `[END THINKING]` crudo en la historia y cerraban un bloque que no era suyo. El unico test que lo pinchaba corria contra un fixture sin bloque de texto: no podia fallar. El brazo THINKING NO puede vivir en `neutraliseResultMarkers`: esa regla se aplica tambien al contenido de un mensaje `assistant` (foldToolMessages), que SI lleva delimitadores nuestros — metido ahi, defusaba el delimitador REAL. Vive en `neutraliseUntrustedBody`, para cuerpos de los que nada es nuestro; el texto hermano usa el brazo suelto, porque el resto de sus marcadores ya los defusa el fold. Cubre `[/THINKING]`, `[END_THINKING]`, `[END\nTHINKING]`, `[THINKING: por que]`, y respeta la prosa (`[thinking about lunch]` no se toca). 2. El tope era POR MENSAJE, que no acota el agregado. Medido sobre la peor sesion del plan: el prefijo de 76 mensajes pasaba de 78.025 a 94.443 bytes y CRUZABA AGENT_CONTEXT_FILE_THRESHOLD_BYTES (92160), con lo que la peticion se externalizaba como documento y, si la subida falla, se trunca la conversacion — un cambio hecho para reducir duplicados provocaba el truncado que los produce. El presupuesto pasa a ser POR PETICION y no es una fraccion fija del umbral sino el HUECO que de verdad queda, gastado de lo NUEVO a lo VIEJO (el razonamiento que explica la llamada a punto de repetirse es el reciente). Misma medicion despues: n=76 -> 78.243 (218 B, no cruza), n=120 y n=1614 -> delta 0, byte a byte como antes de la tarea. Techo absoluto 12 KiB. 3. El recorte cortaba por unidades UTF-16 y partia pares subrogados. No llega crudo al wire (JSON.stringify escapa la mitad huerfana), pero el modelo lee el literal `\ud83d` en mitad del razonamiento. `trimLoneSurrogates` tambien arregla `truncateChars`, que tenia el mismo defecto en el digest del ledger. Los topes citados salen de medir 6.249 bloques `thinking` reales de 1.544 sesiones (p50=235, p90=755, p99=3.076, max=19.502; el 4,8% pasa de 1.200), no de estimar. Dos decisiones deliberadas quedan pinchadas en tests en vez de discutidas: la retencion NO se ata a `hasTools` (las peticiones de compactacion de Claude Code llegan sin tools y con la historia entera), y un turno final que solo lleva razonamiento pasa a ser el `# Current message` en vez de evaporarse. Gemelo OpenAI: `foldToolMessages` es compartido, asi que la defusa en el cuerpo de un resultado aterriza en /v1/chat/completions por construccion; hay un test que lo pincha. Tests: 785 base + 14 nuevos = 799 / 111 suites / 0 fail (serie y paralelo). 10 de los 14 fallan contra el codigo anterior (verificado revirtiendo las fuentes); los otros 4 son pines de decision. Sin verificar contra Qwen real. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…op probe Both the try and the catch assign `args`, so the `= null` initializer was never read. `npx eslint src tests tools` is now clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
probe-agent-loop.js cannot answer whether the model stops repeating itself.
Its cells B and C passed before the correlation fix landed — C vacuously,
because the model emitted no calls at all — so the synthetic probe has no
headroom. Static analysis shows the addressing information is now present for
every duplicate case; it does not show the model uses it.
This harness replays real history instead. Every point where the recorded
session re-issued a call it had already made is a natural experiment: we know
what the real model did with that exact context. Replay the prefix, classify
what comes back as REPEATED / MOVED_ON / ANSWERED / ERROR.
Two facts about the reference transcript that the plan's corpus-level numbers
hide, and that the measurement arm has to know:
- it holds ONE strict-immediate repeat, not 326. That figure is the
whole-corpus total (192 sessions, 15,337 calls). The single strict onset is
preceded by a task-notification, not by a tool_result, so `--mode strict`
yields zero usable scenarios here. `--mode cross` (177 eligible) is the arm
with statistical power and is the default.
- Claude Code writes one JSONL record per content block, not per message.
Read naively, no tool_use is ever preceded by its own thinking and no onset
is ever preceded by a tool_result — 0/179 eligible. Records are regrouped by
message.id first, which recovers 177/179.
Late-session prefixes run to ~340 KiB, far past the 90 KiB externalization
threshold, and that path is a different subsystem. A prefix that fits the budget
is sent verbatim; otherwise the request is the opening task plus a contiguous
tail anchored at the earlier identical call, so the duplicate target and the
result that answered it are always present. Every record says which shape it
used. Largest request in the default sample is 42.4 KiB.
Selection is deterministic — no RNG — so both arms see identical inputs.
Verified by node --check, eslint, and --dry-run; --dry-run --out dumps the exact
request bodies, all 20 of which were checked for role alternation, resolvable
tool_result ids, and presence of the duplicate target. No upstream calls spent.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The harness was mechanically correct and inferentially broken. A design review found the experiment measured the wrong population, with half the treatment provably disabled on a quarter of the cells, at a sample size that cannot reach significance, using a metric that rewards the model for going quiet. Every fix below costs no quota and several SAVE it. Population. The default transcript is now the Read-heavy Qwen2API session instead of the lohari one. Root cause 1 addresses results that cannot be told apart; lohari's repeats are 63% retry-after-empty-Bash-output, which that fix cannot address by construction, and its ledger is evicted on 9/20 cells. Measured on the fitting onsets: Read 69/Bash 14 with 6% empty here, versus Bash 69/Read 10 with 63% empty there. lohari stays available as a second, separately reported stratum; the header forbids pooling them. Selection. --max-calls was applied by slicing AFTER the stride, so `--limit 200 --max-calls 30` silently returned the earliest 30 onsets (everything before call#321) instead of 30 spread across the session; the ceiling now applies during selection. Selection is stratified across gap quartiles and capped at --per-target scenarios per target file, because both reference sessions put 68-80% of their onsets on ONE file and an even stride returned ~3 situations sampled 20 times. The cap is strict: when it binds the run returns fewer scenarios and says so, since topping the sample back up from the same file restores the count while destroying the independence the cap buys. Recording. Four things could not be reconstructed after the fact and are now recorded per cell: ledgerAnchorPresent (computed with the real controller flatten, so a cell whose ledger evicted the anchor is distinguishable from a treated one), residue, timestampMs, and the FULL response text rather than a 500-char slice. Also collisions -- the root-cause-1 dose -- and mutationBetween, since re-reading after an edit is correct behaviour and must be separable from the failure under test. Metric. The headline is now REPEATED/(REPEATED+MOVED_ON): the raw rate falls if the model merely stops calling tools, and the post-fix prompt pushes that way, so a lazier model would have scored as a win. Raw and tool-emission rate are printed alongside. A new TRUNCATED verdict is excluded from both denominators -- and is keyed on output TOKENS, not on stop_reason, because stop_reason is itself changed by the truncation precedence fix: keying on it would let a fix under test decide the classification and bias the very comparison being made. Arms. --base-url-b interleaves A and B per scenario instead of running one arm to completion then the other, so account rotation and upstream drift spread across both instead of loading onto the second. --repeat gives a within-arm noise floor. Paired discordance and an exact two-sided McNemar are computed in-harness, printed next to the realised cluster count so the p-value is read as descriptive. Also: --result-cap 1200 (was 400) so the p90 real result of 980 B survives, with a neutral ellipsis replacing a banner that announced missing content and was itself a reason to re-fetch; --stream to measure the shipped path; --profile to inspect a transcript's population before spending anything; and the pre-registration, the power limits and the known confounds stated in the header before any number is collected. tests: 799 baseline + 22 new = 821, 0 fail. The instrument had none.
Ran the pilot the design review demanded before the paired A/B, and it came back the way the review feared. 24 upstream requests, qwen3.8-max, pre-fix service at 309da59 against post-fix at 785486d: pre-fix 95b7b0c1, budget 48, collisions p50=6 0/8 REPEATED pre-fix 33f8544e, budget 90, collisions p50=44 0/8 REPEATED post-fix 95b7b0c1, same 8 byte-identical cells 0/8 REPEATED The null is not the vacuous one that made probe cell C worthless. Tool-emission was 100% in every arm: the model engaged on every cell and picked a DIFFERENT call. The instrument was audited against exactly that artifact before the result was believed -- prefixSigs held 20-52 signatures per cell and the about-to-be-repeated signature WAS in the set, so a repeat would have been caught. The absence is real. The cause is structural, not a tunable. Every runnable onset is windowed: 0 of 99 fitting onsets in 95b7b0c1 and 4 of 94 in 33f8544e survive as a full prefix even at the 90 KiB ceiling, and those 4 have gap<=6 and collisions<=3, so they carry no headroom by construction. Recorded prefixes run to ~340 KiB and the proxy externalises above 90 KiB, which is a different subsystem. The window strips the accumulated task state, and it shows in the output: pre-fix cells replied `cd … && ls package.json` and `cat package.json`, a model re-orienting in a repo it no longer has the history for rather than continuing the paging loop that produced the duplicate. MOVED_ON on this sample does not mean "used the numbered result", it means the replay changed the behavioural regime. So the duplicate-rate claim stays UNPROVEN, and the header now says so instead of inviting the next agent to spend 40 requests comparing 0% against 0%. A runtime WARNING fires whenever every selected cell is windowed, which on both reference transcripts is always. What the 24 requests did buy: - No regression, two-sided as pre-registered. 8/8 concordant pairs, discordant b=0 c=0. The predicted upward risk -- the ledger PRIMING repeats by printing the exact strings -- did not materialise here. - The lazy-model risk did not materialise either: the post-fix arm emitted MORE calls than pre-fix (11 vs 9) at identical 100% tool-emission, so the raw metric was not being flattered by silence. - Prompt cost measured on real agentic requests instead of a synthetic one: +6679 input tokens over 8 byte-identical bodies, +15.8% versus pre-fix. Larger than the +12.7% measured on a bare prompt because the ledger grows with tool history. - Zero protocol residue and zero errors in either arm. tests 821 / suites 111 / fail 0 (799 baseline + 22 harness tests, no new tests here). eslint clean over src, tests and tools.
…ned nothing
An image inside an Anthropic tool_result never reached the model. That is exactly
the shape Claude Code sends when it Reads an image file, so screenshots and image
reads silently did nothing through this proxy.
The image was never lost. Measured 2026-09-08 against real Qwen: it is uploaded,
harvested, moved into files[] and delivered, and the outgoing body for the failing
case is byte-identical to a working control apart from one string. What was lost is
the TEXT that says an image exists.
flattenAnthropicMessages builds resultContent from `.filter(b => b.type === 'text')`,
and a Claude Code image read carries no text block at all (37/37 image-bearing
tool_results across 1,556 real session files have zero text blocks). resultContent is
therefore '' in 100% of real cases, and foldToolMessages renders it as `(empty)`:
{"role":"user","content":"[TOOL RESULT Rfym21#1: Read]\n(empty)\n[END TOOL RESULT]"}
The model reads the authoritative record of what the tool returned, is told it
returned nothing, and answers NO_IMAGE while an unexplained image rides along in
files[]. The prompt even contradicted itself — the history ledger already printed
`Rfym21#1 Read {"path":"magenta.png"} -> (1 image)` two blocks earlier.
Both media scans stay untouched: the invariant that only the LAST turn's media is
uploaded is intact and still pinned. What changes is only what the result body SAYS.
- flattenAnthropicMessages writes `[1 image returned by this tool]` when it diverts
image blocks to the media bypass, appending it after any result text.
- harvestCurrentTurnMedia (the twin, per CLAUDE.md) writes the same note when it
strips media out of a role=tool body, which used to leave the literal `[]`.
- foldToolMessages renders an empty array body as `(empty)`, like an empty string:
`[]` reads as a real JSON value, not as "the tool returned nothing".
- the ledger digest recognises the note so it counts the attachment once and both
paths keep rendering the identical `-> (1 image)` line.
The note deliberately does NOT claim the image is attached. Only the last turn's
media is uploaded, and even there URL dedupe or HARVEST_MEDIA_CAP can drop it;
promising an attachment the model cannot see is worse than the `(empty)` it
replaces. It states the checkable fact and agrees with the ledger.
Cost: 0 bytes on every request without a tool_result image (pinned byte-identical);
+30 bytes, measured +6 input tokens, only when one is present.
Live verification against real Qwen (qwen3.8-max, /v1/messages), probe cell H —
the exact Claude Code shape, image-only tool_result, fresh 160x160 magenta PNG:
HTTP 200 stop=end_turn -> "magenta"
…o score Three defects, all of which let this probe report the wrong verdict: 1. The oracle matched /magenta/ against the raw answer, and the file being read is named `magenta.png`. Any answer that merely repeated the filename scored as "the model saw the image". Hit live: a pre-fix run whose text was a reasoning dump about `magenta.png` was scored SEES_IMAGE. The filename is now stripped before scoring, and NO_IMAGE is checked first. 2. It printed only the first 140 chars, so an answer that never reaches a colour was indistinguishable from one that does. Now 500. 3. It scored `content[].type === 'text'` alone. An agentic turn can answer with a tool_use block instead, which scored identically to a lost image. stop_reason and the tool_use names are now printed for every cell. Cell I is replaced by the non-confounded control. The old cell declared a `Read` tool and its history asked the model to read the file, so the model correctly reasoned it had not read it yet: it measured the agent contract, not image delivery. The control keeps the history and the image-last shape and drops the tools, which is what the cell was for. Post-fix, against real Qwen (qwen3.8-max): H) tool_result con image (Claude Code) HTTP 200 SEES_IMAGE stop=end_turn -> "magenta" I') historia + image ultimo msg, sin tools HTTP 200 SEES_IMAGE stop=end_turn -> "Magenta"
…ot a guess Measured 2026-09-08 by uploading a real PNG through this service's own upload path: x-oss-date = 20260909T042756Z x-oss-expires = 300 x-oss-signature-version = OSS4-HMAC-SHA256 The presigned URL Qwen hands back dies 5 minutes after it is signed. The cache served it for 10 (IMAGE_CACHE_TTL_MS), and in CACHE_MODE=file — the mode README.md recommends for the Docker deploy — for ever, across restarts, because that branch was a bare fs.existsSync with no expiry at all. Past the signature the OSS answers 403 `AccessDenied / Request has expired`, so we hand the upstream a dead URL, no error surfaces anywhere on our side, and the model answers as if no image had been sent. The rationale comment claimed we had no lower bound on a Qwen URL's lifetime. We do, and it is printed in the URL's own query string; the comment is corrected in place. - presignedUrlExpiryMs parses x-oss-date (ISO basic, which Date cannot parse) plus x-oss-expires into an absolute deadline, and returns null when the URL does not say. - cachedUrlIsUsable prefers that deadline, minus a 30 s margin so the URL survives the trip out and the upstream's own fetch. With no signature it falls back to 4 minutes from insertion, shorter than the old 10-minute bound on purpose: the whole benefit of this cache happens inside one turn (measured: 6 uploads in 77 s). - Both cache modes check it. In file mode an expired entry is unlinked so addCache can rewrite it, which self-heals caches already on disk with no format change. - A re-cached signature is deleted before being set again, so it goes back to the end of the insertion order the FIFO eviction reads as age. IMAGE_CACHE_TTL_MS stays as the upper bound for a hypothetical unsigned URL. Pre-fix, both behavioural tests fail exactly where they should: file mode reports a dead on-disk URL as a hit (true !== false), and default mode reuses a dead URL instead of re-uploading (1 !== 2).
Measured live against real Qwen (2026-09-08, qwen3.8-max, probe-matrix cell F,
LOG_LEVEL=INFO so the gate's own warn was visible): of 5 gate rejections on
/v1/chat/completions with tools, 3 were `invalid_control` and 2 were
`invalid_tool_call:tool_errors`. Zero were `bare`.
All three invalid_control rounds had the same shape — reasoning prose leaked
into the answer channel, followed by a perfectly well-formed pair:
"The image is clearly visible - it's a solid magenta/fuchsia color. I can
directly identify the dominant color without needing any tools.
<agent_final>Magenta</agent_final>"
The answer is correct and complete. `unwrapExactTag` threw it away because its
regex is anchored at BOTH ends, so only a whole-string wrap parsed; anything
else carrying a tag fell to `invalid_control`, which — alone among the rejection
families — had no give-up budget, so it burned all 3 attempts and surfaced as
HTTP 429. The Anthropic twin already delivers this exact shape with 200
(createAgentTagStripper, whose own comment says judging "prose + wrapper" as
invalid only makes the whole turn fail). This is parity, not new policy.
- parseAgentControlText accepts exactly one well-formed pair with text around
it and keeps BOTH halves, tags stripped. Keeping the outside text rather than
just the body is deliberate: the proxy itself prepends image markdown to
`answer` before this parse, so body-only would delete the image. Unbalanced,
doubled, and mixed-family shapes still reject — but no longer fatally.
- invalid_control gets the give-up budget intercepted/malformed_protocol already
have: the last attempt delivers the stripped text with finish_reason stop.
`bare` and `empty` keep their veto; there the model never declared a close, so
delivering would fabricate a conclusion (config/index.js:58).
- exhaustedError returns 502, not 429. Nothing here was rate limited, and 429
made chat.js label it `rate_limit_error` — telling an agentic client to back
off and retry the whole turn against the account it just failed on.
- The invalid_control retry hint now names the constraint still enforced; the
old text ("malformed or mixed wrapper") never told the model what to fix,
which is why all 3 attempts failed identically. Retry-only text, so it costs
nothing on the per-request prompt budget.
- logger.shouldLog is case-insensitive. This repo's own .env sets
`LOG_LEVEL=info` lowercase; `levels['info']` was undefined and
`undefined >= 1` is false, so every log line was silently off — including the
only trace this failure leaves in production.
- .env.example documents AGENT_TURN_MAX_ATTEMPTS,
AGENT_TURN_ALLOW_PROSE_WITH_TOOLS and AGENT_TURN_ACCEPT_BARE_FINAL, which
existed only in src/config/index.js.
Live after the fix: 8 cell-F runs, 0 gate rejections of any kind, 0 HTTP 429,
3 successful 200s carrying exactly the shape that used to be rejected (the
other 5 were account-level WAF 502s, unrelated). Because nothing is rejected,
nothing is retried — which also stops the gate tripling upstream load on a
single account, the thing that was tripping the WAF.
Tests: 854 pass, 0 fail (840 baseline at 3f0f48b + 14 added).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…aid it 03b0587 made every tool_result image body say `[1 image returned by this tool]`. That is true for the turn in progress and false for every turn after it: the two twin media scans deliberately stop at the last final assistant answer, so a previous turn's image is never re-uploaded and files[] is empty. Measured live against qwen3.8-max (two runs per cell, same account, minutes apart, the outgoing body differing in exactly that one string): on a previous-turn result the note made the model invent a colour 2/2, while the `(empty)` it replaced answered NO_IMAGE 2/2. It swapped "I lie that the tool returned nothing" for "I lie that you can see it" — on a shape ~94x more common in the user's corpus (37 image tool_results against 3,482 turns that come after one, since every later request re-sends the same stale result). The note now has two forms and the position picks one: current turn -> [1 image returned by this tool] earlier turn -> [1 image returned by this tool, not included in this request] The ledger digest agrees (`(1 image)` / `(1 image, not included)`); leaving it optimistic left the model believing the optimistic half. The turn boundary is recomputed in flattenAnthropicMessages from the same rule the scan uses, over the input shape. NEITHER MEDIA SCAN CHANGED, and `.media` is still set for every media result, so a drift between the two expressions of the rule can only ever produce a wrong sentence, never a lost image — pinned end to end by a case matrix asserting positive-note <=> image-in-files[]. Also here, because they are the same mechanism or the same function: - The note is no longer whitelisted back out of the digest with a regex over the result BODY. A tool result is untrusted text: any fetched page or file could assert an unbounded attachment count, and the matched line was deleted from the digest rather than kept, so it could also make one of its own lines disappear. Both write sites now go through writeToolResultMediaNote, which records the exact line on a non-enumerable property; the digest strips only that line, only its last occurrence, and takes the count only from there. A forged line now stays visible and counts for nothing. - One writer for both paths means the twins can no longer diverge in text. The comments claiming they "write the MISMA nota" were true only for the current turn; that is now stated, and enforced by construction rather than by comment. - flattenAnthropicMessages' tool_result branch kept `text`, diverted `image` and dropped everything else with no note, no droppedBlockTypes and no warning, so a result whose only block was something else folded to `(empty)`. Over the same 1,564 real sessions: 37 image-only results (the case 03b0587 fixed) and 326 `tool_reference`-only results — 8.8x more frequent, still folding to "the tool returned nothing" under a caption that tells the model to reuse it. The branch is now symmetric with the top-level image branch 20 lines below: what we cannot represent is announced, and an image the bypass cannot convert (source.type 'file', base64 with no data) no longer dies in .filter(Boolean). - Corrected the rationale at toolResultMediaNote: HARVEST_MEDIA_CAP does not slice the harvested array (it cuts the traversal) and a dedupe hit means the identical URL is already on the last message. Neither can drop a delivered image; the earlier-turn case is the real and sufficient reason. Tests: 874 pass, 0 fail, 116 suites (854 baseline at a9c9f57 + 20 added: 2 here, 7 for the block types, 7 for the forgery, 4 for the cache read in the next commit). Every new assertion was run against a9c9f57 first: 19 fail there, and the behavioural ones fail for the stated reason — the previous-turn body says `[1 image returned by this tool]` with files[] empty, `tool_reference` folds to `(empty)`, and a forged `[9 images returned by this tool]` renders as `line one line three (9 images)`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… ship null 3f0f48b taught the upload cache to expire, and expiry in file mode means `fs.unlinkSync` — the first code in src/ ever to remove a cache entry. Before it, "does it exist" and "read it" could not disagree. The upload path asked both separately: if (cacheIsExist(sig)) return item(getCache(sig).url) and shipped whatever `.url` came back. When the entry dies between the two calls — a second PM2 worker in CACHE_MODE=file, which is what the README recommends for Docker, or simply the TTL crossing in between — `getCache` returns `{status: 404, url: null}` and the assembled body carries `{"type":"image","image":null}`: an image the model never sees, with no error on our side. Reproduced by stubbing the two calls to disagree. One lookup now, and its status is checked. This also drops the second readFileSync/statSync that file mode paid on every hit, since getCache already calls cacheIsExist internally. Tests: +4 (the three disagreement shapes — vanished, unreadable, present but empty — plus a real-hit regression that the cache is still used). All four fail at a9c9f57, the first with "un fallo de lectura del cache tiene que volver a subir". Full suite 874 pass / 0 fail / 116 suites. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The acceptance gate for this bug has now shipped twice with an oracle that could not fail, and the second attempt was the one that was supposed to fix the first. - The fixture was named magenta.png and the image was magenta. 5e62b00 stripped that filename from the ANSWER, but it still travels in the PROMPT — inside the folded call and inside the ledger line `Rfym21#1 Read {"path":"magenta.png"} -> ...` — so a model echoing the name still scored as having seen the image. Measured: with that filename, the cell where the image is provably never uploaded scored SEES_IMAGE 2/2. Fixtures are now `file_7f3a.png` (no colour at all) and, in the new HN cell, `azul.png` over magenta pixels, so filename-echo is directly observable rather than merely excluded. - Every cell asserted delivery, and cell H is structurally incapable of failing the way the fix failed: it only ever gets more confident. New cell J is the same conversation one turn later, where the twin scans deliberately do not re-upload and files[] is empty (verified offline through buildInternalRequest: H -> files=1, J -> files=0), so NO_IMAGE is the only correct answer. J is a hallucination control, and it is the shape ~94x more common in real sessions. - Every cell now prints PASS/FAIL against a declared expectation instead of a bare observation, plus stop_reason and the tool_use names, so a turn that answered with a tool call is not mistaken for a lost image. - The scorer is exported and pinned by tests/probe-toolresult-oracle.test.js (+5). An acceptance gate with no test of its own is exactly how a broken gate ships; `score('magenta.png', 'magenta.png')` must not be a sighting, and now a test says so. NOT RUN LIVE. Twelve upstream attempts across ~20 minutes, including a 5-token no-image control, all returned HTTP 500 with ret ['FAIL_SYS_USER_VALIDATE', 'RGV587_ERROR::SM::…被挤爆啦…']: an account-level rejection, not shape-specific. The J cell's expectation is therefore argued, not measured — see the note in 08e504a for what WAS measured (the positive note on a previous turn, 2/2 fabrication). Full suite: 879 pass, 0 fail, 117 suites (854 baseline at a9c9f57 + 25 added). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…top it faking closes Repairs three findings that adversarial verification confirmed against the previous commit (a9c9f57). Each one is reproduced, fixed, and pinned by a new test. 1. RESIDUE LEAK (found independently by all three verifiers, the worst of the set). The tolerant unwrap splices the tags out of the MIDDLE of the text, so the delivered string stopped being a contiguous substring of `cleanedText`. `rebaseResidueSpans` located it with `source.indexOf(target)`: that returned -1 for every tolerated round and DROPPED ALL SPANS, and `peelDeliverableText` returns the text untouched when the span list is empty — so a raw `[END TOOL CALL]` reached the client as assistant text again, reversing plan Task 7 (20 leaks measured over 192 real sessions). Reproduced end-to-end through `handleStreamResponse` on this branch's HEAD: round text: 'Ya inspeccione el archivo.\n[END TOOL CALL]\nEso es todo.\n\n<agent_final>Listo</agent_final>' before: spans=[] delivered='...\n[END TOOL CALL]\n...' LEAKS=true after: spans=[{text:'[END TOOL CALL]',at:27}] delivered has no marker LEAKS=false The unwrap now returns the surviving `segments` (`from`/`to` in source coordinates, `at` in output coordinates) and the rebase moves each span arithmetically through them. `indexOf` stays as the path for contiguous results (exact wrapper, `bare`, `invalid_control`), ambiguity guard included. Both modes still revalidate every span against the destination, so anything that straddles a removed tag is dropped rather than mis-deleting. 2. THE `bare` VETO WAS BREACHED. The previous commit claimed it did not overturn that policy; in effect it did. Any prose carrying one balanced pair ANYWHERE was promoted from `bare` (vetoed) to `final` and delivered with finish_reason=stop on attempt 1: 'Next I will read the file and then emit <agent_final>the summary</agent_final> when done.' -> {kind:'final'} -> 200, delivered as a finished task. A plan is not a conclusion. The close tag must now be the LAST non-whitespace of the message: a tag with text after it is an incidental mention, not a close. This accepts 3/3 of the live-measured shapes (reasoning prose, then the pair at the end) and the proxy's own image markdown, which is always PREPENDED (`pendingImages` is flushed the moment the answer channel starts). Prose AFTER the close, and a pair inside a code fence, go back to `invalid_control` — the pre-a9c9f57 behaviour; neither was ever observed live. 3. THE invalid_control GIVE-UP IS REMOVED, not narrowed. Its justification ("the model DID declare the close, only the wrapper was mis-shaped") is false for nearly everything it fired on: after the end-anchor, what still lands in invalid_control is exactly the shapes that declare no readable close — unbalanced, reversed, doubled, both families at once. It delivered 'task complete' + 'I need your DB password' as one completed turn, and '<agent_final>half wrapped answer' (no close tag at all) as stop. It was not gated on real exhaustion either: the loop's `break` also fires when there is no requestSender, so the FIRST malformed round shipped as stop with attempts=1. And on SSE it was unreachable for its own test's shape, because streamed text trips the 422 first. config/index.js:58 stands: exhaustion fails explicitly. The tolerant unwrap accepts the measured shape on attempt 1, which is what actually fixed the 429. Also corrected, all claims rather than code: - "CERO fueron `bare`" (commit message, source comment, test header) was falsified by a later n=5 run on the same cell that DID log a `bare` rejection. The family is minority, not ruled out. The veto stays — an exhausted `bare` is still 502. - "doubled shapes still reject" was false. A string that BEGINS with the open tag and ENDS with the close is absorbed whole by `unwrapExactTag`'s lazy body. That is pre-existing and untouched; it is now pinned as a documented known gap so the neighbouring test is not misread as covering it. - "openai-agent-runtime.js:703 is the only 429 reachable from /v1/chat/completions" was false: chat.image.video.js:97 returns 429 on upstream RateLimited, reachable via routes/chat.js for t2i/t2v/image_edit and as the default fallback. The conclusion (this 429 was ours) rests on the gate's own warn in the logs, not on that grep. Hardening while here: the open/close offsets come from a lowercased copy, and toLowerCase can change a character's LENGTH (U+0130 -> 2). Those offsets now also rebase residue spans, so a skew would bite the answer. They are validated against the original before any cut; a mismatch rejects the round instead. Not touched, by construction: src/utils/chat-helpers.js and src/controllers/anthropic.js are absent from this commit, so both twin media scans stay byte-identical and the image-delivery invariant holds. tool-prompt.js:44-52 injection boundaries untouched. buildAgentTurnDirective is byte-identical (1513 B), so per-request prompt cost is unchanged; the angle form is not re-taught. Live verification was NOT possible: the Qwen account is WAF-challenged (upstream_waf_challenge, ~700 ms, rejected at the edge before generation) on every request, with and without an image payload. 8 upstream requests spent confirming that, 0 reached the model, no 429/RateLimited. Probe-matrix cells C and F remain unverified against real Qwen for this change and should be re-run when the account recovers. Tests: 889 pass, 0 fail (879 baseline at 8411eb4 + 10 added), run serially with --test-concurrency=1. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…in scans diverge
El tope de medios por turno estaba declarado DOS veces como literales independientes:
chat-helpers.js#harvestCurrentTurnMedia (ruta OpenAI) y el bucle en linea de
anthropic.js#buildInternalRequest (ruta Anthropic, la que corre Claude Code). Nada los
relacionaba. Un mutation test bajando el de anthropic.js de 4 a 2 —— media entrega de
imagenes menos en la ruta del usuario —— dejaba las 889 pruebas en verde.
Ahora la constante vive una sola vez y se exporta; anthropic.js la importa.
El guardian no compara numeros: mete UNA entrada (6 imagenes pegadas, la forma que manda
Claude Code) por los DOS barridos y exige la misma entrega. Comparar constantes habria
pasado igual con un barrido que dejara de respetar su tope.
Mutantes verificados —— cada uno aplicado, visto fallar, revertido:
A anthropic.js redeclara `= 2` -> 5/6 fallan (parity + single-literal)
B la constante compartida 4 -> 2 -> 2/6 fallan (pin de capacidad)
C el barrido ignora el tope, constante -> 2/6 fallan (SOLO lo pilla la parity;
intacta un test de constantes habria pasado)
Ademas clava el desacuerdo que anthropic.js:390 documentaba sin probar: `delivered` en la
nota se decide por POSICION (currentTurnStartIndex) y el corte del tope ocurre despues, en
el barrido, asi que un turno de 6 resultados con imagen produce 6 notas positivas y solo 4
imagenes en files[]. Se clava tal cual esta hoy —— es un pin de un agujero conocido, no una
afirmacion de que este bien. Arreglarlo cambiaria comportamiento y queda fuera.
Sin cambio de comportamiento: la unica diferencia funcional es de donde lee el 4.
Suite: 889 + 6 = 895 tests / 119 suites / 0 fail. Confirmado por suma por-fichero
(41 ficheros = 895), que no sufre el subconteo bajo carga.
Mutation testing cut buildToolHistoryLedger's maxEntries default from 40 to 3 and all 895 tests stayed green. The mechanism was tested; its reach was not — and reach is what decides whether the ledger sees the call the model is about to repeat at all. With the default at 3 the block would name 3 of every 40 executed calls and nothing would chirp. The two existing cap tests both pass maxEntries EXPLICITLY (3 and 40), so neither one exercises the default. This test does, at the boundary: - 40 distinct executed calls -> 40 entries, no omission note (claiming omissions when the list IS exhaustive pushes the model to re-call "just in case" — the bug inverted). - 41 -> exactly 40 entries plus the note, and the retained ordinals are Rfym21#41..Rfym21#2. The one that goes is the OLDEST; keeping the old ones would be the worst possible split, since the call about to be repeated is the last one. - The same 41 with maxBytes at 1e6 still yields 40, so the assertion measures the ENTRY cap and not the byte cap. Without that separation the test would stay green while silently measuring the wrong limit the day an entry grows fat enough for the 6000 B cap to bite first (today the block is ~1.8 KB). Verified by mutation: 40 -> 3 fails this test and only this test (896 tests, 895 pass, 1 fail); reverted, 896/896. No source change — the value itself is left at 40 deliberately; moving it needs measurement first. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The driver is the instrument that decides whether numbering the folded tool
history and injecting an executed-call ledger reduce duplicate tool calls. It
was untracked and untested, and a design review found it manufactured empty
results at high rate — including on the most natural call for its own task.
Reproduced before fixing, on the driver as it stood:
Grep{pattern:'legacyFormat\(', glob:'src/**/*.js'} -> "No matches found"
Bash: grep -rn 'legacyFormat(' src/ -> "No matches found"
Bash: sed -n '200,400p' src/core/pipeline.js -> "(no output)"
Bash: wc -l src/core/pipeline.js -> "23" (repo file count)
Bash: cat src/core/pipeline.js -> first 60 of 1278 lines
Read{limit:500} -> 200 lines, no marker
Two root causes. The grep scope was built by deleting every '*' from the
pattern, so 'src/**/*.js' became the literal prefix 'src///.js' and matched
nothing; and an invalid-regex catch returned NO MATCH rather than falling back,
while the task's own target string 'legacyFormat(' is an invalid JS regex that
grep matches literally. A harness that tells the model a file is empty and then
counts the re-read as a duplicate is measuring itself.
Simulator: proper glob semantics (** spans directories, * does not, a slashless
glob filters the basename) in ONE implementation shared by Glob, Grep's filter
and grep --include; literal fallback for an uncompilable pattern; wc -l counts
the named file; cat/head/tail honour -n and direction; sed -n and awk NR page;
clustered flags (-rln) are read letter by letter; the stage splitter respects
quotes, so `awk 'NR>=10 && NR<=12' f` is no longer shredded at the &&; pipelines
evaluate left to right, so `grep pat X | wc -l` counts matches rather than X's
lines; Read honours its limit and marks truncation; past EOF returns a
Read-shaped notice; and an unrecognised command returns an ERROR, never silence.
Silence is a lie the model cannot detect; an error it can react to.
Metric: cross-turn duplicates are separated from same-message repeats (the
corpus definition is cross-turn, and same-message was measured at exactly 0);
`seen` is now read for a whole turn before being written, and a repeat inside
one message consumes one ordinal rather than two. On the OpenAI path unparseable
arguments no longer collapse to {}, which had made two DIFFERENT malformed calls
duplicates of each other. The key now names the same equivalence class as the
proxy's ledger, pinned against canonicalJson.
Instrumentation the analysis needs: gapTurns/gapDistinct back to the original
(raw, so an in-ledger-window flag can be applied post-hoc at any window size),
per-call `finish`, distinctRatio, full provenance (driver content hash, git head,
argv, task id, seed), and distinctTargetsCovered to match arms on task PROGRESS
rather than on turn index — post-fix injects ~6 KB more and crosses the
externalisation threshold earlier, so the same turn index is not the same amount
of work done. Progress is now declared by each tool result rather than inferred
by scanning its body for anything path-shaped, which had let one
`Glob src/**/*.js` jump coverage to 21/23 without a line being read.
Tasks: the old single task — find every call site across 23 files — has an
expanding frontier, distinct/total ~1.0. In the corpus that ratio is 0.99 in
sessions below 5% duplicates and 0.62 above 25%, so it was a LOW-duplicate-regime
task, which explains its 20-call/0-duplicate null at least as well as low power
does. Replaced with five bounded-revisit tasks differing in traversal order and
in what drives the revisit, so one behavioural quirk cannot carry all five.
The import graph is now a DAG by construction: the first version drew edges
uniformly and produced 99 cycles reachable from core/pipeline, which makes the
trace task unanswerable and would have had the model looping forever while the
harness scored every lap as duplicates the fix failed to prevent.
MAX_TOKENS 1024 -> 4096: agentic turns truncated, and stop_reason precedence
under truncation is itself one of the changes under test.
--self-test runs the calls the tasks plausibly generate, fails on any empty or
error, and separately checks the answers are TRUE — a full but wrong answer is
worse than an empty one. It spends zero upstream requests and must be run first;
the absence of exactly this check is why 15 requests were wasted once. It has
already earned its place: it caught two regressions introduced by this very
change before either reached the wire.
Suite: 896 baseline + 56 new = 952 tests / 119 suites / 0 fail, confirmed by
both `npm test` and the per-file sum.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Qwen answers an exhausted daily quota with `data.code = 'RateLimited'` and the
text "You've reached the upper limit for today's usage.". Neither path told the
client what that was:
/v1/messages HTTP 500 {"error":{"type":"api_error"}}
/v1/chat/completions HTTP 502 {"error":{"type":"upstream_error"}}
Both read as "the server is broken", so an agentic client retries against a wall
and burns another account from the pool on every turn. The native APIs answer 429
for exactly this reason. Observed live in the user's own sessions (2026-08-21).
The detection now lives once, in utils/upstream-error.js (isRateLimitError /
rateLimitRetryAfterSeconds / describeUpstreamFailure), and each controller only
translates it to its own wire shape: Anthropic 429 `rate_limit_error`, OpenAI 429
`insufficient_quota`. Both wire names sit next to each other in that file so the
twins cannot drift apart.
Mid-stream the HTTP status is already committed, so the error event (Anthropic)
and the error frame + [DONE] (OpenAI) carry the signal instead. On the OpenAI
agent path that is the ONLY channel: handleOpenAIAgentStream writes its opening
role delta before consuming upstream (chat.js:407), so a 429 status is
unreachable there by construction — pinned by a test that says so.
Retry-After only when the upstream actually sourced a wait: `data.num` is in
hours, the same reading chat.image.video.js:88 already does. No field, no header
— a fabricated wait is worse than none, because the client obeys it literally.
Everything that is not a quota refusal keeps its old status, type and code,
including the gate's deliberate 502 for protocol exhaustion.
Tests: 952 -> 971 (+19 new), 0 fail, 122 suites, per-file sum; every
pre-existing file's count unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… that governs it maxBytes is what actually bounds the ledger on real traffic — a realistic entry weighs ~223 B (ASCII) to ~333 B (long path), so the byte cap bites long before maxEntries=40 does and the ledger never reaches its advertised 40 entries. The test that landed in 1c31c6f pins maxEntries and deliberately passes maxBytes: 1_000_000, so the only number production reads had no test at all. Re-measured reach over 199 real Claude Code sessions with duplicates (18,008 tool calls, 1,970 re-issues of a call already made), not one. Reach = at the moment the call is about to repeat, the ledger built from the prior history still names the earlier instance: 6,000 B 81.0% 5,315 B/request mean 9,000 B 88.7% 7,612 B (+7.8pp for +2,297 B = 67 onsets/KB) 12,000 B 91.4% 9,276 B (+2.6pp for +1,664 B = 31 onsets/KB) 16,000 B 95.3% 11,836 B (+3.9pp for +2,560 B = 30 onsets/KB) 24,000 B 96.5% 14,761 B (+1.3pp for +2,925 B = 9 onsets/KB) uncapped 100.0% 30,091 B (+1.9pp for +12,549 B = 3 onsets/KB) The single-session curve did not generalise, again: the worst session (33f8544e) reads 59.8% at the shipped default, the 199-session aggregate reads 81.0%. Two of the three mechanisms killed in this effort died of exactly that error, so the value is chosen on the aggregate. The curve bends after 16,000; 12,000 sits on the steep part and buys 10.4pp over the old default. Raising maxEntries instead buys almost nothing: at 12,000 B, 40 -> 60 entries moves reach 91.4% -> 91.8%, so maxEntries stays 40 and its pin stays untouched. The externalisation-threshold objection turned out to be the weak half of the argument. Over 25,576 real request boundaries, 71.2% were ALREADY past the 90 KiB threshold at the old cap (the median conversation is 169 KB), and going to 12,000 newly pushes 116 of 25,576 = 0.45% of requests across. That is where the bytes are needed: 76% of re-issues happen at already-externalised boundaries, where reach was 77.0% at 6,000 and is 89.4% at 12,000. Body-size delta measured through the real buildInternalRequest on all three shapes: fresh turn 6,033 B unchanged (no tool history, no ledger); no-tools-with-tool-history 267,773 / 1,249,735 B unchanged (the hasTools gate holds); with tool history +4,382 B (+1.57%) at a 120-message prefix and +5,020 B (+0.40%) on the full worst session. The new test is a REACH test, the lower bound the suite never had: 60 heavy calls that overflow the cap must still yield >=30 of them, newest-first, plus the omission note, and it asserts <40 lines so it cannot silently drift back into measuring maxEntries. Mutation drill on the default: 6000 fail, 9000 fail, 11000 pass, 12000 pass, 16000 fail, 24000 fail — pinned on both sides. The two neighbouring "block never exceeds its cap" assertions had 6000-era ceilings hardcoded (8192 and 6000); they now read the same constant, so a default raised without measuring breaks them too. Per-file test sum (the authoritative count; the glob run under-reports): baseline at 08c724f = 971 across 43 files, 0 fail. Now 972, 0 fail. Exactly one file changed count: tool-repetition 48 -> 49. 971 + 1 = 972. This run npm test agreed at 972 / 122 suites / 0 fail. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`npm test` reported 944, 970, 975, 983 and 986 tests on a byte-identical tree, every one with `ℹ fail 0` and exit code 0. A run 42 tests short was indistinguishable from a pass, so every quality claim in this branch rested on a gate that lies. Cause, confirmed rather than guessed: node propagates `--test-force-exit` to the child process it spawns per test file — it is visible in the child's `process.execArgv`. When a child finishes it calls `process.exit()`, which does not flush stdout still buffered for the pipe back to the runner, so a tail of that child's reporter output is discarded. The runner counts what arrived, sees no failure, exits 0. Reproduced with no project code at all: two throwaway files (3000 tests + 3 tests) under `--test-force-exit` reported 3000 on 1 of 5 runs, the 3-test file vanishing whole; without force-exit the same pair reported 3003 every time. Dropping `--test-force-exit` is not available: 26 of the 43 files in tests/ never exit without it, because `src/utils/account.js` starts ref'd `setInterval`s at module load and everything reaching `chat-helpers.js` inherits them. The suite then hangs forever instead of finishing short. So `npm test` now runs tools/test-gate.js, which checks the reported counts against tests/expected-counts.json. Short run -> retried (the loss is a race, so a genuinely deleted test is short on every attempt while a truncated one is not) -> still short -> exit non-zero with the arithmetic spelled out. More tests than expected also fails, so the baseline cannot rot. No summary, a nonzero runner exit, or a watchdog timeout all fail loudly too. Also fixes `npm test -- tests/foo.test.js`, which used to be a lie: the package script's glob won and the whole suite ran anyway. The filter is now honoured and the gate says it skipped itself. Baseline: 972 (per-file sum at e5babf7) + 14 new gate tests = 986 tests / 122 suites / 0 fail, verified by two independent per-file sums.
…t the 429 Repairs 08c724f, which failed adversarial verification on two counts. 1. THE HEADLINE WAS FALSE FOR THE PATH THAT PRODUCED EVERY INCIDENT. "map ... to 429 on both API paths" does not hold on /v1/messages with stream:true — the only mode Claude Code runs, and the mode of every quota refusal in the user's logs. handleAnthropicStream sets the headers (:1203) and writes message_start (:1211) before reading one upstream byte, so res.headersSent is already true when the quota packet arrives and the catch (:2665) can only take the event branch. The 429 is unreachable there by construction. That is not a defect to paper over: committing the response early is what makes in-protocol `ping` legal (:1156-1160), which is how the bridge's false "dead stream" was killed. The reachability matrix now lives in upstream-error.js and is pinned by tests, so the claim cannot be made again: /v1/messages stream:false 429 body.error.type /v1/messages stream:true 200 event error.type <- the user's path /v1/chat/... llano stream:false 429 body.error.type /v1/chat/... llano stream:true 429 body.error.type /v1/chat/... agente stream:true 200 frame error.type Also fixed: the previous test comments credited the OpenAI agent path to Claude Code. Inverted — Claude Code speaks /v1/messages. The consequence was a real hole, not just wording. When the event is the only channel, everything the Retry-After header would have carried has to travel inside it, and the wait was being dropped. `retry_after` now rides in the Anthropic error EVENT and the OpenAI error FRAME, on the same rule as the header: only when the upstream actually sourced it (`data.num`, in hours). 2. THE BURN LOOP THE COMMIT CLAIMED TO FIX WAS NEVER CLOSED. Its own justification was that a wrong status makes the client "burn another account from the pool on every turn" — then nothing marked the exhausted account. HTTP 4xx/5xx go to recordError, which deliberately does not cool down, and the quota refusal is not even an HTTP status: it arrives inside a 200 SSE body that request.js never inspects. Better status, same server-side loop. AccountRotator gains recordQuotaExhausted() — a third class beside recordError (no cooldown, account still valid) and recordFailure (needs maxFailures of evidence). One refusal is conclusive. The account leaves the draw until the upstream's own wait elapses, or one hour when it gave none: long enough to break the hot loop, short enough that a misclassification cannot strand a good account for a day. resetFailures deliberately does NOT lift it — account.js:516 calls that for every account on the token-refresh timer, which would revive the loop by itself. Both controllers report through one shared helper so the twins cannot drift. Dashboard consequence, closed here too: status.kind reads cooldownEndsAt, which only the failure counter sets, so a benched account would have shown "active" while rotation skipped it. cli-support now merges both cooldowns, latest wins. Tests: 986 -> 1009 (+23), 122 -> 127 suites, 0 fail, per-file sum over 44 files; only tests/upstream-quota-429.test.js moved (19 -> 42), every other file's count byte-identical. The gate's own whole-suite run agrees at 1009/127. All 23 were watched failing first. NOT CLOSED — ccproxy, out of this repo. The user's transcripts hold 154 lines of `Upstream error mid-stream: 500 You've reached the upper limit for today's usage.`, and every one of the 281 mid-stream lines reports 500, whatever the event said. The bridge synthesizes a fixed 500 and does not branch on error.type, so this change does not reach Claude Code as a 429 until ccproxy maps rate_limit_error itself. What does survive today is the message text and now the wait. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…hes the model
La verificacion adversarial tenia razon y el numero es peor de lo que estimaba.
Subir el tope a 12.000 con el ledger dentro de `envelope.prefix` no mejoraba nada:
EMPEORABA lo que el modelo lee, por debajo del 6.000 de partida.
Medido sobre 286 sesiones reales de Claude Code, 1.604 reemisiones de una llamada
ya hecha (el 96,1% en fronteras ya externalizadas). Alcance = con la llamada a punto
de repetirse, el ledger construido con la historia PREVIA todavia nombra la instancia
anterior; "bloque" es lo que el ledger contiene, "inline" es lo que sobrevive a
buildAgentContextLivePrompt, que es lo unico que el modelo lee:
bloque inline
antes, cap 6.000 73,3% 73,3%
antes, cap 12.000 84,7% 39,7% <- la subida, sola, costaba 33,6 puntos
ahora, cap 12.000 84,7% 84,7%
En las externalizadas solas: 72,2/72,2 -> 84,0/37,4 -> 84,0/84,0.
Causa, exactamente la que se reporto: el prefijo se retiene inline recortado por
CABEZA Y COLA (headRatio 0.55) y el bloque va del mas nuevo al mas viejo, asi que la
rebanada de cola conservaba sus entradas VIEJAS y el hueco compactado se comia las
NUEVAS — la llamada que esta a punto de repetirse, la unica razon por la que el bloque
existe. Con 6.000 B cabia entero en esa cola por casualidad aritmetica; con 12.000 no.
No era pre-existente: lo creaba la subida.
Arreglo, en el orden que se pidio — primero la colocacion, despues el tope:
buildBudgetedAgentPrompt separa el ledger del prefijo y le da seccion propia, con
presupuesto reservado ANTES del reparto por pesos (tope: un cuarto del pool, que en
produccion no muerde) y recorte propio por renglones enteros, conservando los MAS
NUEVOS y con la nota de omision puesta. El tope de 12.000 y el pin de maxEntries=40
de 1c31c6f se quedan como estaban.
Rejilla de 48 sobres externalizados (94-384 KB crudos, 8-60 herramientas, 30-120
llamadas, system prompt de 3 a 50 KB): antes 0 de 48 completos —sobrevivian 23-24 de
30-37 entradas y en 44 de 48 faltaba la mas nueva—; ahora 48 de 48 completos. Cuesta
~2,9 renglones de historia reciente inline (12,0 -> 9,1 de media): ~2,5 KB por renglon
de resultado crudo a cambio de ~333 B por llamada nombrada, y 13-14 llamadas mas
nombradas. Se acepta a sabiendas.
Un agujero propio del arreglo, encontrado y cerrado aqui: reconocer el bloque solo por
su cabecera —o por cabecera + leyenda— dejaba que un texto del cliente que las imitara
se llevara el trato del ledger; sin ningun renglon `#n `, el recorte por renglones lo
dejaba en '' y borraba ~11,8 KB de reglas del cliente del prompt inline. Ahora hacen
falta las tres cosas: cabecera, leyenda literal y un renglon de entrada detras.
tests/tool-repetition.test.js 49 -> 53. Las cuatro nuevas se vieron fallar antes de
pasar: la de supervivencia inline falla en 41b2e4d con "la entrada mas nueva (Rfym21#60)
desaparecio del prompt inline; sobrevivieron 23 entradas Rfym21#46..Rfym21#24"; la de cabecera +
leyenda falla al quitar la tercera condicion. Ninguna prueba podia ver esto: todas
llamaban a buildToolHistoryLedger directamente y ninguna pasaba el bloque por el
presupuesto.
Tambien: el guardia del test del tope de entradas ya no compara contra un numero
escrito a mano (decia `< 4000` con el default en 6.000 y siguio verde al subirlo a
12.000, con el margen sin vigilar) sino contra el mismo bloque construido sin tope de
bytes — una igualdad que no envejece. Y se corrigen los dos comentarios de
anthropic.js y chat-middleware.js que afirmaban que el prefijo "nunca se externaliza",
que es la creencia que causo el fallo.
Suma por archivo, con guardia de "ningun archivo sin contar": 1009 (41b2e4d, 44
archivos, 127 suites, 0 fail) - 49 + 53 = 1013. Medido: 1013 / 127 / 0 fail, 44
archivos, ningun archivo a cero. Gate y lint en verde. 0 peticiones upstream.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adversarial verification found the gate shipping two defects of its own, one of them attacking the exact property the gate exists to guarantee. A third, in the escape hatch, turned up while measuring the fix. 1. NOT_ALL_RAN. `evaluate()` read only tests/suites/fail. It parsed pass/skipped/todo/cancelled and asserted on none of them, so a disabled test — which keeps its slot in `tests` — was invisible. Marking all six tests of tests/harvest-media-cap.test.js `it.skip` (the file guarding the image-delivery twin-scan invariant) produced a PASS banner byte-identical to an honest run's, exit 0, `skipped 6` printed right there in the output. Unlike the truncation race this is deterministic: it survives all three retries and gets committed. Now checked as pass + fail === tests, before the count checks so a skipped-and-truncated run reports the cause that will not go away, and never retryable. Verified on node v24: tests = pass + fail + skipped + todo + cancelled; suites are not in pass, and a `todo` test is not counted as pass despite its check mark. 2. bless could only ratchet up, and lied when it could not. It seeded its running maximum from the baseline on disk, so deleting a test file ran three clean attempts reporting 998/124, printed "BLESSED: 1013 tests / 127 suites", exited 0, wrote nothing, and left `npm test` permanently red with no documented escape. The max now runs over the attempts alone — computeBlessed() takes no baseline argument, so it cannot be floored by one — and bless refuses any unclean run, including one with skips, so a skip cannot be laundered into the baseline. 3. CI could never reach the watchdog: 600000 ms x 3 attempts = 30 min inside a 10-minute job. TEST_GATE_TIMEOUT_MS is pinned to 150000 (~30x the ~5s the suite actually takes) and timeout-minutes raised to 15, so a hung runner is reported as TIMEOUT instead of looking like CI infra flake. 4. The escape hatch under-counted too. Every place telling you to confirm the real number shipped a per-file sum whose grep lacked `-a`; tests/tool-prompt.test.js emits bytes that make grep call the stream binary and print "Binary file ... matches" instead of that file's summary line. Measured 892 where the truth was 1025 — off by exactly its 133 tests, from the one command whose job is to be the trustworthy second opinion. Two pre-existing fixtures described impossible runs (tests 944 with pass 972). A real truncated run loses tests and pass together, so they are corrected to move in step; they were latent nonsense that only became load-bearing once pass was asserted on. Zero production files touched: `git diff --stat -- src/` is empty. The image-delivery invariant, twin-scan lockstep, injection boundaries and the angle-form ban are untouched by construction, and their guarding tests pass. Tests 1013 -> 1026 (test-count-gate.test.js 14 -> 27, +13; suites 127 unchanged). Per-file sum, twice: 1026 tests / 127 suites / 0 fail / 0 skipped / 0 todo. Gate green 3/3 with zero retries needed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ever measured buildToolHistoryLedger's maxBytes default went 6000 -> 12000 on a REACH curve: 81,0% -> 91,4% of re-issues end up named by the block. Reach is a necessary condition for the block to work, not evidence that it works, and the effect was never measured -- there has never been an arm with the block off. Its intervention CLASS is measured, though. The corpus carries a natural experiment: a client hook that replaces the tool_result with "Wasted call -- file unchanged since your last Read. Refer to that earlier tool_result instead.", 364 firings across 79 sessions. It is a strictly stronger version of what the ledger says -- it sits inside the result the model just asked for, names the specific offence, 95 B, impossible to miss -- and conditioned on the population it fires in (Reads that are already re-issues) it is null: train hooked 162/253 = 64,0% unhooked 185/313 = 59,1% RR 1,08 [0,90, 1,42] holdout hooked 34/79 = 43,0% unhooked 36/80 = 45,0% RR 0,96 [0,59, 1,86] Per-session sign test: 26 up, 12 tied, 22 down. One key carried 32 warnings and the loop survived all 32. The train CI upper bound rules out any large benefit. A weaker restatement further up-context cannot do more. The cost is real. Measured on both paths, three shapes: fresh turn (tools, no tool history) ledger 0 B -> 0 B 0 with tool history ledger 11.887 B -> 5.883 B -6.004 B no-tools + tool history ledger 0 B -> 0 B 0 Plus, on an externalised request, the block takes up to a quarter of the inline pool (LEDGER_POOL_SHARE): ~2,9 lines of recent inline history, ~2,5 KB of raw result each = ~7 KB of actual tool results evicted to name calls at ~333 B. That is where 76% of the re-issues happen. 6000 and not 0: there is evidence the benefit is unmeasured and that its closest analogue is null, but no evidence the block does harm. 6000 halves a certain cost against an uncertain benefit and keeps the artifact so it can be A/B'd for real (randomised BY SESSION, not per request). The dedicated inline section from 258a658 stays -- that fix is independently correct (it made the NEWEST entries survive the budget instead of the oldest) and lowering the cap does not undo it; the block simply fits with more room. New test measures the MECHANISM, not the constant: it counts the ledger's bytes in the assembled content that goes upstream, on both paths. Verified it catches a call site that passes its own maxBytes while the default stays at 6000 -- an assert.equal on the constant would not. Tests: 1026 baseline + 1 new = 1027 / 127 suites / 0 fail (per-file sum and gate agree). Floors relaxed 30 -> 15 entries: 60 heavy calls now fit 18, not 35. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…fix is truncated The 6000 B cap stands. This adds the test that was outstanding across three workflows -- does the ledger survive buildBudgetedAgentPrompt in the >92 KiB regime -- and, in writing it, found that the revert to 6000 had quietly disarmed the guard for the dedicated-ledger-section fix (258a658). Measured by putting the ledger back inside envelope.prefix (weight 34, headRatio 0.55), on the file's own externalised fixture, at the PRODUCTION inline budget of 48 KiB: cap 6.000 18 of 18 entries arrive, byte-identical -> regression INVISIBLE cap 12.000 23 of 37 arrive, Rfym21#46..Rfym21#24 -> the 14 NEWEST lost, header gone entirely So at 6000 the block fits in the tail slice by arithmetic accident, exactly as the code comment says it used to, and every assertion still passes. Confirmed against the suite: with the regression injected, the pre-existing "la entrada MAS NUEVA sobrevive" test PASSES. The only surviving guard was the neighbouring test, and only at AGENT_CONTEXT_LIVE_PROMPT_BYTES=12000, which is not production. Two tests, both of which fail with that regression injected: - survival: runs BOTH caps and asserts the block arrives byte-identical, not just that its newest entry survived. Two guards first -- request > 92160 B, and the compaction separator lands BEFORE the ledger -- so it cannot pass outside the regime it claims to measure. The 12000 arm stays although it is no longer the default: it is the only arm that watches the mechanism at the production budget. - degradation above the section ceiling: with the production budget and this envelope shape the ledger's own section tops out at ~12,8 KB. At 6000 the block uses 5.882 B (under half of it) and arrives whole; at 12000, 11.886 B, grazing it; at 24000 it no longer fits and 37 of 40 arrive -- the 37 newest, whole lines, omission note intact. That is "raising the cap moves the block towards the cut, not away from it", as an assertion instead of a story. Docs corrected where they overstated: agent-turn.js claimed the inline-survival test was what pinned this, and request.js claimed the suite catches the in-prefix regression. Both now say what is actually true, including that the 6000 arm is blind to it. Tests: 1027 baseline + 2 new = 1029 / 127 suites / 0 fail. Per-file sum 1029, gate 1029, blessed. eslint clean. No upstream calls, no dev server, no VPS. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ntarla La verificacion adversarial de 40df21e encontro que ese commit desarmo la unica guarda del arreglo de la seccion propia del ledger (258a658) y ademas escribio en los comentarios que la guarda seguia en pie. fb904db ya rearmo la guarda y corrigio los dos comentarios de src/. Reproducido aqui de forma independiente -- con la regresion inyectada, tests/tool-repetition.test.js falla 3 veces en HEAD -- y quedaban tres cosas. 1. Un tercer comentario de la misma clase seguia vivo, y en el propio archivo de tests: «ninguna pasaba el bloque por el presupuesto. Esta si.» El test que introduce esa frase corre al tope que se envia (6.000) y a ese tope NO ve la regresion: con el ledger devuelto al interior del prefijo pasa igual. 2. La correccion de fb904db introdujo una afirmacion nueva que tampoco es cierta: que el brazo de 12.000 es «el unico que vigila el mecanismo al presupuesto de produccion». Medido inyectando la regresion: el test de degradado (24.000) tambien la ve, al mismo presupuesto. 3. La ceguera se contaba como «casualidad aritmetica» de un fixture. No lo es. La rebanada de cola del prefijo mide ~7,1-7,8 KB con el presupuesto de produccion (medido: un bloque de 7.146 B cabe entero, uno de 7.778 ya no), asi que CUALQUIER bloque acotado a 6.000 B cabe. Es una relacion entre dos numeros que nadie vigilaba. El test nuevo la vigila, y de paso convierte el arreglo de 258a658 en un diferencial que no necesita parchear el codigo: entierra el bloque en el prefijo dentro del fixture con una sangria de un byte -- la cabecera deja de estar a principio de linea y splitAgentLedger no la reclama --, que es la forma que el bloque tenia antes de tener seccion propia. cap 6.000 bloque 5.882 B seccion 18/18 enterrado 18/18 IDENTICOS cap 12.000 bloque 11.886 B seccion 37/37 enterrado 23/37, Rfym21#46 se ve Reproduce exactamente los numeros que fb904db midio parcheando request.js. Visto fallar antes, con dos perturbaciones y la suite entera: - peso del prefijo 34 -> 20 (la cola encoge por debajo del tope): falla el brazo de 6.000 con «el brazo del default ha dejado de ser ciego a la regresion». Es el UNICO test de los 1.030 que lo coge. - splitAgentLedger devuelto a no partir (deshacer 258a658): falla el brazo de 12.000 con «llegaron 23 de 37 entradas». Lo cogen 4 tests, este entre ellos. Sin cambio de comportamiento: el diff de src/ es solo comentarios. Tests: 1029 + 1 = 1030 / 127 suites / 0 fail. Suma por fichero 1030 (44 ficheros, ninguno perdido), gate 1030, expected-counts actualizado. eslint limpio. Upstream: cero completions, cero dev server, cero VPS. Una unica llamada de AUTH se disparo al principio al requerir los modulos con el .env real, que es lo que hace tambien npm test en este repo; a partir de ahi todo se midio en copias del arbol en el scratchpad con ACCOUNTS vacio, que reproducen 1029 exactos en HEAD. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ript path The duplicate-onset replay harness carried a hardcoded absolute path to one developer's Claude Code session as its default transcript. Session transcripts are the operator's own work and their paths carry a username, so nothing in the repo should name one. --transcript / TRANSCRIPT now supply it and there is no default; the harness exits 2 with the conventional location instead of guessing. The test that pinned the old default now pins its absence. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Rfym21
pushed a commit
that referenced
this pull request
Sep 11, 2026
…ails under CI load CI (ubuntu-latest, 2 vCPU) reported 1035 of 1074 tests with fail 0 and exit 0 on 3 of 3 attempts for a tree that is whole (main after #167), each attempt missing the tail of different files. The same tree reports 1074 locally and on an idle 4-core VPS, and that VPS reproduces the drop (1069) as soon as CPU load is added. The loss is `node --test`'s parent runner losing the end of a child's piped output on --test-force-exit; it is load-dependent, and CI is always loaded, so the gate's retries could never save it. tools/test-gate.js: stop using the parent runner. Every file runs as `node --test --test-isolation=none --test-force-exit <file>`, its stdout is read to EOF before anything is counted, and the per-file summaries are summed (sumSummaries: a file with no summary voids the attempt as NO_SUMMARY, it is never subtracted quietly). Bounded pool (TEST_GATE_CONCURRENCY, default min(4, cores - 1)); one watchdog covers the whole attempt. --test-force-exit stays: 26 files never exit without it. .github/workflows/ci.yml: 300 s per attempt, 2 attempts (measured: ~10 s local, 77 s on the 4-core VPS at concurrency 1). tests/tool-prompt.test.js (P14): threshold 400 -> 1500 ms. Measured 50 ms on a Mac, 210-225 ms on the VPS idle, 445 ms under load; the quadratic regression it guards was ~2 s, so 1500 still catches it and stops flaking on a shared runner. tests/test-count-gate.test.js: three tests for sumSummaries. Baseline 1074 -> 1077. Verified: gate PASS 1077/128 locally; on the VPS under six CPU hogs, single attempt, 2 of 3 runs PASS 1074 and the third failed loudly on P14 — never a short pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Agentic parity work on both compatibility surfaces, driven by measurement against real Claude Code traffic rather than by inspection. 41 commits, 43 files, +12.5k/-187. Tests go 678 → 1030, all green.
The honest summary first: most of this fixes defects that were reproducibly wrong. Two commits implement a mitigation whose benefit I could not demonstrate, and I say so explicitly below rather than dressing it up.
Bugs fixed, each reproducible before and verified after
Anthropic protocol shape
stop_reasonreportedtool_useon a turn upstream truncated withlength, so clients executed a call the model was cut off mid-emitting.max_tokensnow wins.tool_use.idused thecall_prefix while the inbound path already mintedtoolu_— two id namespaces in one controller./v1/chat/completionsstill emitscall_.toolsarray (ortool_choice: "none"), history folding was skipped, so an assistant message carrying onlytool_userendered as empty content and the whole assistant turn vanished while itstool_resultsurvived with the nonexistent role"tool". Claude Code's compaction requests have exactly this shape.thinking/redacted_thinkingblocks were dropped, discarding the model's own record of why it made the previous call.Images — an image inside a
tool_resultnever reached the model. The image was uploaded and delivered correctly; the folded text claimed the read had returned nothing, so the model was told in words it got nothing while an unreferenced image sat infiles[]. Also: an upload-cache entry that vanished mid-read shippednull, and the cache TTL was a guess rather than the URL's own signed expiry.Quota and errors — Qwen's
RateLimited("upper limit for today's usage") surfaced as HTTP 500 on/v1/messagesand 502 on the OpenAI path, so a client could not distinguish quota exhaustion from a server fault and retried against a wall. Both now return 429 in the native shape for their surface. Separately, the OpenAI turn gate manufactured a 429 out of a correct answer via an over-strict wrapper unwrap, and a dead account could be re-picked immediately after being marked dead.Residue —
stripToolCallResidueran at four sites on the Anthropic path and none on the OpenAI one, which computed residue spans and discarded them. Measured leak rate in real traffic: 20 turns in 29,352.Test gate —
npm testprinted 944/970/975/983/986 on byte-identical trees, every time withfail 0and exit 0, so a run 40 tests short was indistinguishable from a pass.tools/test-gate.jspins the expected count intests/expected-counts.jsonand fails loudly on a short run. Every count in this PR is a per-file sum, not one aggregate run.Invariant hardening —
HARVEST_MEDIA_CAPwas two independent literals in two files with nothing relating them; halving one left the entire suite green. Now one exported constant with a test that fails when the twin scans diverge.The part I could not prove
Two commits number folded tool calls so results are addressable, and inject a ledger of already-executed calls. They were built on the hypothesis that duplicate tool calls come from the model being unable to tell which
[TOOL RESULT: Read]answered which call.That hypothesis did not survive measurement, and neither did two successors. A labelled dataset of 15,336 real tool calls across 192 sessions, six independent analyses and twelve adversarial validators produced: label collision is a correlate, not a cause (a call carrying 40 colliding same-name calls did not repeat on unfixed code, and a live differential probe — "five Reads, what did the second return" — passes identically with and without the numbering); the 90 KiB externalisation threshold does not discriminate (duplicates are 18/18 above it and first-time calls 163/163); retry-after-empty-result is refuted with the sign reversed.
The intervention class has since been measured directly, 364 times, by a client-side hook that is a strictly stronger version of the ledger — it lands inside the tool_result the model just requested, in 95 bytes. Conditioned on the population it fires in: RR 1.08 (CI 0.90–1.42) train, 0.96 holdout. Null.
So the byte cap went back from 12000 to 6000 (
40df21e), and a test now pins what the model actually reads of the ledger after prompt-budget truncation, rather than what was written. The numbering is kept — it is protocol-quality-correct on its own terms and reverting carries more churn risk than keeping — but it should not be extended on the strength of its premise.Worth recording for anyone tempted to add hard suppression: ~41% of repeated bare-path Reads have an intervening
Edit/Writeto that file. Suppressing repeats would break correct behaviour.Testing
npx eslint src tests toolsclean.toolu_/call_split confirmed on the wire by running the same probe against builds with and without these commits.tools/dev-probes/are instrumentation, not tests. Several of them exist to record that a design cannot reproduce the bug it was built for.Notes for review
src/utils/tool-prompt.js,agent-turn.jsandchat-helpers.jsare shared, andcontrollers/anthropic.js/utils/openai-agent-runtime.jsare twins..env.exampledocuments the ledger cap.<tool_call>angle form is never re-taught — Qwen's platform intercepts it and injectsTool <name> does not existsinto the model's context. Pinned by tests.🤖 Generated with Claude Code