Skip to content

fix(tools): make native /v1/messages tool calling work by moving off Qwen's intercepted <tool_call> delimiter - #164

Merged
Rfym21 merged 26 commits into
Rfym21:mainfrom
maxff77:feat/anthropic-claude-code-compat
Sep 1, 2026
Merged

fix(tools): make native /v1/messages tool calling work by moving off Qwen's intercepted <tool_call> delimiter#164
Rfym21 merged 26 commits into
Rfym21:mainfrom
maxff77:feat/anthropic-claude-code-compat

Conversation

@maxff77

@maxff77 maxff77 commented Aug 31, 2026

Copy link
Copy Markdown

Problem

Native /v1/messages tool calling on qwen3.x-max-thinking was effectively broken: a Claude Code session against this gateway would die with 上游返回了残缺、非法或不存在的工具调用, and the model would narrate "tool infrastructure failure" instead of calling. Root cause (measured, not guessed): Qwen's serving platform runs its own server-side agent loop. When the model emits the native <tool_call> tag, the platform intercepts it, looks the name up in ITS registry (which has none of the client's tools), and injects Tool <name> does not exists back into the model's generation context. Verified: a session death correlated second-by-second with 5 role:function interceptions named Bash/Read; auto_search:false does not disable the interceptor.

Fix

Take the platform interceptor out of the loop by moving the tool-call wire format off Qwen's native delimiter:

  • Canonical wire format is now [TOOL CALL] / [END TOOL CALL], defined once in agent-turn.js so the system prompt, folded history and every retry hint teach the same shape. The platform no longer consumes the marker as a call, so no does not exists injection.
  • The legacy <tool_call> angle form stays recognized on read (RL-habit emissions) but is never taught or written.
  • Trigger-gated payload recovery (from the earlier work on this branch): the delimiter is a length-bounded trigger; the call is recovered from the JSON payload in a 128-char window behind it, with the tool name taken only from the payload's name key and the trigger required to be the first non-whitespace content — this narrows the injection surface a free-text scan would open.
  • Result bodies neutralise both marker forms (case-insensitive) so untrusted tool output cannot fire the trigger.
  • Defect A: Qwen's own role:function registry deltas are dropped from the client stream and logged, instead of being delivered as the assistant's answer.

The tool-prompt module is shared with /v1/chat/completions, so the change applies to both surfaces; the OpenAI path was re-verified after the swap.

Verification

  • Unit suite 160/160 (node --test tests/*.test.js); key guards mutation-checked (trigger regex, neutraliser case-flag, canonical constant, markdown-link suppression, bare/decorated bracket closers).
  • Live on a staging container built from this branch:
    • /v1/messages tool calling: 18/18 single + multi-turn probes, 0 does not exists in output, 0 unrecovered markers.
    • /v1/chat/completions (OpenAI format): 6/6 single, 5/6 multi requiring a fresh 2nd tool call (the one miss is the model answering early, not a format break).

Notes

  • Three adversarial review passes ran against the diff; one live bug (a /g vs /gi case mismatch in the result-body neutraliser that let an uppercase <TOOL_CALL> execute) was caught and fixed here, plus coverage and false-positive gaps. Residuals with no corpus evidence (fused [TOOLCALL], full-width bracket closers, [Tool calls] heading edge case) are deliberately left unhandled.
  • The 128-char window and recovery-% tolerances were measured on the angle-tag corpus and are inherited (documented in-code) — unvalidated for the bracket form, which the model is instructed to write cleanly.

PEDRO LOBATO CARCAMO and others added 19 commits August 30, 2026 00:59
…e answer

Qwen's backend parses the <tool_call> convention this gateway injects into a
native function_call, dispatches it against its own tool registry
(code-interpreter, amap, fire-crawl, image-generation), and when the name is
not there it streams back a synthetic result:

  {"role":"function","content":"Tool Read does not exists.","phase":"answer"}

createUpstreamDeltaNormalizer only ever looked at delta.phase, so anything
carrying phase "answer" became assistant text. Claude Code users saw
"Tool Bash does not exists." concatenated into their answers, one sentence per
tool name, interleaved with tool calls that worked.

Drop role:"function" deltas in the normalizer. It is the single choke point
shared by anthropic.js, chat.js and openai-agent-runtime.js, so all three
paths are covered at once. Logged at warn level with the phase and name, so a
real drop is visible rather than silent.

Verified against raw upstream captures: 10/10 frames carrying that sentence
have role "function" and empty usage; 0/10 have role "assistant". Sibling
frames from the same registry carry code_interpreter sandbox errors, which
this also stops leaking.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Recognition required an exact `<tool_call>` opener, tolerating at most four
spaces or tabs. The model almost never writes one: across 149 decoded live
captures exactly one clean opener appears, against `<tool_call\n>`,
`<tool_call">`, `<tool_call=`, `<tool_call type="function">`, `<tool_call_id_1>`
and `<tool_call style="font-weight: bold;">`. The JSON payload inside is
almost always valid — only the tag is wrong, because the prompt teaches
`<tool_call>` next to an attributed `<tool_response tool_call_id=… name=…>`
that foldToolMessages re-seeds on every tool result, and the model generalises
the decoration.

It also failed silently: a malformed tag left hasParseError() false with no
error and no log, so nothing retried and nothing was recorded.

Keep a tool_call-ish opener as a bounded trigger, stop requiring it to be well
formed, and recover the call from the first balanced JSON object within 128
characters of it. Measured over 87 captured turns that attempted a call: exact
delimiter 43, trigger-gated payload 82.

The trigger narrows the injection path but does not close it, so two further
rules carry the boundary: the tool name comes from the payload's `name` key and
never from the trigger tail, and the trigger must be the first non-whitespace
content of the visible answer. Both costs were measured (2 and 8 turns of 159)
before choosing. Tool result bodies now neutralise the result markers, so a
file containing one cannot end its own block and have its trailing text read as
instructions.

A missing `arguments` key means `{}` — a zero-parameter tool stays callable.
The streaming and whole-text paths share the trigger, gate, window and
extractor while keeping their own buffering; they agree on all 153 replayed
turns.

Live on staging: /v1/messages on qwen3.8-max-thinking 12/12, from 3/36.
Recognition misses on qwen3.8-max: 0/24. Suite 150 tests.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Qwen's serving platform runs its own server-side agent loop. When the model
emits <tool_call> — the native format our prompt taught — the platform
intercepts it, looks the name up in ITS tool registry (which has none of our
tools) and injects "Tool <name> does not exists" back into the model's own
generation. The model then narrates "tool infrastructure failure" instead of
calling. Verified: a session death at 2026-08-30 19:56 matches 5 role:function
interceptions named Bash/Read second by second; auto_search:false does not
disable the interceptor (18/18 probes passed, interceptions still fired).

The canonical wire format becomes [TOOL CALL] / [END TOOL CALL], defined once
in agent-turn.js (dependency leaf) so the system prompt, folded history and
every retry hint teach the same shape. The legacy angle form stays recognized
on read (RL-habit emissions) but is never taught or written again. Result
bodies now also neutralise call markers — both forms — so quoted untrusted
content cannot fire the trigger.

Suite 156/156; 3 mutations (trigger arm, neutraliser, canonical constant)
kill 2/1/4 tests respectively.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adversarial review (3 layers) surfaced one live bug and two coverage gaps in
9fa39fb; all patched forward.

1. Live: neutraliseResultMarkers defused the angle-form call marker with /g,
   not /gi, while TOOL_CALL_TRIGGER_RE is case-insensitive. An uppercase
   <TOOL_CALL> in an untrusted tool-result body survived neutralisation and, when
   the model echoed it as the first content of its answer, executed Bash —
   verified by execution. Added the i flag; hardened the hostile-body test with
   upper/mixed case plus a re-emit assertion.

2. The six retry-hint builders and the live-context notice in chat.js /
   anthropic.js / request.js had no test preventing a single-site revert to the
   native <tool_call> literal — which would re-seed the exact format the platform
   intercepts, on the retry path. Pinned with a source-scan lockstep test.

3. The widened bracket trigger executed a Markdown link
   [tool calls](url) … {json} at answer-start. A regex negative-lookahead fixed
   the full-text parser but diverged from streaming at the ]/( chunk boundary
   (the stream can't complete the look-ahead). Replaced with an
   isMarkdownLinkTail check at the payload-resolution point, where both parsers
   see the same tail; verified full-text and streaming now agree.

Suite 158/158; the i-flag, source-scan and isMarkdownLinkTail guards each
mutation-confirmed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Blind-hunter resent 7 more findings; 3 patched, 2 tests added, 2 deferred.

- Bracket closer decoration class now excludes '[' as well as ']', matching
  consumeTrailingCloser's grow-terminator check (`!slice.includes('[', 1)`); the
  two halves of the same function no longer disagree on whether '[' can appear
  inside a closer.
- Removed the negative prompt rule "never write these as angle-bracket tags": it
  raised the salience of the RL-biased angle form with no measurement that it
  helps, and disclosed the platform-interception mechanism to anyone who can read
  the system prompt. The positive "write exactly [TOOL CALL]" rule already stands.
- Documented that the 128-char payload window and the recovery percentages were
  measured on the ANGLE corpus and are unvalidated for the bracket form; and that
  the two hand-mirrored *_MAX literals are currently dominated by the angle floor
  (the bracket literal is a redundant safety cushion).
- Added tests: a bare bracket closer truncated at flush (`[END TOOL CALL` with no
  ]) is consumed; a decorated closer at the class limit is consumed. The first
  surfaced a real nuance — over-truncation before the word CALL leaks, exactly as
  `</tool_c` does for the angle form.

Deferred with evidence: the platform-injection defense (already mitigated by
Defect A's drop+log, the signal used to measure the fix) and the pre-existing
retry-hint duplication (desync-to-native is pinned by the loop-1 source-scan test).

Suite 160/160; the bare-closer and '[' -exclusion changes mutation-checked.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Correctness-only gate: js.configs.recommended + node globals, commonjs
source type. No style rules, no Prettier — upstream merges stay viable.
no-unused-vars ignores args/caught errors; no-empty allows the empty-catch
idiom used deliberately across the codebase. public/ (Vue/Vite frontend
with its own toolchain) is out of scope.

Also switches npm test to --test-force-exit: the suite leaves handles
open and plain node --test never exits, which would hang CI forever.

package-lock.json is force-added (gitignore still lists it): npm ci in
the CI workflow requires a committed lockfile.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
authorizeLogin declared chatBaseUrl inside the try but read it in the
catch's log line, so any auth failure died with a ReferenceError that
masked the real error. Hoisted above the try, mirroring the correct
pattern already used by initiateDeviceFlow. Also drops the unused
getProxyAgent import (proxying goes through applyProxyToFetchOptions).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two destroy() methods on the Account class; JS keeps the later one, so
the first (line 661) never ran. Its only unique work cleared
this.saveInterval — a property never assigned anywhere in the file.
The surviving destroy() covers refreshInterval plus the CLI timers.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Each of the flagged sites was eyeballed individually (no --fix sweep):

- Unused imports superseded by live code paths: applyProxyToAxiosConfig
  x4 (files use getProxyAgent directly), getTimezoneHeader (moved into
  header-profile), refreshAccountToken (accountManager method is the one
  called), apiKeyVerify, config, initSsxmodManager (a documented no-op
  kept upstream for backward compat — not a missing init).
- Dead code superseded by live replacements: handleVideoCompletion
  (resolveVideoResultContentUrl is the live path; never called),
  extractVideoTaskIDFromPayload (inlined at both call sites),
  isDeepResearch flag (computed, never branched on), redis isConnecting
  (write-only shadow of connectionPromise).
- Dead initializers/assignments per no-useless-assignment; every site
  was verified to be reassigned on all paths before any read. The LZW
  tail change in cookie-generator was additionally differential-tested:
  1008 old-vs-new customEncode cases, 0 mismatches. The tool-prompt
  drain() clears are covered by the parser test matrix.
- 10 form feeds (U+000C) inside comments replaced with spaces.
- upload.js re-throw now carries { cause } per preserve-caught-error.
- Tests: dropped never-asserted accumulator and unused constants; no
  assertion touched. Suite stays 160/160.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
One job, no path filter (the docker-build path-filter trap, D40), all
branches. Node lts/* matches the image's node:lts-alpine.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- eslint.config.mjs: dedicated **/*.mjs block (sourceType module + node
  globals) so future .mjs sources using process/Buffer don't false-fail
  no-undef; rationale comment on the three rule relaxations.
- ci.yml: timeout-minutes 10 (a hanging test no longer burns the 360-min
  default), permissions contents:read, concurrency group keyed on
  workflow+ref with cancel-in-progress, lint before tests (cheaper, and
  a red test must not hide lint results), header no longer overclaims —
  the job gates the Node backend only.
- package.json: engines node>=22 (--test-force-exit and ESLint 10 need
  modern Node; CI lts/*, docker lts-alpine and local dev all comply) and
  lint:fix convenience script; lockfile re-synced.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- tool-prompt.test.js: the markdown-link matrix test now asserts what its
  comment always promised — the streaming parser's reconstructed visible
  text (textDelta + recoveredText) equals parseToolCallsFromText's
  cleanedText for the same input. The lint sweep had deleted the
  never-asserted accumulator; the right fix was the missing assertion.
- cli-support.test.js: split the bundled authorizeLogin regression into
  two independent tests (non-ok response / fetch throws) so a first-assert
  failure can't hide the other scenario; the non-ok test now also pins the
  FIRST logged error to the response detail (status 403, body) — that
  pre-catch log's content is half the point of the original fix.
- models.js: drop the commented-out isDeepResearch consumer too —
  uncommenting it after the declaration's removal would ReferenceError.
- cookie-generator.js: comment on the flush block explaining why it lacks
  the in-loop twin's enlargeIn reset (dead store — only numBits is read
  after), so an upstream-sync edit doesn't reintroduce it "for symmetry".

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ive tool call

In long tool-heavy sessions the model regresses to the native angle form;
the platform intercepts it server-side, injects "Tool <name> does not
exists" into the generation, and the model narrates failure — the turn
arrives as valid-looking prose, no retry fires, the session dies.

The gateway already sees the interception live: the role:function deltas
Defect A drops. Surface them per attempt (normalizer grows
.interceptedToolNames, same call signature), and when an attempt had
drops, zero accepted tool calls, and tools in play, retry exactly once
with the canonical hint from agent-turn.js telling the model to re-emit
using the [TOOL CALL] form. Both Anthropic loops; the streaming loop
allows this retry even after the narration streamed — tool_use after
stray narration beats a dead session. The retry log line carries the
dropped names, greppable after UPSTREAM_NORMALIZER bursts.

Verified live 2026-08-31 08:33: 13 UPSTREAM_NORMALIZER drops named
exactly the tools a dying Claude Code session reported.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… to OpenAI loop and malformed bracket protocol

Review findings on the interception defense, all applied:
- terminalFinish now also vetoes intercepted (both Anthropic loops), matching
  missing_tool/empty discipline.
- Non-stream: an empty retry after an interception retry no longer trades the
  narration for a 502 — the intercepted attempt's cleanedText is kept as a
  delivery fallback.
- Observability: any rejection with drops logs "; dropped: <names>" (masking
  reasons included); giving up on a second interception/malformed turn logs a
  dedicated line. These use logger.warn — logger.warning does not exist on the
  singleton, so the pre-existing `logger.warning?.()` calls were silent no-ops.
- Masked-hint: when required/missing_tool (Anthropic) or any masking reason
  (OpenAI) wins the slot while drops are present, the canonical intercepted
  hint is appended — priority and cap untouched.
- interceptedToolNames dedupes and caps at 20 entries; JSDoc documents the
  mutate-in-place contract the non-stream reset relies on.
- OpenAI adoption (human-approved): attempts carry interceptedToolNames,
  evaluateOpenAIAgentAttempt gains intercepted + malformed_protocol ahead of
  the agent_final/agent_blocked acceptance that shipped the incident, and
  runOpenAIAgentTurn enforces the shared one-shot recovery cap (second
  incident delivers as-is instead of exhausting with 429).
- malformed_protocol (new, live leak evidence 2026-08-31): orphan bracket
  closers or an answer-start {"name":…,"arguments":…} payload with no opener
  leak as visible text with zero calls. Detection via
  containsOrphanProtocolResidue (tool-prompt.js, reusing the bounded closer
  regex; retry signal only — leaked JSON is never executed), same slot as
  intercepted (drops win), shared cap, canonical hint.

Tests: 21 new across anthropic-interception-retry, agent-protocol and the
tool-prompt lockstep (now covering both new hints). Suite 190/190, lint 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Live leaks (2026-08-31 10:12-10:17) survived the retry defense: the model
drops the opener entirely and answers with a bare {"name","arguments"}
payload plus [END TOOL CALL], which no trigger regex can fire on, and on
multi-call turns decideRetryReason short-circuits on emittedCalls so the
residue is invisible to the retry loop. Retry also cannot un-stream text
already on the wire. Deliberate renegotiation of the prior "leaked JSON is
never executed" boundary, gated hard:

- matchToolCallOpening: synthetic zero-length opener at answer start (or
  right after a completed call, whitespace only between) when the text
  ahead is a leaked-payload shape; shared by stream and full-text parsers.
- All gates or nothing: balanced JSON, MANDATORY bracket closer with a
  strict adjacency rule (only whitespace between payload and closer; any
  other character releases everything as prose immediately), tool name
  only from the payload keys, gated by allowedToolNames. Rejections come
  back as PROSE (never recoveredText, never errors) so the existing
  malformed_protocol retry keeps firing on the leaked text.
- Echo invariant pinned, not re-neutralized: neutraliseResultMarkers
  already disarms bracket closers inside folded tool results, so payloads
  quoted verbatim from results can never satisfy the closer gate. A
  regression test pins the disarm->gate chain; payload shapes inside
  results stay untouched (rewriting them would corrupt legitimate JSON).
- Duplicate closers after any accepted call (regular or salvaged) are
  swallowed, across chunk boundaries too (leak Rfym21#2 doubled closers).
- isLeakedToolPayloadShape single-sources the payload-shape predicate for
  both the residue detector and the synthetic gate.
- createUpstreamDeltaNormalizer({clientToolNames}): only client-declared
  tool names count as interception evidence; platform-internal drops
  (web_search / no-name) no longer fire false intercepted retries or burn
  the shared protocol-recovery slot. Drops are still logged.
- anthropic stream loop: per-attempt attemptVisibleText now feeds the
  malformed_protocol / missing_tool checks (cumulative visibleText still
  feeds empty + the after-prose guard), so a clean retry attempt is no
  longer re-condemned by the previous attempt's leaked residue.

All three live leak samples are pinned end-to-end (anthropic stream,
anthropic non-stream, OpenAI agent loop), plus the gate-rejection matrix,
stream/full-text lockstep, and mutation checks for every gate.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Thirteen triaged findings against d201273, all pinned by tests:

- P1: an unbalanced synthetic candidate no longer swallows everything to
  end-of-text (which leaked raw [TOOL CALL] markup and killed a valid
  later call). It is released as consumed protocol debris (visible, no
  emittedProse, no fence-tracker feed - the releaseDebris precedent) and
  scanning resumes at the next regular trigger, so the later call parses
  on both paths. NOTE: the finding's claim "baseline parsed it" was
  verified false (baseline suppressed the later call via the first-content
  gate and swallowed the markup); the implemented behavior is a deliberate
  improvement consistent with the debris precedent, not a baseline
  restoration. The stream hold now runs BEFORE the opening matcher so a
  regex trigger completing mid-buffer cannot outrun an undecided
  candidate (lockstep).
- P2: no non-empty allowedToolNames -> no salvage (fail-closed canSalvage
  in matchToolCallOpening; the legacy allow-everything gate semantics
  must never mint synthetic tool_use). Regular triggers keep legacy.
- P3: synthetic rejections log/record only the error TYPE - JSON.parse
  e.message embeds payload fragments on modern V8.
- P4: the stream-end bare-closer acceptance checks the TRUE remainder of
  the text, not the 63-char window; closer + a screen of whitespace +
  prose is an adjacency violation, not a closer.
- P5: waiting for the mandatory closer is capped by TOOL_CALL_SPAN_MAX;
  endless upstream whitespace can no longer grow the buffer unboundedly.
- P6: the payload-shape predicate is scoped to the LEADING object (keys
  in a later real call no longer arm spurious candidates on ordinary
  JSON answers), the stream hold gives up after 256 held chars without a
  "name" key (large JSON answers stream incrementally again - VG3), and
  hasTriggeredWithoutCall() ignores synthetic_rejected warnings.
- P7: rejected synthetic spans no longer feed the code-fence tracker
  (backticks inside JSON strings are not markup) while still setting
  emittedProse; a genuine later trigger is handled by the normal trigger
  path instead of leaking as "documentation".
- P8: at flush inside closerSwallow, a viable literal prefix of a
  duplicate closer ([END TOOL C + EOF) is swallowed as protocol residue;
  non-closer text is still delivered.
- P9: nameless platform frames never count as interception evidence when
  the filter is active - the 'unknown' log placeholder cannot impersonate
  a client tool literally named "unknown".
- P10/P11/P12: pinning tests for the OpenAI-runtime clientToolNames
  wiring, the per-attempt residue classification in the anthropic stream
  loop, and the angle-form arm of duplicate-closer swallowing - all three
  previously survived their reverts.
- P13: chat-helpers reuses normalizeAllowedToolNames from tool-prompt
  (no cycle) instead of an inline reimplementation.

Suite 228/228 green, lint 0. All 5 original spec mutations plus the
P10/P11/P12 revert-mutations verified failing exactly their pinned tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@maxff77

maxff77 commented Aug 31, 2026

Copy link
Copy Markdown
Author

Pushed 13 more commits hardening the tool-call protocol end to end, all verified live against Claude Code sessions:

Interception recovery (270d6a6, 3d6c4df): when the Qwen platform eats a native <tool_call> emission server-side (injecting "Tool X does not exists" into the model's context), the gateway now detects the dropped role:function deltas and retries once with a hint to re-emit in the bracket format — instead of delivering a dead narration turn. Covers both Anthropic loops and the OpenAI agent runtime, with a shared one-shot cap.

Opener-less salvage (d201273, 6d3fdab): models under long tool-heavy sessions sometimes emit the payload WITHOUT the opening marker (bare {"name":…,"arguments":…} + [END TOOL CALL], observed live) — previously leaking verbatim as visible text. The parser now recognizes that shape as a synthetic opener and converts it into the intended tool call, behind six fail-closed gates (answer-start position, both keys, balanced JSON, declared tool name, mandatory adjacent closer, outside code fences). Untrusted tool-result content cannot arm it (closer markers are already disarmed at fold time — now pinned by a regression test). Everything the gates reject stays prose and falls back to the retry path.

CI + ESLint (c82b637bb910d6): GitHub Actions gate (tests + lint on every push/PR) and an ESLint 10 flat config, correctness-only rules, zero-behavior-change cleanup.

Suite: 228 tests green, lint clean. The salvage went through an adversarial review cycle (13 findings patched, each pinned by a mutation-verified test).

PEDRO LOBATO CARCAMO and others added 2 commits August 31, 2026 14:47
…_tool_call retry, control-char JSON repair

Live 2026-08-31 ~14:08: the model emitted a full [TOOL_CALL] payload inside
its think phase; loop B streamed it verbatim into the client's thinking
block — never executed, no retry signal, success narrated in the answer.
Loop A already defends this surface; B and C now match it:

- B/C accumulate per-attempt raw think text and parse it with the shared
  parser at attempt end. Standalone pure calls are promoted to real
  tool_use under A's exact guards (zero answer calls, empty answer text,
  empty think cleanedText, zero think-parse errors, zero answer-side tool
  errors, non-empty allowedToolNames — fail closed). Anything else with
  calls/errors in think becomes emission evidence for a new one-shot
  thought_tool_call retry that shares the single protocol-recovery budget
  with intercepted/malformed_protocol and consumes the after-prose
  allowance like intercepted.
- Canonical hint lives only in agent-turn.js's buildAgentRetryHint
  (bracket form interpolated; no angle form; lockstep tests extended).
- 13:36 class: buildToolCallPayload now repairs raw C0 control chars
  inside JSON string literals — strictly after strict JSON.parse fails,
  never altering a strict-parse-accepted payload, logging type-only.
- Silent give-ups now log: loop B exhaustion and tool_error-after-prose
  breaks, plus the 5 logger.warning?.() no-ops (logger has warn only)
  in anthropic.js/chat.js converted to logger.warn.
- Tests: reconstructed 14:08 fixture (baseline-red), promotion rows,
  meta-discussion no-fire, shared-cap pins, repair unit rows including
  strict-first ordering pin, loop A promotion-guard pin (read-only).
  Suite 252/252 green, lint 0; all four spec mutations verified
  red-then-restored.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- R1: comments no longer claim verbatim A-parity — B/C use A's guards
  (openai-agent-runtime.js:232-243) plus two deliberately stricter
  conjuncts (fail-closed empty allowlist; zero answer-side tool errors);
  line citations unified to the verified range.
- R2: fail-closed empty-allowlist guard pinned in both loops.
- R3: loop C's thought_tool_call cap membership pinned (three non-stream
  mirrors of the stream cap tests).
- R4: thought_tool_call > missing_tool priority pinned in both loops.
- R5: answer-side zero-tool-error promotion conjunct pinned (truncated
  stream payload / native unknown-tool fixtures).
- R6: when required/tool_error masks think evidence, the retry hint now
  appends the canonical thought_tool_call hint (intercepted-appendix
  pattern), both loops.
- R7 (HIGH): promotion now respects terminalFinish — a think call from a
  content_filter/refusal/length turn no longer promotes or executes.
- R8: loop C logs a neutral exhaustion warn after the retry loop; the
  stream exhaustion warn no longer asserts delivery that may not happen.
- R9 (HIGH): narrationFallback extended to thought_tool_call — the 14:08
  shape with an empty retry delivers the narration instead of a 502.
  malformed_protocol deliberately excluded: its cleanedText is the leaked
  protocol residue itself.
- R10: loop C strips a promoted call from the delivered thinking block
  (attempt segment replaced by the parse's cleanedText; searchTable and
  prior-round deliberation preserved).
- R11: previously-unexecuted log paths now covered — tool_error-after-
  prose break + degraded-delivery warns (stream), non-stream protocol-
  failure warn + 502 body, and both chat.js legacy compensation lines.
- R12: searchTable exclusion from the think accumulator pinned.
- R13: logToolError sanitizes invalid_json reasons at the log layer (cut
  before the first quote / " in JSON" / " at position", capped) — V8
  payload echoes never reach the log; the error object keeps the full
  reason.

Suite 272/272 green, lint 0. Mutations re-verified red-then-restored:
R2 (allowlist conjunct → true, both loops), R3 (C cap membership),
R4 (priority swap, both loops), R5 (error conjunct dropped, both loops),
R6 (appendix removed, both loops), R7 (terminal conjunct dropped),
R8 (C exhaustion warn silenced), R9 (fallback reverted to intercepted-
only), R10 (containment removed), R11 (break warn silenced),
R12 (searchTable accumulated), R13 (sanitizer bypassed).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@maxff77

maxff77 commented Aug 31, 2026

Copy link
Copy Markdown
Author

Pushed 2 more commits (48fa51b, 7066233) — a third live failure class, found and fixed:

Think-phase tool-call leak. In long agent sessions the model sometimes emits its entire bracket tool call inside the think phase. The Anthropic loops streamed think content verbatim (no parsing at all), so the call reached the client as a thinking block, never executed, and the model then narrated success — a hallucinated tool run. The OpenAI loop already defended this surface; the Anthropic stream + non-stream loops now match it:

  • Per-attempt think text is parsed at attempt end with the shared parser. A standalone pure call (nothing else in the think text, zero parse errors, name in the declared tool set) is promoted to a real tool_use — same guards as the OpenAI loop, plus two stricter conjuncts (fail-closed on an empty tool whitelist; zero answer-side tool errors), plus terminal-finish discipline (never promote from content_filter/refusal/length turns).
  • Anything else that parsed as a call or produced parse errors becomes a one-shot thought_tool_call retry (shared protocol-recovery budget with the existing intercepted/malformed_protocol reasons) with a canonical hint telling the model its call sat in unreachable reasoning.
  • Non-stream responses strip the leaked protocol from the delivered thinking block on promotion.

Control-char JSON repair. Second verified failure class: payloads with raw newlines/tabs inside JSON string literals (Bad control character in string literal) died as invalid_json. buildToolCallPayload now runs a strict-parse-first repair (escape raw C0 inside string literals only) — valid JSON is untouched by construction, and unrepairable payloads keep the original strict-parse error.

Observability. Five logger.warning?.() calls were silent no-ops (the logger only exposes warn) — converted; the previously-silent give-up breaks (attempt exhaustion, tool_error after visible prose) now log; JSON.parse messages are sanitized before logging (V8 embeds payload fragments in e.message).

Hardened by a 4-layer adversarial review: 13 findings patched in 7066233, 12 of them pinned by mutation tests (guard deletion / priority swap / cap removal each verified red before restore). Suite: 272 tests green.

PEDRO LOBATO CARCAMO and others added 5 commits August 31, 2026 18:07
…ed retry, residue-free delivery

Three prod incidents (2026-08-31 16:31-16:39) hit loop B's documented
'prose seen -> tool_error -> no retry -> deliver as-is' ceiling: raw
[TOOL_CALL] spans reached the client and the calls never executed.

Layer 1 — shared parser salvage (fail-closed, triple-gated):
repairLooseToolPayload quotes bare keys/values deterministically (a bare
value reuses the NEXT existing quote as closer or is rejected);
extractTriggerNameHint recovers the tool name from bracket-trigger tails
([TOOL_CALL]Bash{...}); salvage passes only with strict JSON.parse +
non-empty allowlist + argument keys within the tool's declared
input_schema.properties. Runs at both truncated_tool_call condemnation
sites. Schema-less callers (loop A, chat.js) are byte-identical.

Layer 2 — loop B text-suppressed retry: tool_error after prose consumes
the existing retriedAfterVisibleText slot; the retry forwards only
tool_use blocks (text/thinking suppressed at the emit layer, detection
accounting untouched). No new attempt budget.

Layer 3 — residue-free delivery: both parse paths record condemned spans
(residueSpans); stripToolCallResidue subtracts exactly those at delivery
(B recoveredBuffer, C cleanedText when errors remain). Emptiness for the
502 decision is judged on stripped text (residue-only turns still 502);
required-unfulfilled after streamed prose closes end_turn + warn instead
of poisoning a half-delivered message.

302/302 tests (272 baseline + 30 new, one fixture per spec matrix row,
incident-3 reconstructed byte-exact); 3 mutation checks pinned.

Co-Authored-By: Claude Fable 5 (1M context) <noreply@anthropic.com>
…dened gates, positional residue strip

Adversarial-review triage on top of the salvage layers:
- salvage honors emittedProse exactly like canonical calls at both truncated
  sites; malformed input is never more executable than well-formed
- no-closer truncated spans reject salvage (tail may be the answer); the
  condemned residue record is bounded to provably-protocol bytes
- schema gate validates required keys; vacuous {} rejects; salvage rejections
  get their own error type (salvage_rejected)
- streaming salvage runs only at flush, never on a mid-stream over-cap span
- stripToolCallResidue is position-driven (recorded offsets, fail-open on
  mismatch); the first-indexOf + trim fallback is gone; rejected synthetic
  payloads never enter the residue ledger (they may BE the answer)
- B: banked attempt-side recovered text survives a suppressed retry;
  emptiness judged via debris-only subtraction with consistent normalization
- C: delivery strip gated on the delivered round's residueSpans and applied
  to the round's raw text before agent tags; narrationFallback carries its
  round's raw text + spans
- buildInternalRequest: duplicate tool names get no schema (fail closed)
- repairLooseToolPayload: EOF counts as a literal delimiter
- tests: 49 salvage/wiring tests incl. production e2e through
  handleAnthropicMessages with patched sendChatRequest

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ast so a last-wins regression is catchable

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…-only turn 502s instead of shipping content: []

Verified repro (repro-item9-corner.js): an envelope-less, unbalanced,
leak-shaped payload as the whole turn is recorded as debris (item 8 only
excluded REJECTED spans from the ledger) with zero toolErrors; after the
malformed_protocol retry is spent, the delivery strip emptied cleanedText
AFTER the emptiness guard had already passed on the unstripped text →
HTTP 200 with an empty content array. The strip now runs before the
guard, so the all-residue turn takes the no-content 502 — honoring the
frozen 'never an empty-content message' row, same discipline as B's
strippedVisibleText.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…all; orphan bracket closers join the residue ledger

Both verified by execution against the pre-fix code:
- bare-unbalanced debris turn → was 200 with content: []; the C delivery is
  now computed once, post-strip, and the residue-only/empty decision (spans
  recorded + stripped text empty) joins the invalid_tool_call_error 502 —
  never an empty-content message
- orphan-closer-only turn ([END TOOL CALL]) → was 200 delivering the raw
  closer; whole-text parses (scan loop AND fast path) now register orphan
  bracket closers in the residue ledger — fence/inline-code examples exempt
  via the same code tracker, spans inside recorded spans not double-entered
  — so delivery strips them and a closer-only turn 502s
- regression guard: prose + stray closer delivers the prose (closer
  removed), status 200 — prose is never upgraded into a 502

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@maxff77

maxff77 commented Sep 1, 2026

Copy link
Copy Markdown
Author

salvage-3 — prose-adjacent tool-call leak recovery (5 commits, 7066233..6c27347)

A fourth leak class: the model emits its tool call as [TOOL_CALL]Bash{command:find …", "…"}} — the name sits outside the JSON and an unquoted key/value breaks quote parity, so extractBalancedObject never closes → truncated_tool_call, the raw span reaches the client, the call never runs. Hit 3× in 9 minutes on a live tool-heavy session.

Three fail-closed layers (each behind a triple gate: non-empty allowlist + strict JSON.parse + input_schema key/required validation):

  1. Deterministic quote repair (repairLooseToolPayload) — quotes bare keys, opens bare string values reusing the next existing " as closer (never guesses where a value ends), backslashes/C0 chars JSON-escaped so bytes round-trip; tool name recovered from the bracket-trigger tail. Valid JSON is a fixed point.
  2. Text-suppressed retry (loop B) — a tool_error after prose consumes the existing single compensation slot with a retry that forwards only tool_use blocks; its text/thinking never reach the wire.
  3. Residue-free delivery — both parse paths record condemned spans in one ledger; delivery strips exactly those (positional, fence-immune), never a second search. Emptiness for the 502 decision is judged on stripped text (a residue-only turn 502s invalid_tool_call_error, never an empty content array); orphan bracket closers join the ledger so a lone [END TOOL CALL] is stripped/502'd, while prose + a stray closer delivers the prose.

The security boundary this cycle fixed: salvage now honors the parser's emittedProse position gate exactly like a canonical call — a malformed span after visible prose is suppressed, not executed. Earlier it inverted the anti-injection doctrine (malformed was more executable than well-formed). The real incident span is first-content, so recovery is unaffected.

Process: 2 adversarial review loops (3 independent reviewers), ~23 findings triaged, every behavioral fix mutation-pinned. 325/325 tests (51 salvage). Deployed to the staging container and validated live: both loops return tool_use (non-stream stop_reason: tool_use; stream emits thinking → tool_use → stop_reason: tool_use). Loop A / chat.js untouched (they pass no schemas → salvage fail-closed off).

@Rfym21
Rfym21 merged commit dc2e8ec into Rfym21:main Sep 1, 2026
maxff77 pushed a commit to maxff77/Qwen2API-1 that referenced this pull request Sep 3, 2026
_bmad-output/ holds local planning artifacts from the bmad tooling; it was
committed by mistake in Rfym21#164 and does not belong in the project tree.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants