Reply plugins: keep each choice of an n > 1 answer apart

[fylorn]

Problem

Chat Completions with n > 1 and Gemini with candidateCount > 1 carry several choices (candidates) in one answer, told apart by each choice's index. The reply plugin chain only looked at the first element of choices / candidates and sent all text through a single lane:

• Text of interleaved choices went into one buffer. A block-mode text plugin received hel + wor + lo + ld as one block and wrote it back into choice 0 (HELWORLOLD).
• Half a secret held back at the end of one choice's text was handed out at the head of the next text piece, which could belong to the other choice.
• Chat tool call fragments were keyed by the tool call index only, so call 0 of choice 0 and call 0 of choice 1 were glued into one call (unparseable arguments), and choice 1 lost its call.
• Choices after the first element in a stream frame, and every choice after the first in a whole body, never went through the plugins.

Fix

plugin/reply/chat.rs and plugin/reply/gemini.rs now keep state per choice, keyed by the choice's index field (falling back to the array position when it is missing):

• each choice has its own text lane (lane number = choice index) and its own open/closed state;
• Chat: each choice has its own tool call buffer, renumbering and finish_reason rewrite; a finish_reason ends only that choice, while [DONE], error frames and the usage chunk still close every choice;
• frames that carry several choices are handled choice by choice; a choice emptied by the plugins is removed from the frame, and the frame is dropped only when no choice is left;
• synthesized frames (held text, rewritten tool calls) carry the right choice index;
• whole bodies run every choice / candidate through the plugins.

reply/mod.rs is unchanged.

Tests

New tests in plugin/reply/tests (all four fail on main):

• chat_choices_are_kept_apart_in_a_stream: two interleaved choices, each with text and tool calls; a block-mode text plugin plus a tool call rewriter. Each choice gets its own text and its own rewritten calls.
• chat_choices_hold_back_apart_in_stream_mode: a stream-mode plugin with a known secret split across two frames of choice 0 and choice 1 text in between. The held half stays with choice 0.
• gemini_candidates_are_kept_apart_in_a_stream: interleaved candidates, with a final frame carrying both candidates (including a function call in the second one).
• whole_bodies_run_every_choice_through_the_plugins: Chat and Gemini whole bodies with two choices.

cargo fmt --all -- --check, cargo clippy --workspace --all-targets -- -D warnings and cargo test --workspace pass locally.

Not covered here: a reply plugin instance is still one per answer and onReplyText / onReplyTextEnd do not say which choice a piece belongs to, so a stream-mode plugin that buffers text in its own state still sees the choices interleaved.

🤖 Generated with Claude Code