Codex compaction on converted routes; mid-conversation developer messages stay in place

[fylorn]

Two follow-ups to #304, both for Codex sessions routed to a non-Responses upstream (Anthropic / Chat / Gemini / Bedrock).

1. Codex compaction on converted routes

The contract Codex uses

Codex compacts remotely whenever the provider is named "OpenAI" (capabilities.rs#L33). ThinkWatch's takeover registers its shadow provider under that name, so every Codex compaction through the gateway takes this path (turn.rs#L1474, tasks/compact.rs):

• Request: an ordinary streaming Responses request through the same build_responses_request: the whole history, then {"type": "compaction_trigger"} (compact_remote_v2_attempt.rs#L95). Tools and tool_choice are as usual.
• Response: Codex reads the stream and requires exactly one response.output_item.done whose item is {"type": "compaction", "encrypted_content": "…"}, plus a response.completed. Other items are ignored, and zero or two compaction items fail (compact_remote_v2.rs#L430-L490).
• Afterwards: the history becomes the retained user messages, then the compaction item, then the rebuilt context (build_v2_compacted_history). Every later request sends that item back verbatim.

Converted routes used to refuse compaction_trigger (#304), so these sessions could never compact.

What the gateway does now (tw_dialect::compaction)

• Request: the history converts as usual, and a final user message asks the upstream for a handoff summary (compaction::INSTRUCTION). The summary must cover goal and constraints, decisions and failed approaches, files and commands, running state, open tasks, and anything carried over from an earlier summary. Tools, reasoning settings and tool_choice are unchanged, so the request has the same prefix as the previous turn and its prompt cache still applies. (Changing tool_choice would invalidate Anthropic's message cache.) A test pins that the system prompt, tools and every message except the last match a normal turn on the same history.
• Response: Session::is_compaction(). The upstream's text blocks (thinking and stray tool calls excluded) become the single compaction item, with added then done then response.completed including usage. This works streamed, whole and stream-collected.
  • The item's encrypted_content is our own carried format, tw1.c.<base64url(summary)>. Codex treats it as opaque and the gateway can read it back.
  • While the summary is being collected, the stream emits response.in_progress every 32 upstream deltas, because Codex treats 300 s without an event as a dropped stream.
  • If the upstream writes no text, the client gets response.failed. Returning an empty summary would make Codex discard the conversation.
• Next request: a carried compaction / context_compaction item decodes to the summary in place, as a mid-conversation system turn prefixed with compaction::SUMMARY_PREFIX. It is a system turn because the upstream wrote it, not the caller, so content filtering and redaction treat it like other system text. OpenAI's own encrypted compactions are still refused with a clear message, and a damaged carried item is refused too.
• Passthrough to OpenAI later (failover, route change, ChatGPT account): strip_carried replaces a carried item with a developer message carrying the same summary. OpenAI can't read our format, and dropping the item would lose the whole earlier context. Bodies without carried items are still left byte for byte.
• Usage and cost of the summarisation call are recorded like any other request: the gateway sniffs the upstream's own usage (gateway e2e test).
• tw-store: the transcript shows a carried summary as text, and search indexing ignores the gateway's summarisation instruction.

2. Mid-conversation developer / system messages stay in place

Codex adds developer messages as a session goes on: permission changes, collaboration mode, tool updates, and the rebuilt context after a compaction. #304 still appended every one of them to the system prompt, so the system prefix changed whenever one appeared, which resets any prefix cache that starts at the system prompt.

• IR: new Role::System. Leading system/developer messages still form Request::system; any that come after the conversation has started become Role::System messages where they appear (ir::system_turn). This applies to the Responses developer/system items and to Chat and Anthropic system messages.
• Responses target: a native developer message in place.
• Anthropic, Chat, Gemini, Bedrock targets (ir::fold_system_turns): the message becomes a user turn wrapped in <system-reminder>…</system-reminder>. Claude Code uses the same convention, so Claude reads it as a harness message, not the user's words. Message count and indices are unchanged, so client cache breakpoints still line up.
  • Chat also uses the user turn, not a mid-conversation system message: many non-OpenAI models' chat templates accept a system message only at the start and reject one later.
  • A system turn sitting between a tool call and its result moves to just after the result, inside the same message, because Chat requires results to follow the call immediately and Anthropic and Bedrock require them first in the user message.
• A test encodes two consecutive Codex requests, the second adding an assistant reply, a developer message and a user message. For all four targets it checks that the system prompt and tools are identical and that the first request's messages are an exact prefix of the second's.
• Caveat: converted Codex→Anthropic requests carry no cache_control breakpoints today. The IR only has breakpoints that Anthropic or Bedrock clients put there, and Responses clients have none. So this change helps the automatic prefix caches now (OpenAI, DeepSeek, Kimi, GLM, Gemini implicit caching); Anthropic caching for converted requests needs breakpoints first. That is a separate decision, not made here.

API (additive for the enterprise edition)

New ir::Role::System (the enterprise code only compares and constructs Role values; it never matches on it exhaustively). Also new: ClientShape.compaction, ir::{system_turn, system_reminder, fold_system_turns}, the compaction module, and Session::is_compaction. No tw-gateway source changes.

The DeepSeek Harness passthrough cleaner (harness::clean, Anthropic to a non-DeepSeek Anthropic upstream) still merges dsh's system entries into system, as before. Its two conversion tests in tw-gateway/tests/harness.rs now expect the in-place reminder.

Tests

• tw-dialect/tests/codex_compaction.rs (new), built on Codex-shaped Lite requests:
  • The compaction request has the same prefix as a normal turn on every target.
  • Streamed answers from Anthropic (with thinking), Chat, Gemini and Bedrock come back as exactly one compaction item that Codex accepts, with keepalives and usage; the whole and collected paths work too.
  • An answer without text fails.
  • The next request after compaction carries the summary in place on every target; OpenAI's compaction is still refused.
  • strip_carried on passthrough.
  • The developer-message prefix-stability test for all four targets.
  • A Chat mid-conversation system message reaches a Responses upstream as a developer message.
• Unit tests: compaction decode, carried summary in place, carried format round trip and garbage, fold_system_turns including the tool-result reordering, and refusals.
• tests/caller.rs cross-check: mid-conversation developer/system messages and carried summaries survive caller-text removal and are not caller text.
• Gateway e2e (tests/conversion.rs): a compaction through a mock Claude returns one compaction item and records usage (40 in, 17 out); a carried summary sent straight through to OpenAI arrives as a developer message.
• tw-store: transcript shows the carried summary; search reads the user's last words, not the instruction.

Checks: cargo fmt --all --check, cargo clippy --workspace --all-targets -- -D warnings, cargo test --workspace --no-fail-fast (3139 passed, 0 failed, 8 ignored), scripts/smoke.sh (79 passed, 0 failed).

🤖 Generated with Claude Code