Skip to content

fix(common): render tool-result content blocks as text, not escaped JSON or base64 - #90

Open
malihashar wants to merge 1 commit into
skillsynchq:mainfrom
malihashar:fix/tool-result-blocks
Open

malihashar wants to merge 1 commit into
skillsynchq:mainfrom
malihashar:fix/tool-result-blocks

Conversation

@malihashar

Copy link
Copy Markdown

Closes #89

What is wrong

Many tools return their result as a list of blocks, for example one text block and one image block. Every MCP tool does this, and so does any screenshot or image-reading tool. txcript turns that whole list into one JSON string, in view, in the MCP read_session tool, in search, and when converting to Codex, pi, Campfire, Cursor and OpenCode.

So you see escaped text (\n and \") instead of the real text. A screenshot shows up as its entire base64 data. Nothing warns you.

How to see it

Make a Claude Code session with a tool result that has a text block and an image block of about 150 KB, then:

Command Before After
txcript view <id> 147,297 bytes, with the whole base64 inline 513 bytes: the text, then [image: image/png]
MCP read_session refused: over the 100000 byte limit works, 1,259 byte response
continue --with codex --out 149,245 byte file full of escaped JSON and base64 2,427 byte file with plain text
search one hit line was 146,790 characters of JSON hit lines are the real text lines

What this PR changes

One helper, ToolOutput::as_text, in src/common.rs. It turns a tool result into readable text:

  • Plain text stays as it is.
  • A list of blocks becomes its text blocks joined with a blank line, and an image becomes [image: <media type>]. This is the same placeholder that PR fix(codec): preserve multimodal artifacts and images across harness conversions #46 uses. If there is no media type, it is [image].
  • Any other block type stays as compact JSON.
  • Everything else (JSON objects, numbers, empty lists, lists of strings) stays exactly as it is today.

It replaces about 10 private copies of the same JSON-to-text code in view, search, the pager, and the Codex, pi, Cursor, OpenCode, Grok and fx converters. Grok and fx already did the text part, but gave up and printed the whole JSON as soon as one block was an image. No new dependency, option or type.

I left two converters alone on purpose: Antigravity and Cursor Desktop. They write compact JSON deliberately so it reads back as JSON, and changing that would break their round-trip tests. Images there still export as base64. Formats that keep the structure (Claude Code, Simple, Hermes, Grok Bot) are untouched.

How it was tested

  • 7 unit tests for the helper: text, quotes and emoji, several large images, a missing media type, unknown blocks, non-block JSON, and a 1 MB text.
  • 1 test for the terminal pager.
  • 2 regression tests in tests/regression/tool_result_blocks.rs. They use one session with four results (a text block, text plus image, a plain string, a JSON object) and check view, search, and the Codex, pi and Grok exports. The plain string and JSON object must stay byte for byte the same. These tests fail on the old code and pass on the new code.
  • The full suite passes (479 tests, up from 469), and cargo fmt and cargo clippy are clean.

What I could not test

Questions for the maintainer

  1. Is [image: <media type>] the right placeholder?
  2. Should text blocks be joined with a blank line (as Grok and fx do), or a single newline (as pi's reader does)?
  3. Should Antigravity and Cursor Desktop also get readable text, even though that changes how their results read back?
  4. Should the helper be public, or only visible inside the crate?
  5. Should pi and Cursor write real image blocks instead of a placeholder? That is bigger work and is related to fix(codec): preserve multimodal artifacts and images across harness conversions #46.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Tool results made of several blocks (MCP tools, screenshots) show as raw JSON, and images as base64 text

1 participant