Skip to content

fix(analyze-files): send text files to Claude as text, name the rejected files - #1157

Open
ionut-mihalache-uipath wants to merge 5 commits into
mainfrom
fix/analyze-files-anthropic-non-pdf
Open

ionut-mihalache-uipath wants to merge 5 commits into
mainfrom
fix/analyze-files-anthropic-non-pdf

Conversation

@ionut-mihalache-uipath

@ionut-mihalache-uipath ionut-mihalache-uipath commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • for models that use the Anthropic Messages API (ChatAnthropic, which includes ChatAnthropicBedrock, and Vertex ChatAnthropicVertex), Analyze Files sends a text file (text/*, application/csv, application/json, application/xml, +json/+xml) as an Anthropic text document (source.type: "text"). The file content is not changed
    • decoded by its UTF-16/UTF-32 byte-order mark if it has one, else as UTF-8 or, failing that, Windows-1252 (Excel's "CSV" encoding)
    • content that decodes to a NUL character is binary (or BOM-less UTF-16) and is sent as a file block, as before, rather than as garbled text
    • the text document carries no guardrail attachment tags (extras): Claude clients send it to the provider as is. File and image blocks keep them
  • every other format is still sent as a file block, as before; when the model rejects it, the user now gets a clear error (below)
  • a file declared application/octet-stream or with no type whose bytes start with %PDF- is sent as application/pdf, for every provider (SharePoint "Convert to PDF" downloads arrive this way)
  • MIME types are compared without their parameters, so application/pdf; name="inv.pdf" is sent as a PDF
  • a 400 on a file-analysis request becomes a user FILE_ERROR, "The model could not read the attached files", naming each file and its type, or "The attached files are too large for the model" when the provider reports a context-window overflow
    • files are named with the type they are sent as (a .csv declared application/vnd.ms-excel is named text/csv)
    • when every attachment is a PDF, the error suggests the PDF may be damaged, password-protected or too long instead of "convert to PDF"
    • a 400 carrying the gateway's own (non-blank) detail, a content-filter rejection, and a 400 that names the file limit it hit (too many images, image or document too large) keep the existing mapping
    • the provider's message is still never copied into the error
  • the unsupported-MIME-type error now names the attached files

Bedrock Converse, OpenAI and Gemini requests are unchanged; Converse already reads CSV natively.

Why

Analyze Files on Claude failed for a CSV and an unlabeled SharePoint PDF download. The Anthropic Messages API rejects any base64 document whose media type is not application/pdf (document.source.base64.media_type: Input should be 'application/pdf'), even for a correctly typed text/csv, and the user saw a generic "review the model settings" error.

Testing

  • unit tests for the text document, decoding, model detection, PDF sniffing, the error mapping and the tool wiring; the text document is pinned through the langchain_anthropic and Vertex Anthropic formatters
  • ruff check, ruff format --check, mypy, full pytest
  • live on alpha, through build_file_content_blocks_for and the model:
    • anthropic.claude-sonnet-4-5-20250929-v1:0 over Bedrock invoke (UiPathChatAnthropicBedrock) and claude-sonnet-4-6 (UiPathChatAnthropic): UTF-8 CSV, cp1252 CSV, unlabeled PDF and PDF with MIME parameters are all read; an XLSX gets "The model could not read the attached files" naming the file
    • on Bedrock invoke, a CSV labelled application/csv, a UTF-16 CSV with BOM and a JSON file are read; a PNG labelled text/plain is sent as a file and gets the named error
    • the same Bedrock model over Converse: unchanged, CSV and PDFs read

Known limits

  • on Claude over the Messages API, Office files (XLSX, DOCX, PPTX) are still not readable; they now fail with an error that names the file and suggests converting to PDF
  • the memory node's file attachments are not changed

🤖 Generated with Claude Code

https://claude.ai/code/session_018uaE4Knma5aBySxy4Qxu8n

Copilot AI balanced review requested due to automatic review settings October 14, 2026 11:30

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

PDF detection, binary rejection, and PowerPoint ordering contain correctness gaps.

3 open findings
What changed in this PR

Updates Analyze Files to support non-PDF attachments with Claude-compatible models and provide clearer file-related errors.

Changes:

  • Converts text and Office Open XML files into Anthropic text documents.
  • Detects unlabeled PDFs and Claude model capabilities.
  • Improves provider error mapping and attachment diagnostics.
File Description
pyproject.toml Bumps package version.
uv.lock Updates locked project version.
src/​uipath_langchain/​agent/​exceptions/​llm.py Adds file-analysis error mapping.
src/​uipath_langchain/​agent/​multimodal/​__init__.py Exports model capability detection.
src/​uipath_langchain/​agent/​multimodal/​invoke.py Routes attachments by model capability and content.
src/​uipath_langchain/​agent/​multimodal/​text_documents.py Extracts text from plain and Office files.
src/​uipath_langchain/​agent/​multimodal/​utils.py Adds PDF sniffing and provider detection.
src/​uipath_langchain/​agent/​tools/​internal_tools/​analyze_files_tool.py Integrates conversion and file-aware errors.
tests/​agent/​multimodal/​test_text_documents.py Tests text-document conversion.
tests/​agent/​multimodal/​test_utils.py Tests model detection and PDF sniffing.
tests/​agent/​test_llm_file_analysis.py Tests user-facing file errors.
tests/​agent/​tools/​internal_tools/​test_analyze_files_tool.py Tests Analyze Files integration.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@ionut-mihalache-uipath
ionut-mihalache-uipath force-pushed the fix/analyze-files-anthropic-non-pdf branch from 8fffec2 to 275d464 Compare October 16, 2026 12:42
@ionut-mihalache-uipath ionut-mihalache-uipath changed the title fix(analyze-files): send non-PDF files to Claude as text, name the rejected files [PC-5223] fix(analyze-files): send text files to Claude as text, name the rejected files [PC-5223] Oct 16, 2026
@ionut-mihalache-uipath ionut-mihalache-uipath changed the title fix(analyze-files): send text files to Claude as text, name the rejected files [PC-5223] fix(analyze-files): send text files to Claude as text, name the rejected files Oct 16, 2026
@ionut-mihalache-uipath
ionut-mihalache-uipath force-pushed the fix/analyze-files-anthropic-non-pdf branch from 66d173f to ac968b3 Compare October 26, 2026 14:52
…ted files [PC-5223]

Claude over the Anthropic Messages API (Anthropic, ChatAnthropicBedrock,
Vertex) accepts a base64 document only as application/pdf, so a CSV attached
to Analyze Files failed with a 400 and a "review the model settings" error.

- On those models a text file (text/*, e.g. CSV) is decoded (UTF-8, else
  Windows-1252) and sent as a text document. Converse, OpenAI and Gemini are
  unchanged.
- A PDF declared application/octet-stream or without a type is recognised by
  its signature, and MIME parameters (application/pdf; name=...) no longer
  stop a PDF or image from being sent as one.
- A provider 400 on an Analyze Files request now names the attached files and
  says whether the model could not read them or they exceed its context
  window; unsupported-MIME errors name the files too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018uaE4Knma5aBySxy4Qxu8n
…ions accurate

- application/csv, application/json, application/xml and +json/+xml types
  count as text, so a CSV labelled application/csv no longer hits the 400.
- A UTF-16/UTF-32 byte-order mark picks the encoding, and decoded text with a
  NUL character is rejected as binary instead of sent as garbled text.
- A content-filter 400 keeps the generic mapping; when every attachment is a
  PDF the error no longer advises converting to PDF.
- Files are named with the type they are sent as, not the declared one.
- One helper reads the gateway's ProblemDetails detail for both mappings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018uaE4Knma5aBySxy4Qxu8n
An application/octet-stream PDF that kept its source file's name
(Budget.docx) was renamed by its extension before the %PDF- check ran, so
it was sent as a Word document. The check now reads the declared type.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018puK13ufeccUUczMHLct3F
…iles 400s

A 400 that names the limit it hit (too many images, image or document too
large) already maps to a specific file error; the file-analysis mapping no
longer replaces it with the generic "could not read" message.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ly paths

Sniffed PDFs, PDFs and images with MIME parameters, and files left to the
model still carry the attachment id and name in extras. The text document
sent to Claude carries none, and no tag reaches the Anthropic payload.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0136ibA6NKQ2zinUJHapvQud
@ionut-mihalache-uipath
ionut-mihalache-uipath force-pushed the fix/analyze-files-anthropic-non-pdf branch from ac968b3 to 5d80749 Compare November 2, 2026 19:54
@sonarqubecloud

sonarqubecloud Bot commented Nov 2, 2026

Copy link
Copy Markdown

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants