Skip to content

fix: Accept unsigned reasoning output - #90

Open
jiahuei wants to merge 2 commits into
mainfrom
fix/unsigned-reasoning-output
Open

jiahuei wants to merge 2 commits into
mainfrom
fix/unsigned-reasoning-output

Conversation

@jiahuei

@jiahuei jiahuei commented Oct 29, 2026 •

Copy link
Copy Markdown
Member

Summary

  • Responses and Messages output normalization and projection now accept reasoning summaries without a signature instead of failing when the part completes. Responses omits unavailable encrypted state and Messages emits an empty signature; no continuation state is invented.
  • Keeps request/history validation strict, preserves existing signatures and signature-stream consistency, and retains Chat rejection of signatures it cannot represent. Removes the redundant completed-reasoning validation helper and updates the output/history documentation.
  • Adds SDK-parsed cassette-derived parity across all six translated routes in streaming and non-streaming modes, including exact upstream requests and rejection before transport when unsigned output is replayed as summary history. Signature removal happens only in explicitly synthetic in-memory test streams; no cassette is modified relative to main.
  • Incorporates main through 70ad86c, preserving hosted web search and its remapped Messages block indices alongside unsigned reasoning output.

Verification

  • .venv/bin/python -m pytest --cov=python/molto --cov-report=term-missing --cov-report=xml:coverage.xml --junitxml=junit/pytest.xml: 2,316 passed in 88.68 seconds on Python 3.12.9, using offline replay only.
  • uv run --extra lint ruff check --output-format github --config pyproject.toml .: passed; the resolved Python files also passed the focused check.
  • uv run --extra lint ruff format --diff --config pyproject.toml .: passed; the resolved Python files also passed the focused check.
  • npx prettier@3.8 --check .: passed; resolved documentation also passed its focused check.
  • git diff --check and git diff --cached --check: passed before committing.
  • git diff --quiet -- tests/cassettes and git diff --cached origin/main --quiet -- tests/cassettes: passed before committing; cassette files match main, with no new recordings.
  • Independent merge-resolution review found no correctness or scope issues.
  • GitHub CI for the merge commit is in progress. Python 3.13 and 3.14 were not run locally.

jiahuei added a commit that referenced this pull request Oct 29, 2026
## Summary

- Adds `Molto.messages(..., citation_policy="auto")` for conditional
citation buffering on translated Responses routes. It buffers text when
normalized tools or history signal possible citations; otherwise text
streams immediately and unexpected citations are dropped. `"buffer"`
always buffers each text part, `"drop"` remains the default, and opaque
transport extensions do not select automatic buffering. Native Messages
and Chat upstream behavior remain unchanged.
- Preserves point citations without splitting text at zero-length
ranges. When no original excerpt exists, the containing block supplies
the fallback `cited_text`. Existing offset validation, blank-text
rejection, provider metadata, and SDK-compatible text deltas remain
intact.
- Extends terminal-reason coverage with normal completion and refusal,
and adds coverage for automatic buffering, history, transport overrides,
point citations, and SDK text iteration.
- Merges `amer-hosted-web-search`, retaining its Anthropic continuation
metadata, provider-specific missing/empty/placeholder ciphertext
rejection, and recorded continuation coverage. No cassette contents
differ from that branch.

Removal of output-only missing reasoning-signature checks is tracked
independently in [PR #90](#90).

## Verification

- `.venv/bin/python -m pytest`: 2,293 passed in 35.78 seconds on Python
3.12.9 after separating the reasoning-output work. Tests used offline
replay only.
- Focused citation and terminal-reason tests: 154 passed, 267
deselected. Earlier outdated point-citation expectations were corrected
before the passing full suite.
- `uv run --extra lint ruff check --output-format github --config
pyproject.toml .`: passed.
- `uv run --extra lint ruff format --diff --config pyproject.toml .`:
passed.
- `npx prettier@3.8 --check .`: passed.
- `git diff 4235d01 HEAD --exit-code`: passed; the final tree exactly
matches the verified hosted-search-only merge. Cassettes remain
unchanged relative to `amer-hosted-web-search`.
- The offline replay workflow targets PRs against `main`, so PR #89
relies on the local suite above while its base remains
`amer-hosted-web-search`.
@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown

Coverage

Coverage Report (Python 3.14) •
FileStmtsMissCoverMissing
adapters
   anthropic.py4594690%147, 157, 195, 223, 260, 311, 316, 320, 327, 331, 335, 341, 347, 351–353, 356–358, 361, 369, 371, 376, 382, 389, 413–414, 521, 583, 609, 633, 635, 637, 697, 704, 727, 753, 756, 823, 843, 868, 872, 875, 881, 885, 939
   openai.py6466590%112, 192–194, 196, 205, 362, 367, 379, 418, 420, 457, 466, 481, 492, 517, 532, 557, 589, 593, 730, 741, 745, 776, 838, 841, 846, 905, 907, 921, 957–960, 969, 973, 980, 984, 1038, 1113, 1183, 1218, 1254, 1260, 1271, 1281, 1295, 1304, 1316, 1389, 1391, 1445, 1447, 1456, 1462, 1515, 1522, 1528, 1535, 1539, 1553, 1564, 1571, 1577, 1581
upstream
   anthropic.py5073094%98, 104, 117–118, 241, 346, 365, 396, 419, 488, 568, 588, 629, 631, 634–635, 740–741, 744, 750, 761, 770, 815, 832, 845, 849, 901–902, 937, 939
   openai.py7623895%156, 158, 185, 199, 211, 223, 287–288, 297, 428, 457, 615, 623, 625, 628, 662–663, 736, 748, 764, 804, 828, 914, 993, 1014, 1173, 1205, 1241, 1243, 1247–1248, 1387, 1429, 1433, 1446–1447, 1464, 1467
TOTAL390352886% 

Tests Skipped Failures Errors Time
2316 0 💤 0 ❌ 0 🔥 31.436s ⏱️

@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown

Coverage

Coverage Report (Python 3.12) •
FileStmtsMissCoverMissing
adapters
   anthropic.py4594690%147, 157, 195, 223, 260, 311, 316, 320, 327, 331, 335, 341, 347, 351–353, 356–358, 361, 369, 371, 376, 382, 389, 413–414, 521, 583, 609, 633, 635, 637, 697, 704, 727, 753, 756, 823, 843, 868, 872, 875, 881, 885, 939
   openai.py6466590%112, 192–194, 196, 205, 362, 367, 379, 418, 420, 457, 466, 481, 492, 517, 532, 557, 589, 593, 730, 741, 745, 776, 838, 841, 846, 905, 907, 921, 957–960, 969, 973, 980, 984, 1038, 1113, 1183, 1218, 1254, 1260, 1271, 1281, 1295, 1304, 1316, 1389, 1391, 1445, 1447, 1456, 1462, 1515, 1522, 1528, 1535, 1539, 1553, 1564, 1571, 1577, 1581
upstream
   anthropic.py5073094%98, 104, 117–118, 241, 346, 365, 396, 419, 488, 568, 588, 629, 631, 634–635, 740–741, 744, 750, 761, 770, 815, 832, 845, 849, 901–902, 937, 939
   openai.py7623895%156, 158, 185, 199, 211, 223, 287–288, 297, 428, 457, 615, 623, 625, 628, 662–663, 736, 748, 764, 804, 828, 914, 993, 1014, 1173, 1205, 1241, 1243, 1247–1248, 1387, 1429, 1433, 1446–1447, 1464, 1467
TOTAL394852887% 

Tests Skipped Failures Errors Time
2316 0 💤 0 ❌ 0 🔥 1m 1s ⏱️

@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown

Coverage

Coverage Report (Python 3.13) •
FileStmtsMissCoverMissing
adapters
   anthropic.py4594690%147, 157, 195, 223, 260, 311, 316, 320, 327, 331, 335, 341, 347, 351–353, 356–358, 361, 369, 371, 376, 382, 389, 413–414, 521, 583, 609, 633, 635, 637, 697, 704, 727, 753, 756, 823, 843, 868, 872, 875, 881, 885, 939
   openai.py6466590%112, 192–194, 196, 205, 362, 367, 379, 418, 420, 457, 466, 481, 492, 517, 532, 557, 589, 593, 730, 741, 745, 776, 838, 841, 846, 905, 907, 921, 957–960, 969, 973, 980, 984, 1038, 1113, 1183, 1218, 1254, 1260, 1271, 1281, 1295, 1304, 1316, 1389, 1391, 1445, 1447, 1456, 1462, 1515, 1522, 1528, 1535, 1539, 1553, 1564, 1571, 1577, 1581
upstream
   anthropic.py5073094%98, 104, 117–118, 241, 346, 365, 396, 419, 488, 568, 588, 629, 631, 634–635, 740–741, 744, 750, 761, 770, 815, 832, 845, 849, 901–902, 937, 939
   openai.py7623895%156, 158, 185, 199, 211, 223, 287–288, 297, 428, 457, 615, 623, 625, 628, 662–663, 736, 748, 764, 804, 828, 914, 993, 1014, 1173, 1205, 1241, 1243, 1247–1248, 1387, 1429, 1433, 1446–1447, 1464, 1467
TOTAL394852887% 

Tests Skipped Failures Errors Time
2316 0 💤 0 ❌ 0 🔥 1m 3s ⏱️

@koladefaj

koladefaj commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

hi @jiahuei, I tested follow-up requests using real unsigned DeepSeek reasoning through Messages → Chat and Responses → Chat with the summary capability profile. When I sent the first response back as conversation history, Molto rejected both requests before reaching DeepSeek. With the raw profile, both already worked.

After removing the two adapter checks and the Responses request-builder check, both summary-profile follow-ups succeeded. Live OpenAI tests with synthesized unsigned history also accepted normal follow-ups and tool-loop requests.

Anthropic behaved differently: it rejected missing or empty signatures, so keeping its upstream check matches what I observed.

Would it make sense to allow unsigned summaries during request normalization and leave signature validation to each upstream request builder, keeping the Anthropic check and Chat’s rejection of signed state? The replay tests in tests/core/test_unsigned_reasoning.py could then cover which destinations accept unsigned history and which reject it.

@iAmir97

iAmir97 commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

Automated review (Claude)

Reviewed the diff relaxing output-side reasoning-signature checks in the Anthropic/OpenAI adapters and upstream clients, plus the lazy-PyYAML fix.

The core design holds up: relaxing signature requirements only on the output/projection side while keeping request/history-replay strict was traced end-to-end, and no crash or wrong-output path was found from the removed checks themselves.

Three smaller findings:

  1. python/molto/utils/test.py:10 — still imports yaml eagerly, undermining this PR's own goal of making PyYAML test-only. A plain molto install (no [test] extra) hits the exact ModuleNotFoundError this PR claims to fix, just via this module instead of io.py.

  2. python/molto/upstream/openai.py:177 (_reasoning_part) — assigns signature=item.encrypted_content without the (signature or None) normalization used by the sibling _classify_thinking(). An upstream Responses reasoning item with encrypted_content: "" and a non-empty summary — newly reachable now that _validate_completed_responses_reasoning is removed — isn't deferred at response.output_item.added ("" is not None), and emits a spurious extra PartDeltaEvent(ThinkingDelta(signature_delta="")) right before PartEndEvent. Verified by execution. Currently inert end-to-end since every downstream adapter gates on truthiness of signature_delta, so no client-visible bug today — but it's a reproducible inconsistency the PR's own parametrized test (encrypted_content: "" case) doesn't catch, since that test only asserts the first/last events.

  3. tests/core/test_unsigned_reasoning.py:27 — re-implements _collect/_assert_lifecycle instead of reusing the equivalent helpers already in tests/core/test_web_search.py. The copies have already drifted (missing the output_tokens_details workaround, doesn't accept "response.incomplete"), and duplicate ~45 lines of SDK-stream-accumulation logic that now has to be kept in sync in two places.

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants