fix(pydantic-ai): Capture reasoning and OTEL output messages - #7010
fix(pydantic-ai): Capture reasoning and OTEL output messages#7010vgrozdanic wants to merge 4 commits into
Conversation
Capture ThinkingPart as reasoning, write gen_ai.output.messages on chat spans, and record provider reasoning token usage. Align request text parts to content and stop dual-writing deprecated gen_ai.response.text. Co-Authored-By: opencode <noreply@opencode.ai>
vgrozdanic
changed the title
Codecov Results 📊✅ 95072 passed | ⏭️ 6343 skipped | Total: 101415 | Pass Rate: 93.75% | Execution Time: 338m 57s 📊 Comparison with Base Branch
All tests are passing successfully. ✅ Patch coverage is 93.22%. Project has 2516 uncovered lines. Files with missing lines (1)
Coverage diff@@ Coverage Diff @@
## main #PR +/-##
==========================================
+ Coverage 89.88% 89.89% +0.01%
==========================================
Files 193 193 —
Lines 24834 24876 +42
Branches 8912 8936 +24
==========================================
+ Hits 22321 22360 +39
- Misses 2513 2516 +3
- Partials 1417 1422 +5Generated by Codecov Action |
![sentry-warden[bot]](https://avatars.githubusercontent.com/in/2755838?s=60&v=4){kind=small}
gen_ai.request.messages text and reasoning parts must use the text key so truncate_and_annotate_messages still truncates long prompts. Output messages continue to use content per the OTEL shape. Co-Authored-By: opencode <noreply@opencode.ai>
Tool calls already live on gen_ai.output.messages; drop the second attribute on chat spans. Co-Authored-By: opencode <noreply@opencode.ai>
Stop setting deprecated gen_ai.response.text on invoke_agent spans; use the OTEL output messages shape instead. Co-Authored-By: opencode <noreply@opencode.ai>
| output_tokens=50, | ||
| details={details_key: 12}, | ||
| ) | ||
| _set_usage_data(span, usage) |
There was a problem hiding this comment.
I cannot find a single new test that uses the pydantic-ai package as a user would. All the tests are full of unnecessary mocks.
I want to know that your changes work with pydantic-ai, not some mocks you made up.
In it's current state, it doesn't make sense to review the PR further.
Imports in the middle of function w/o justification, most things Any-typed, old style type annotation which the repo doesn't use anymore should all be removed as well.
There was a problem hiding this comment.
This is following existing test patterns, unless you are proposing refactor of all the tests for this package, which is IMO out of the scope for this small bug fix.
If you prefer, I can close this PR, and open an issue in sentry-python to support reasoning and proper tool calls format
2 participants
Pydantic AI spans now record assistant output only as
gen_ai.output.messages(text, reasoning, and tool calls on chat spans; final agent text on invoke_agent).ThinkingPartis surfaced as reasoning in request history and model responses, and provider reasoning token usage is attached when the count is greater than zero.Deprecated
gen_ai.response.textandgen_ai.response.tool_callsare not set. Request message parts keep the existingtextkey so truncation and consumers ofgen_ai.request.messagesstay compatible.