Repository navigation
Include session and turn IDs in CLI setup #1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,105 @@ | ||
| # Session and turn IDs during application setup | ||
|
|
||
| Read this alongside the selected SDK integration reference. Connecting a gateway URL does | ||
| not identify which requests belong to one task. During setup, discover the | ||
| application's execution and turn boundaries, attach their IDs to each relevant | ||
| model request, and verify propagation with synthetic local checks. | ||
|
|
||
| ## Choose the boundaries | ||
|
|
||
| | Header | Application meaning | Lifetime | | ||
| | --- | --- | --- | | ||
| | `x-understudy-session-id` | One conversation or task execution, as defined by the application | Reuse across its model calls, tool continuations, retries, and fallback calls. Start a new ID for a new independent execution. | | ||
| | `x-understudy-turn-id` | One logical step within that session, such as a user message and all assistant work it triggers | Reuse across that step's model calls and retries. Change when the next logical step begins. | | ||
|
|
||
| A model request or tool-loop iteration is not automatically a new turn. For a | ||
| one-shot task, the session has one turn: use the same value for both IDs, or | ||
| omit the turn header. Prefer both for applications with multiple turns. | ||
| Strongly recommend session IDs for session-level evaluation; online model | ||
| experiments require them for stable assignment. | ||
|
|
||
| Reuse existing opaque execution/conversation and message/step IDs when their | ||
| lifetimes match these definitions. A user/account ID, workload name, or | ||
| process-wide constant is not an execution ID. If suitable IDs do not exist, | ||
| allocate them at the application entrypoint and logical-turn boundary, then | ||
| carry them through the existing context, queue messages, and retry paths. | ||
| Persist them with an existing resumable job when necessary; do not regenerate | ||
| them inside the model-call helper or on job redelivery. | ||
|
|
||
| Both IDs are opaque and case-sensitive, must contain non-whitespace text, and | ||
| must fit within 256 UTF-8 bytes. Prefer header-safe ASCII IDs such as UUIDs. | ||
| Do not truncate, lowercase, or derive IDs from prompts, email addresses, or | ||
| other sensitive content. Neither header signals completion or proves an | ||
| outcome. They also do not replace the gateway's per-request | ||
| `x-understudy-request-id`; retain that separately when the app already records | ||
| request correlation. | ||
|
|
||
| Before editing, describe the chosen boundaries, where each ID comes from, | ||
| and which call sites will receive them. If the framework cannot propagate | ||
| context or the boundary is unresolved, report that gap instead of inventing | ||
| IDs per request or claiming complete session attribution. | ||
|
|
||
| ## Attach at request scope | ||
|
|
||
| Keep credentials and base URLs on the shared client. Add session and turn | ||
| headers through the installed SDK's per-request options, merging existing | ||
| request headers. Never mutate shared client defaults to switch sessions: | ||
| concurrent executions can otherwise inherit one another's IDs. | ||
|
|
||
| Illustrative TypeScript fragments below use IDs already carried in the | ||
| application's context; they do not create IDs inside the inference loop. | ||
| Keep the application's existing model, messages, tools, and stream handling. | ||
| These fragments assume `existingOptions.headers` is a plain object; if the app | ||
| uses `Headers` or header tuples, merge using that representation instead of | ||
| object spread. | ||
|
|
||
| ```ts | ||
| const attributionHeaders = { | ||
| "x-understudy-session-id": context.sessionId, | ||
| "x-understudy-turn-id": context.turnId, | ||
| }; | ||
|
|
||
| // OpenAI SDK: the second argument contains transport options, not model input. | ||
| const result = await openai.chat.completions.create( | ||
| existingParams, | ||
| { ...existingOptions, headers: { ...existingOptions.headers, ...attributionHeaders } }, | ||
| ); | ||
| ``` | ||
|
|
||
| ```ts | ||
| // Anthropic SDK: keep consuming the stream as the application already does. | ||
| const stream = anthropic.messages.stream( | ||
| existingParams, | ||
| { ...existingOptions, headers: { ...existingOptions.headers, ...attributionHeaders } }, | ||
| ); | ||
| ``` | ||
|
|
||
| For Vercel AI SDK, use the generation call's `headers` option. For Mastra, | ||
| LangChain, LlamaIndex, and other wrappers, inspect the installed version's | ||
| supported per-invocation header/context hook and verify that it reaches the | ||
| underlying model requests, including tool continuations. Framework memory | ||
| thread IDs do not automatically become Understudy headers. Do not invent | ||
| framework options or replace a shared client with mutable session defaults. | ||
| For raw HTTP, merge into the request's headers using its existing header | ||
| representation (`Headers`, tuples, or an object). | ||
| For the direct Python OpenAI and Anthropic clients, use `extra_headers` on the | ||
| existing model call, merging the application's existing extra headers. | ||
|
|
||
| ## Verify and report | ||
|
|
||
| Use a local fake transport or existing integration tests with wholly synthetic | ||
| IDs and payloads. Inspect outbound headers and check: | ||
|
|
||
| - two calls in one turn, including a tool continuation or retry, retain both IDs; | ||
| - the next turn retains the session ID and changes the turn ID; | ||
| - a new independent execution changes the session ID; | ||
| - interleaved concurrent executions keep their own IDs; | ||
| - background jobs, fallbacks, and resumptions preserve the intended context; | ||
| - existing authentication, workload headers, payloads, and stream behavior remain | ||
| intact. | ||
|
|
||
| Report the boundary mapping, verified call paths, and any missing propagation. | ||
| Local checks prove header construction, not hosted capture persistence. If a | ||
| bounded live verification is already authorized, inspect the resulting | ||
| metadata through the supported capture surface without printing private | ||
| payloads; otherwise leave hosted verification explicitly pending. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.