chore: update coding-agent plugins, and resolve trace orgs interactively - #314
Conversation
Bump the `bt-daemon` dependency from 5fbfb5e to 2bb3baf, picking up "Improve hook routing handling" (#21) and "Support bulk transcript imports" (#13). Three upstream behavior changes reach `bt`: - `TraceHostServices::resolve_route` now takes a `RouteRequirements` struct instead of a `destination_required` bool. Its new `interactive_auth` field is false for hooks, so `bt` suppresses every prompt for the rest of the process when it is unset — a missing profile or org can no longer block an agent's turn on a prompt. - Setup, managed run, and import now resolve a credential and require a resolved org before writing a route, so the tests covering them supply a synthetic credential rather than relying on ambient auth. - A managed run that produces no accepted trace events is now an error. The fake agents in the run tests emit no hook events, so those tests assert the new failure; the invocation settings and config-isolation assertions they exist for are all written before that check. `bt trace import` also gained variadic session ids and `--all`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
Latest downloadable build artifacts for this PR commit
Available artifact names
|
Tracing requires a resolved org — `require_resolved_org` in the plugin runtime rejects a route without one — but `bt` never offered a way to choose it, so `bt trace setup`/`run`/`import` hard-failed with "organization choice required for tracing" and advice to "select an organization during setup" even when setup was the command that failed. Projects already resolve interactively via `ui::select_project`. Orgs now resolve the same way, reusing `switch::select_org_for_switch`, which auto-selects when the credential has exactly one org and prompts otherwise. This is what upstream's `RouteRequirements::interactive_auth` was added to allow: hooks leave it false and keep tolerating an unresolved org, since the daemon accepts their events without one. The gap is wider than API keys. A default org comes from `--org`, the config file `bt init`/`bt switch` writes, or an org-bound API key profile; `commit_oauth_profile` stores `org_name: None`, so an OAuth profile carries no org either. Anyone who authenticated without running `bt init` hit this. Non-interactively it now reports "organization choice required in non-interactive mode; pass --org <NAME> or set BRAINTRUST_ORG_NAME", mirroring the project message. Org resolution also runs before the project picker so the projects offered belong to the org the traces will land in. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Added org resolution (22fd15e), and split the managed-run fail-open fix out to braintrustdata/braintrust-coding-agent-plugins#22. Orgs now prompt instead of failingReproduced against the built binary before the fix:
Org resolution also runs before the project picker, so the projects offered belong to the org the traces will actually land in. Where a default org actually comes fromWorth recording, because it's narrower than it looks — only three sources:
Non-interactively it now reports TestsTwo added:
810 tests pass, clippy clean, fmt clean. |
Stephen Belanger (Qard)
changed the title
Moves the `bt-daemon` pin from 2bb3baf to 1ca65d4, which adds "keep managed runs fail-open when nothing was traced" (#22). A managed run that flushes zero accepted sessions no longer discards the wrapped agent's exit status, so the three `trace run` tests go back to asserting success. Their fake agents emit no hook events, which is exactly the case that used to turn a successful agent into a failed `bt trace run`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2 participants
Bumps the
bt-daemondependency on braintrust-coding-agent-plugins to1ca65d4(latestmain), picking up:Adapting to those changes surfaced two problems worth fixing rather than absorbing. One was fixed upstream in plugins#22, which has landed and is included in this pin; the other is fixed here.
1.
resolve_routetakesRouteRequirementsTraceHostServices::resolve_routenow takes a struct instead of adestination_requiredbool. Its newinteractive_authfield is false for hooks, sobtsuppresses every prompt for the rest of the process when it is unset — a missing profile or org can no longer block an agent's turn on a prompt.base.no_input = trueinresolve_authdid not already do this:btgates prompts on a process global, whichcrate::auth::resolve_authconsults later in the same run, so that assignment was inert.2. Orgs now resolve interactively instead of hard-failing
Reproduced against the built binary before the fix:
require_resolved_orgrejects a route with no org, butbtnever offered a way to choose one — and the advice to "select an organization during setup" is circular when setup is the command that just failed. Projects already resolve interactively throughui::select_project; orgs now do the same viaswitch::select_org_for_switch, which auto-selects when the credential has exactly one org and prompts otherwise. This is what upstream'sinteractive_authwas added to allow — hooks leave it false and keep tolerating an unresolved org, since the daemon accepts their events without one.Org resolution runs before the project picker, so the projects offered belong to the org the traces will actually land in.
Where a default org actually comes from. Only three sources:
--org/BRAINTRUST_ORG_NAMEbt initandbt switchwrite (config/mod.rs,init.rs:78)org_constraint()requiresauth_kind == ApiKey && org_bound == Some(true)commit_oauth_profilestoresorg_name: None, org_bound: Some(false), so an OAuth profile carries no org at all. This was never an API-key-only gap: anyone who authenticated without runningbt inithit it, OAuth included.Non-interactively it now reports
organization choice required in non-interactive mode; pass --org <NAME> or set BRAINTRUST_ORG_NAME, mirroring the existing project message.3. Test changes
Credentials. Setup, managed run, and import now resolve a credential and require a resolved org before writing a route. The affected tests previously ran with no auth at all; they now go through
bt_trace_command(), which supplies a synthetic key and org. The two that pass an explicit--profilealso seed a synthetic auth store and secret so they resolve offline.Import help. Bulk import made session ids variadic, so the assertions moved from
<SESSION_ID>to[SESSION_ID]...and now also cover--all.Added.
trace_commands_require_an_org_when_the_credential_resolves_none— bare API key, no org,--no-input, across setup and runtrace_setup_adopts_the_configured_org_without_prompting— the config-file org is adopted silently;--no-inputmakes any attempted prompt a failure rather than a hangThe interactive org prompt itself is not covered — that needs a TTY and a live org list, which would mean faking
resolve_org_optionsbehind a seam.The three
trace runtests still assert.success(). Their fake agents emit no hook events, which is precisely the case plugins#22 fixed, so they double as end-to-end confirmation that the fail-open fix works throughbt.Testing
cargo build,cargo clippy --all-targets(no new warnings),cargo fmt --check, andcargo testall pass — 810 tests, including all 41 CLI tests.🤖 Generated with Claude Code