diff --git a/CHANGELOG.md b/CHANGELOG.md index 370c33e..1de2a6c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,15 +1,38 @@ # Changelog -## Unreleased +## 0.5.5 (2026-11-04) + +### Behaviour changes + +- **Piped stdin is read only when requested**: when a prompt is supplied, `aictrl run` no longer reads non-terminal stdin unless a standalone `-` is one of its message operands. For example, use `git diff origin/main...HEAD | aictrl run "Summarize the changes" -` to include piped context. With no message operands, non-terminal stdin is still read to EOF. Selected stdin has no read timeout; close the producer or apply your own deadline. A `-` used as an option value, such as `--model -` or `--command -`, does not select stdin. (#256) +- **Config loading stops inserting `$schema`**: loading `aictrl.json` or `aictrl.jsonc` no longer inserts a `$schema` property. A `tui.json` created while migrating TUI configuration also omits the inserted property. Existing `$schema` values remain unchanged. (#256) +- **Standalone upgrades use npm installation instructions**: a binary outside a package manager's installation tree no longer downloads and runs a remote installer during `aictrl upgrade`. It stops with `npm install -g @aictrl/cli@` instead. Installations owned by npm, pnpm, or Bun continue to upgrade through their package manager. Standalone version lookups use the public npm registry and reject responses without a valid version. (#261) ### Features +- **Select work tools per run**: `aictrl run --tools read,glob,grep` selects exact registered work-tool names, and `--tools ''` selects none. The optional `tool_allowlist` configuration sets a default that the CLI option replaces for one invocation. Selection only narrows availability; permission denials still apply. Continuation retains the selection, and delegated tasks inherit its ceiling. Unknown or unavailable names fail before inference, and a selection with `--attach` fails before contacting the receiver. (#254) +- **Command arguments retain their boundaries**: `--command` passes operands as an argument list and selected stdin as a separate final value, preserving empty, quoted, and multiline operands. SDK callers can still supply the existing `arguments` string. (#256) - **GPT-6 models default to `low` text verbosity** — `gpt-6.1-sol`, `gpt-6-astra`, `gpt-6-sol` and `gpt-6-luna` now send `text.verbosity: "low"` on the Responses API, as GPT-5.x models already did. GPT-5.x and GPT-6 now share one rule, adapted from opencode v2 (`plugin/verbosity.ts`). It applies on `@ai-sdk/openai`, on `@ai-sdk/azure` and on `openai/*` models on the Vercel AI Gateway, and skips `-chat`, `-image` and `codex` ids. A `textVerbosity` in the model's `options` or in a variant in `aictrl.json` still overrides the default. (#243) ### Fixes +- **Structured output works with Claude revisions that reject forced tool choice**: schema-mode requests targeting Claude Sonnet 5.5, Opus 5.5, Fable 5.1, or Mythos 5.1 now use automatic tool choice for the initial request and corrective turns. Matching uses the API model ID, including aliases and routed provider IDs. The model must still call `StructuredOutput` with a result-file path; schema validation, retry limits, missing-result failures, and permission checks remain in force. Other model revisions retain their existing forced tool choice. (#262) +- **Built-in guidance follows the available tools**: system instructions and recovery messages no longer tell the model to delegate, search, or edit with tools absent from its request catalog. Custom instructions and plugin text remain unchanged. (#252) - **GPT-5.x no longer sends `textVerbosity` to providers that do not take OpenAI's option** — GPT-5.x ids on OpenRouter, OpenAI-compatible gateways (such as kilo and zenmux), Bedrock and other non-OpenAI packages used to get `textVerbosity: "low"` only because their id contained `gpt-5.`. They now leave it unset. Upstream found OpenRouter's GPT endpoints report no verbosity support. GPT-5.x on Azure (`@ai-sdk/azure`) now gets `low`, as in upstream. Before, only providers with the id `azure` were skipped, so `azure-cognitive-services` already sent it. `-image` ids are now left unset like `-chat` ids. (#243) +### Security + +- **Provider commands require confirmation**: `aictrl auth login ` validates the provider's `.well-known/aictrl` document, rejects redirects, and requires HTTPS except for HTTP on `localhost`, `127.0.0.1`, or `[::1]`. It displays the command and asks before executing it. The default answer is No, so non-interactive use cancels. (#215) +- **Codex sign-in removes PKCE verifier bias**: the verifier is the base64url encoding of 32 random bytes instead of selecting characters with modulo arithmetic. (#215) +- **Command-template shell spans treat caller arguments as data**: `$N` and `$ARGUMENTS` inside a template's shell spans are passed as shell data. Operands, selected stdin, and shell output cannot introduce another shell span or another round of placeholder substitution. Shell syntax written in the original template continues to run. (#256) +- **Project isolation also skips checkout instruction files**: `AICTRL_DISABLE_PROJECT_CONFIG` now skips project and nested instruction files, including `.opencode/AGENTS.md`. Global instructions and explicitly configured instructions remain available. (#256) + +### Documentation and internal changes + +- The README documents the headless execution contracts, tool selection, published benchmark results, and a GitHub Actions example that keeps the model key and GitHub token in separate steps. A released-consumer smoke example verifies the published CLI. `SECURITY.md` describes private vulnerability reporting. (#249, #253, #258) +- Source CI checks reference-token budgets and retains raw captures. The release procedure separately requires a candidate-versus-previous native binary comparison with pinned artifact provenance and reviewed token deltas. These checks make no provider-billing or model-quality claim. (#255) +- Package and documentation links now point to `aictrl.dev`. Outgoing OpenRouter, Vercel AI Gateway, ZenMux, and Kilo metadata uses `https://aictrl.dev/`, and MCP OAuth registration uses `https://aictrl.dev/agent` as `client_uri`. The retired curl installer is removed; npm remains the supported installation route. (#261) + ## 0.5.4 (2026-10-19) ### Features diff --git a/bun.lock b/bun.lock index 06289bb..4732a28 100644 --- a/bun.lock +++ b/bun.lock @@ -22,7 +22,7 @@ }, "packages/cli": { "name": "@aictrl/cli", - "version": "0.5.4", + "version": "0.5.5", "bin": { "aictrl": "./bin/aictrl", }, diff --git a/packages/cli/package.json b/packages/cli/package.json index 5148841..1750818 100644 --- a/packages/cli/package.json +++ b/packages/cli/package.json @@ -1,6 +1,6 @@ { "$schema": "https://json.schemastore.org/package.json", - "version": "0.5.4", + "version": "0.5.5", "name": "@aictrl/cli", "description": "The headless agent harness for CI, cron jobs, and container workflows", "type": "module", diff --git a/packages/cli/src/provider/transform.ts b/packages/cli/src/provider/transform.ts index 38c7c21..e1827c2 100644 --- a/packages/cli/src/provider/transform.ts +++ b/packages/cli/src/provider/transform.ts @@ -76,6 +76,15 @@ function mimeToModality(mime: string): Modality | undefined { } export namespace ProviderTransform { + export function supportsForcedToolChoice(model: Provider.Model) { + // These Claude revisions reject forced tools even with adaptive thinking. + // Match the API target so aliases and routed provider IDs behave alike. + // https://platform.claude.com/docs/en/agents-and-tools/tool-use/define-tools + return !/(?:^|[/.:])claude-(?:(?:sonnet|opus)-5[-.]5|(?:fable|mythos)-5[-.]1)(?:$|[-@:.])/.test( + model.api.id.toLowerCase(), + ) + } + export const OUTPUT_TOKEN_MAX = Flag.AICTRL_EXPERIMENTAL_OUTPUT_TOKEN_MAX || 32_000 export function isGeminiTarget(model: Provider.Model) { diff --git a/packages/cli/src/session/prompt.ts b/packages/cli/src/session/prompt.ts index f0343a6..e993add 100644 --- a/packages/cli/src/session/prompt.ts +++ b/packages/cli/src/session/prompt.ts @@ -66,6 +66,12 @@ const STRUCTURED_OUTPUT_SYSTEM_PROMPT = `IMPORTANT: The user has requested struc export namespace SessionPrompt { const log = Log.create({ service: "session.prompt" }) + function structuredToolChoice(model: Provider.Model, reminder?: string) { + if (!ProviderTransform.supportsForcedToolChoice(model)) return "auto" as const + if (reminder) return { type: "tool" as const, toolName: "StructuredOutput" } + return "required" as const + } + export function hasToolCalls(parts: MessageV2.Part[]) { return parts.some( (part) => @@ -851,7 +857,7 @@ export namespace SessionPrompt { ], tools, model, - toolChoice: contract ? (reminder ? { type: "tool", toolName: "StructuredOutput" } : "required") : undefined, + toolChoice: contract ? structuredToolChoice(model, reminder) : undefined, structured: contract ? { reject, diff --git a/packages/cli/test/session/output-schema.test.ts b/packages/cli/test/session/output-schema.test.ts index 25cb9ba..8ae8ab3 100644 --- a/packages/cli/test/session/output-schema.test.ts +++ b/packages/cli/test/session/output-schema.test.ts @@ -35,6 +35,7 @@ async function run( denied?: boolean beside?: boolean invalid?: boolean + anthropic?: string } = {}, ) { const requests: Record[] = [] @@ -47,6 +48,20 @@ async function run( port: 0, async fetch(request) { requests.push((await request.json()) as Record) + if (options.anthropic === "claude-sonnet-5-5") { + const choice = requests.at(-1)?.tool_choice as { type?: string } | undefined + if (choice?.type === "any" || choice?.type === "tool") + return Response.json( + { + type: "error", + error: { + type: "invalid_request_error", + message: 'tool_choice: type "tool" and "any" are not supported for this model.', + }, + }, + { status: 400 }, + ) + } if ( requests.length > (inputs.length === 1 && inputs[0] === "{}" ? retryCount + 1 : retryCount + inputs.length + 1) ) @@ -88,6 +103,49 @@ async function run( calls.push({ index: calls.length, id: `read_${requests.length}`, type: "function", function: read }) else calls[0].function = read } + if (options.anthropic) { + const prose = args === "prose" + const chunks = [ + { + type: "message_start", + message: { + id: `msg_${requests.length}`, + type: "message", + role: "assistant", + model: options.anthropic, + content: [], + stop_reason: null, + stop_sequence: null, + usage: { input_tokens: 1, output_tokens: 0 }, + }, + }, + { + type: "content_block_start", + index: 0, + content_block: prose + ? { type: "text", text: "" } + : { type: "tool_use", id: calls[0].id, name: "StructuredOutput", input: {} }, + }, + { + type: "content_block_delta", + index: 0, + delta: prose + ? { type: "text_delta", text: "No final tool call" } + : { type: "input_json_delta", partial_json: calls[0].function.arguments }, + }, + { type: "content_block_stop", index: 0 }, + { + type: "message_delta", + delta: { stop_reason: prose ? "end_turn" : "tool_use", stop_sequence: null, container: null }, + usage: { input_tokens: 1, output_tokens: 1 }, + }, + { type: "message_stop" }, + ] + return new Response( + chunks.map((chunk) => `event: ${chunk.type}\ndata: ${JSON.stringify(chunk)}\n\n`).join(""), + { headers: { "Content-Type": "text/event-stream" } }, + ) + } const chunks = [ { choices: [{ index: 0, delta: { role: "assistant" }, finish_reason: null }] }, { @@ -196,16 +254,31 @@ async function run( config: { ...(options.steps ? { agent: { build: { steps: options.steps } } } : {}), ...(options.compact ? { compaction: { auto: true } } : {}), - enabled_providers: options.gemini ? ["fixture"] : ["alibaba"], - provider: options.gemini + enabled_providers: options.gemini || options.anthropic ? ["fixture"] : ["alibaba"], + provider: options.anthropic ? { fixture: { - npm: "@ai-sdk/openai-compatible", + npm: "@ai-sdk/anthropic", options: { apiKey: "test-key", baseURL: `${server.url.origin}/v1` }, - models: { "gemini-fixture": { name: "fixture", limit: { context: 100000, output: 1000 } } }, + models: { + [options.anthropic]: { + name: "Anthropic fixture", + tool_call: true, + reasoning: false, + limit: { context: 100000, output: 1000 }, + }, + }, }, } - : { alibaba: { ...alibabaProvider, options: { apiKey: "test-key", baseURL: `${server.url.origin}/v1` } } }, + : options.gemini + ? { + fixture: { + npm: "@ai-sdk/openai-compatible", + options: { apiKey: "test-key", baseURL: `${server.url.origin}/v1` }, + models: { "gemini-fixture": { name: "fixture", limit: { context: 100000, output: 1000 } } }, + }, + } + : { alibaba: { ...alibabaProvider, options: { apiKey: "test-key", baseURL: `${server.url.origin}/v1` } } }, }, }) return await Instance.provide({ @@ -250,9 +323,11 @@ async function run( try { const result = await SessionPrompt.prompt({ sessionID: session.id, - model: options.gemini - ? { providerID: "fixture", modelID: "gemini-fixture" } - : { providerID: "alibaba", modelID: "qwen-plus" }, + model: options.anthropic + ? { providerID: "fixture", modelID: options.anthropic } + : options.gemini + ? { providerID: "fixture", modelID: "gemini-fixture" } + : { providerID: "alibaba", modelID: "qwen-plus" }, parts: [{ type: "text", text: "Return a result." }], ...(options.plain ? {} @@ -296,6 +371,53 @@ test("missing required field crosses real tool boundary and must be repaired bef expect(result.info.error).toBeUndefined() }, 15_000) +test("Sonnet 5.5 uses auto through validation and missing-output corrective turns", async () => { + const result = await run(["{}", "prose", '{"result":"repaired"}'], 2, { anthropic: "claude-sonnet-5-5" }) + expect(result.requests.map((request) => request.tool_choice)).toEqual([ + { type: "auto" }, + { type: "auto" }, + { type: "auto" }, + ]) + expect(result.requests.every((request) => request.model === "claude-sonnet-5-5")).toBe(true) + expect(result.info.error).toBeUndefined() + expect(result.info.structured).toEqual({ result: "repaired" }) + expect(result.rejected.map((event) => event.errors[0].keyword)).toEqual(["required", "missing"]) + expect(result.outcomes).toEqual([{ status: "accepted", attempts: 3, value: { result: "repaired" } }]) +}, 15_000) + +test("Sonnet 5.5 auto tool choice still enforces the invalid-output retry budget", async () => { + const result = await run(["{}"], 2, { anthropic: "claude-sonnet-5-5" }) + expect(result.requests.map((request) => request.tool_choice)).toEqual([ + { type: "auto" }, + { type: "auto" }, + { type: "auto" }, + ]) + expect(result.rejected.map((event) => event.attempt)).toEqual([1, 2, 3]) + expect(result.info.structured).toBeUndefined() + expect(result.info.error).toMatchObject({ name: "StructuredOutputError", data: { retries: 2 } }) + expect(result.outcomes).toEqual([{ status: "failed", reason: "exhausted", attempts: 3 }]) +}, 15_000) + +test("Sonnet 5.5 auto tool choice still fails when structured output is missing", async () => { + const result = await run(["prose"], 1, { anthropic: "claude-sonnet-5-5" }) + expect(result.requests.map((request) => request.tool_choice)).toEqual([{ type: "auto" }, { type: "auto" }]) + expect(result.rejected.map((event) => event.errors[0].keyword)).toEqual(["missing", "missing"]) + expect(result.info.structured).toBeUndefined() + expect(result.info.error).toMatchObject({ name: "StructuredOutputError", data: { retries: 1 } }) + expect(result.outcomes).toEqual([{ status: "failed", reason: "missing", attempts: 2 }]) +}, 15_000) + +test("compatible Anthropic model retains required initial and forced corrective tool choice", async () => { + const result = await run(["prose", '{"result":"repaired"}'], 1, { anthropic: "claude-sonnet-4-5" }) + expect(result.requests.map((request) => request.tool_choice)).toEqual([ + { type: "any" }, + { type: "tool", name: "StructuredOutput" }, + ]) + expect(result.info.error).toBeUndefined() + expect(result.info.structured).toEqual({ result: "repaired" }) + expect(result.outcomes).toEqual([{ status: "accepted", attempts: 2, value: { result: "repaired" } }]) +}, 15_000) + for (const [name, value] of [ ["wrong scalar type", { result: 42 }], ["wrong nested type", { result: "valid", nested: { count: "42" } }],