diff --git a/.claude/skills/seclai-changelog/SKILL.md b/.claude/skills/seclai-changelog/SKILL.md index 3365c87..638d022 100644 --- a/.claude/skills/seclai-changelog/SKILL.md +++ b/.claude/skills/seclai-changelog/SKILL.md @@ -64,13 +64,13 @@ forward. ## Determine the version -Do not guess the next version. Releases are cut by `seclai/github-tag-action` in `.github/workflows/main-build.yaml`, which reads the **merge commit message**: +Do not guess the next version. Releases are cut by `seclai/github-tag-action` in `.github/workflows/main-build.yaml`, which scans the **commit messages** for a bump keyword. The PR title is not scanned: - contains `#major` → major bump - contains `#minor` → minor bump - otherwise → `DEFAULT_BUMP: patch` -So a change that adds endpoints must say `#minor` in a commit message, or the heading you write will not match the tag that gets cut. The keyword goes in the commit message, not the PR title. Confirm the intended bump with the user when it isn't stated, and flag the mismatch risk if no commit message carries the keyword. +Put `#minor` / `#major` in a branch commit message; never tell the user that the PR title will do. The keyword reaches `main` in the squash commit's message, so keep it when editing the message in the merge dialog. A change that adds endpoints needs `#minor` in a commit message, or it ships as a patch and the heading you write will not match the tag that gets cut. Confirm the intended bump with the user when it isn't stated, and flag the mismatch risk if no branch commit message carries the keyword. Check `git tag --sort=-v:refname | head -1` for the current latest, then apply the bump. diff --git a/.claude/skills/seclai-sdk-review-triage/SKILL.md b/.claude/skills/seclai-sdk-review-triage/SKILL.md new file mode 100644 index 0000000..9ea8150 --- /dev/null +++ b/.claude/skills/seclai-sdk-review-triage/SKILL.md @@ -0,0 +1,181 @@ +--- +name: seclai-sdk-review-triage +description: Triage PR review comments in a Seclai SDK repo (seclai-python, seclai-javascript, seclai-go, seclai-csharp, seclai-cli, seclai-mcp) or in sdk-tools, by evidence class, by how many SDKs share the site, and by whether it touches published surface, into fix here / fix in every SDK / ask / nothing / upstream / push back. Use when addressing review comments or unresolved PR threads, when a /code-review result is pasted in, before replying to or resolving threads, before reporting a review round, and when review rounds keep finding new problems. +--- + +# Triaging SDK review comments + +> **Vendored file — do not edit in place.** The canonical copy lives in the +> `seclai/sdk-tools` repository at `skills/seclai-sdk-review-triage/`, and is +> mirrored into each SDK repo with `git subtree`. Edits made here are reported as +> drift by `sdk-tools/sync.sh --check` and are overwritten on the next pull. + +Adapted from the `seclai` repo's `pr-review-triage` for SDK work, where the stakes +are published surface and wire behaviour, and one finding usually exists in four +SDKs at once. + +**Fetch the PR's own threads first.** A pasted `/code-review` result is a second +input, not a substitute: + +``` +gh api graphql -f query='{ repository(owner:"seclai",name:"") { + pullRequest(number:) { reviewThreads(first:100) { nodes { + id isResolved comments(first:1){ nodes { path line databaseId body } } } } } }' +``` + +**State what the PR is for before reading a finding**, a sentence each: its goal +(sync to which spec, plus any backlog the user agreed to), the design it chose, and +what it ruled out. Weigh every finding against that. A finding that re-opens a +decision already made in this PR or an earlier round is answered by citing the +decision, not by changing the code back. + +**A spec-sync PR changes only what the spec diff changed**, plus backlog the user +agreed to before the review. A Proven finding about behaviour that is already in the +last release goes to an issue, however cheap the fix looks, and the thread links it; +other classes keep their §1 disposition. A Contract finding on prose this PR wrote is +still fixed here. + +**Read `rules.md` next to this file** (each SDK `CLAUDE.md` imports it). Its +cold-review gate runs on your response diff before you reply (§7). + +## 1. Classify each finding + +| Class | Test | Disposition | +|---|---|---| +| **Proven** | You can write a failing test today from input that occurs | Fix (§3) | +| **Contract** | A published statement is false: README, CHANGELOG, docstring, SKILL.md, a comment | Make it true once, by the smallest change; deleting the claim counts | +| **Latent** | Provably wrong when reached; you can name the absent condition | **Nothing**, unless it would ship a break or a wrong request to consumers (§4) | +| **Speculative** | Hypothesised from code shape | Nothing | +| **Opinion** | Naming, reuse, dedupe, where a rule lives, a narrower type | Internals: **nothing**. New public surface: **ask** (§2) | +| **Consequence** | A true statement about an effect elsewhere | Record it in the reply | + +The class is yours to assign; a label arriving with the finding is an input. + +**Check that the finding is right before acting.** Name the command whose output +would prove your answer wrong, and run it: build the consumer, run the audit, read +the backend handler. Confident it is wrong: push back and change nothing. Unsure: +say which part you doubt, then act. Right diagnosis with a wrong suggested fix: take +the diagnosis, replace the repair, say both. + +## 2. What it touches decides the process + +| It touches | Then | +|---|---| +| **Published surface**: a public name or signature, what goes on the wire (query keys, null vs omitted, body shape), what a response deserializes into, version constants, a CHANGELOG heading or **Breaking:** label | **Decide, don't patch.** Check `surface` against the last tag; ask the user when the fix trades one consumer outcome for another | +| **Internal code** in this PR | Fix if proven or contract; otherwise nothing | +| **Trivial**: an internal comment, a test name, helper dedupe, a log line | One pass (delete, or the smallest true change) or nothing. Never a second pass | + +**Unreleased public surface can still change freely; released surface cannot.** +An opinion on a public name added in this PR is a decision to make now, with the +user, because a rename after release is breaking. The same opinion on a released +name is a major-version question. + +**A finding on generated code** (`seclai/_generated/`, `src/openapi.ts`, +`generated/seclai.gen.go`) is fixed in the generator's flags or upstream in the +spec, never in the output. + +**A spec defect goes upstream.** An undocumented null, a response schema that inlines +a string instead of `$ref`-ing the enum, untyped list items: reply on the thread, +and suggest the spec fix in `seclai`. Don't work around it in four SDKs unless the +user asks. + +## 3. One SDK, or all of them? + +**Count the sites across the sibling SDKs before fixing.** Most findings are about a +behaviour the four SDKs share: an optional `agent_id`, how `days` and the dates +combine, how a null clears a field. Fixing it in one SDK leaves three wrong and +invites the next round to find them. Check the siblings, fix them all the same way +in this round, or say the count in the reply ("one of four SDKs"). + +Two or more sites, or a fix that needed another fix where both changed behaviour, +means a design question first: give the options and a recommendation, and let the +user choose before any code. + +## 4. Latent findings and the audit tool + +A latent finding gets **nothing** unless learning it the breaking way would ship a +break or a wrong request to consumers, and nothing else would catch it first. Say +the assessment on the thread, so the reviewer knows it was weighed, not missed. + +For `sdksync.py` in sdk-tools, sdk-tools' CLAUDE.md governs. In short: `surface` and +`returns` report what they cannot read as UNREADABLE, so a parsing limit there is a +false alarm, never a silent miss. A latent parsing finding is not reportable unless +it gets past that guard. The stop condition is **no finding that changes an audit +result on a real SDK file**. Check it by running the audits on all four checkouts +before and after. + +## 5. Measure the round before fixing it + +Signals that the rounds are churning instead of converging: + +- the finding count stays flat or rises across three rounds; +- a fix needed another fix, and both changed behaviour; +- the response diff approaches the size of the change under review; +- more than half the findings land on the previous round's fixes (on a full-diff + review; a delta review lands there by construction). + +**Two firing, or the fix-needs-a-fix one alone: stop fixing finding by finding.** +Name the signal, name the requirement or design that is unclear, propose the way +out (often: make the failure mode loud, so latent cases become visible false +alarms; or split the adjacent work), and let the user decide. + +**The default way out is to freeze the mechanism.** When a fix to a mechanism (a +helper's timeout, fallback, retry or error rules) needs a fix, and both changed +behaviour, propose that this PR stop changing it: later findings on it go to one +issue listing every affected repo, and a fix that created a regression is reverted +rather than fixed forward. + +After the first round, review only the delta since the last reviewed commit, not +`HEAD~1..HEAD` counted by hand. + +## 6. Your own fixes + +**Delete, or add a guard. Do not add a named concept** (a new function, type, +option, or config key) to the code under review to answer a finding. Each new name is +surface for the next round. If a fix seems to need one, stop and say so. + +**Do not widen scope to make a test easy.** Supporting a shape no SDK has, because a +test needed it, is new work with its own review rounds. + +**A fix that adds or changes a mechanism is a new change, not a repair**: a new +option, timeout, fallback, retry, client or error type. Do not write it in answer to +a finding. Write down what it must do for each case (each way a stream or poll ends, +and its outcome, per SDK), agree it with the user, then implement it in every SDK +at once, in its own PR unless the user puts it in this one. Without that, each review round re-derives the design and finds the next +edge case. + +**A test added for a finding must fail without the fix.** Undo the fix, watch the +test fail, restore it. Say that you did. That proves the fix does what it says, not +that it is right; when the cold-review gate runs, its Break prompt checks the second. + +## 7. Replies and threads + +- One reply per thread, posted after lint, tests and the cold-review gate on the + response diff (`rules.md`) pass. The user commits and pushes, so say the change is in + the working tree. +- **Resolve only after the user reports the push** and + `gh pr view --json headRefOid` shows the new head, and only threads you changed. + A pushback, a doubt, or a question for the user stays open unless the user says to + resolve it. +- **Lead with the disagreement** when there is one. Dispute the claim, not the + reviewer. +- A review written against unpushed commits has no threads. Reply with one PR + comment once the commits are pushed. + +## 8. Reporting the round + +Lead with the class counts and how many findings changed behaviour. For example: +"9 findings: 2 proven, 4 latent, 1 contract, 2 opinion; 2 changed behaviour". When +none changed behaviour, say so in the first line: that is the stop condition. Then +list the pushbacks and doubts before the fixes. Report the classes you assigned, +not the ones the review claimed. Say what the cold-review gate found on the +response diff. End with: + +``` +Findings that change behaviour: N +Sibling SDKs checked: N of 4 — fixed in N +Pushed back / asked: N +Below the bar: N — said on the thread +Fixes verified by undoing them: N of N +Cold-review gate on the response diff: N findings, N changed behaviour | skipped (<10 lines) +``` diff --git a/.claude/skills/seclai-sdk-review-triage/rules.md b/.claude/skills/seclai-sdk-review-triage/rules.md new file mode 100644 index 0000000..05e84e6 --- /dev/null +++ b/.claude/skills/seclai-sdk-review-triage/rules.md @@ -0,0 +1,77 @@ +# Rules for changing and reviewing an SDK repo + +> **Vendored file — do not edit in place.** The canonical copy lives in +> `seclai/sdk-tools` at `skills/seclai-sdk-review-triage/rules.md`. Each SDK +> repo's `CLAUDE.md` imports its own vendored copy. + +Adapted from the `seclai` repo's `CLAUDE.md`. These rules apply to every task in an +SDK repo, whether you are writing the change or reviewing one. In sdk-tools they +arrive through the triage skill, and its own `CLAUDE.md` rules for audit findings +take precedence. The gates before this one are each repo's lint and test targets. + +## The cold-review gate: review your own diff before you report + +**Run it before you report a change as done, and before you draft replies to a +review round.** A reply is a published claim, and the next cold review finds what +this gate skips. Skip it for a diff under ~10 changed lines, and say so. Skip it when +you are a subagent: the agent that reports to the user runs it on the whole diff it +reports, including what subagents wrote. Do not report completion while a subagent +you started is still running. + +Delegate it, scoped to the diff being reported. Subagents get this file but not your +reasoning, and not inheriting your belief is the point. Run four prompts in parallel: + +- **Falsify** each claim your report, replies, CHANGELOG and docs are about to + make. Hand over the claim and the artifact that settles it: the code, the spec, + the last release tag. Never hand over your reasoning. +- **Break** each function you added or changed: "produce an input where these tests + pass and the code is wrong." Point the prompt at what has bitten SDK changes: + - a caller-supplied or replaced `HTTPClient`/transport; + - cancellation versus timeout; + - a cut or partial stream; + - an error from a 4xx (auth) versus a 5xx; + - the released behaviour at the last tag; + - the sync/async twin. +- **Read** the prose and test names you added: "which assert a fact, and is each + one true? Which prevent no wrong action a reader would otherwise take? Those are + deleted. For each that ranges over a set (every, each, never, a count), list the + set from the code and name a member it misses." +- **Trace** what the change does to code it does not touch: "list each caller of + every changed function, and each variant it runs under (overloads, the async + twin, the same helper in the sibling SDKs). Name one whose behaviour, comment, + test or doc sentence the change makes wrong." + +**Passing is reporting what came back, including nothing.** Triage its findings with +`seclai-sdk-review-triage`, the same as a reviewer's. A fix arising here runs the +gate again on that fix. + +**Never predict what the next review round will find.** Report what this gate +found on the diff: "the cold-review gate found no finding that changes behaviour", +or the list it did find. That is a claim about this diff, not about the next round. + +## Producing review findings + +For the agent writing findings, not the author triaging them. + +- **State the evidence each finding carries.** + - *Proven*: you can write a failing test today from input that occurs. + - *Contract*: a published statement is false (README, CHANGELOG, docstring, + comment). + - *Latent*: provably wrong when reached, and you can name the absent condition. + - *Speculative*: hypothesised from code shape. + - *Consequence*: a true statement about an effect elsewhere. +- **Say whether the behaviour is already in the last release.** Check the last tag. + A defect the PR did not introduce is reported as pre-existing; in a spec-sync PR + the author files it as an issue rather than fixing it there. +- **Do not report speculative hardening**, such as "a server could send X", "a + caller could reuse Y", unbounded inputs, races or timing margins. Report a latent + finding only when it would ship a break or a wrong request to consumers. +- **Every finding names a concrete failure**: the input or state, and the wrong + result. If it cannot be stated that way, it is an opinion; say so. +- **Contract means false, not incomplete.** A sentence that leaves something out is + an opinion, unless the omission makes what it says false. +- **Read the PR description and the earlier threads before reporting on code they + cover.** Do not report a finding that reverses a decision recorded there; quote + the decision and argue it, if you must. +- **Terminate on severity, not emptiness.** "No finding that changes behaviour or + the published surface" is decidable, and it is the stopping point. diff --git a/.claude/skills/seclai-sdk-sync/SKILL.md b/.claude/skills/seclai-sdk-sync/SKILL.md index 2f72dd5..ef619c2 100644 --- a/.claude/skills/seclai-sdk-sync/SKILL.md +++ b/.claude/skills/seclai-sdk-sync/SKILL.md @@ -97,8 +97,8 @@ schema, and splits the result by how much it costs you: - **SHAPE MISMATCH** (error) — the client commits to a list and the endpoint returns an object, or the reverse. This throws at deserialization; a shipped SDK is already broken. This is the 2026-07 envelope change. -- **NAME DIFFERS** (warning) — same shape, different type name. All four SDKs - drop the spec's `Model` and `Api` affixes by convention, so this tier is mostly +- **NAME DIFFERS** (warning) — same shape, different type name. The SDKs' own + type names often drop the spec's `Model` and `Api` affixes, so this tier is mostly noise by design. Keep it out of CI with `--quiet-renamed`. - **UNTYPED** (warning) — the spec names a schema and the client returns `JsonElement` / `json.RawMessage` / `unknown` / `JSONValue`. Not a defect, but @@ -153,39 +153,101 @@ diff called 21 JavaScript changes breaking when none were: - **REMOVED / BREAKING** (error) — a released signature no longer works: method gone, parameter list changed, or a return type changed in a static language. -- **WARN** — compatible but worth checking: a *trailing* optional parameter - (source compatible; flagged as binary breaking for C#), `unknown` narrowed to a - concrete type in TypeScript (safe), an optional member added to an inline - TypeScript options object such as `opts: { signal?: AbortSignal } = {}` (never - to an empty `{}`, which accepts any value the new type may reject; every - other member unchanged or only widened to accept `undefined`, the parameter's - name and its default unchanged; usually source - compatible, but a caller passing a non-literal object that already carries a - same-named property of another type — `const o = { signal: "abc" }` — no - longer compiles, since excess-property checks only cover literals), an - optional options-object member widened to accept `undefined` - (`x?: T` -> `x?: T | undefined`; source compatible), a TypeScript parameter - type wrapped in `Omit` (usually source compatible: an object literal - that sets an omitted key no longer compiles) or `Partial` (source - compatible), a TypeScript options parameter retyped to another **named** type - whose members, resolved from the declarations under `src/` at the tag and in - the working tree (interfaces, `extends`, aliases, `&`, `Omit`, `Pick`, - `Partial`), keep every old member with the same optionality and type and add - only optional ones — a member whose type names a local type counts as - unchanged only if that type's declaration is identical at both revisions, and - a type that does not resolve (generic, import, indexed access, union) stays - breaking — a required Python or TypeScript parameter made optional in place - (`id: str` -> `id: str | None = None`, `id: string` -> `id?: string`; same - name and position, positional stays positional, type unchanged or only widened - by `None`/`undefined`; not graded this way for C#), or a Python return - annotation (not enforced — but confirm the runtime value did not change too). +- **WARN** — compatible but worth checking. One line per kind: + - **Trailing optional parameter**: source compatible, but flagged as binary + breaking for C#. + - **`unknown` narrowed** to a concrete TypeScript return type, keeping its + wrapper: `unknown` to any type, `unknown[]` to `X[]` or `Array`, + `Promise` to `Promise`, `Promise` to + `Promise`. Changing the wrapper (`unknown[]` to `Promise`) breaks + callers that `.map` or `await`, so it stays BREAKING, as does an + `unknown` anywhere else (a callback parameter, a type-parameter + constraint, `Record`). + - **Options-object member added**: an optional member added to an inline + TypeScript options object such as `opts: { signal?: AbortSignal } = {}`, with + every other member, the parameter's name and its default unchanged. Usually + source compatible, but a caller passing a non-literal object that already + has a same-named property of another type (`const o = { signal: "abc" }`) no + longer compiles, since excess-property checks only cover literals. Never + graded this way for an empty `{}`, which accepts values the new type rejects. + - **Widened to `undefined`**: an optional options-object member `x?: T` -> + `x?: T | undefined`. Source compatible. A callback whose *return* type gained + `| undefined` is not this, and stays breaking. + - **`Omit`**: a TypeScript parameter type wrapped in it. Usually source + compatible; an object literal that sets an omitted key no longer compiles. + - **`Partial`**: a TypeScript parameter type wrapped in it. Source compatible. + - **Retyped to another named type**: a TypeScript options parameter whose new + type, resolved from the declarations under `src/` at the tag and in the + working tree (interfaces, `extends`, aliases, `&`, `Omit`, `Pick`, `Partial`), + keeps every old member with the same optionality and type and adds only + optional ones. A member whose type names a local type counts as unchanged + only if that declaration is identical at both revisions. A type that does + not resolve (generic, import, indexed access, union) stays breaking. + - **Required parameter made optional in place**: Python `id: str` -> + `id: str | None = None`, TypeScript `id: string` -> `id?: string`. Same name + and position, positional stays positional, type unchanged or only widened by + `None`/`undefined`. Not graded this way for C#. + - **Python return annotation changed**: not enforced, but confirm the runtime + value did not change too. + The parameter-level kinds combine: each old parameter must be unchanged or changed in one of these ways, and any added parameter must be optional and at the end. +A reformat is not a change: parameter lists are compared as their split, +whitespace-normalised members, so a trailing comma added when prettier or black +wraps a signature is invisible. Python's bare `*` is kept, since making a +parameter keyword-only does break callers. + +The return type is read up to the `{` that opens the body, at bracket depth 0, +so a lone Go `error`, a TypeScript constructor with no annotation, +`Promise<{ id: string }>`, a generic prettier wraps onto its own lines, and a +conditional type (`T extends U ? { a: 1 } : never`) are all read whole. An async +generator (`async *runStreamingAgent(`) is a method like any other, and a +generic method's type parameters (`paginate(`, wrapped or +holding a `() => void`) are recorded with its return type: changing one, or +its constraint, is BREAKING ("type parameters changed"). Until +2026-10-31 a regex read it instead, and 27 seclai-go methods returning a lone +value, plus the TypeScript constructor, were invisible: removing one reported +"no breaking change". + +A head the scan matches but cannot read — type parameters or a return type it +never closes, a parameter list that does not balance — is reported as +**UNREADABLE** and fails the run, at either revision, even when nothing changed. +Before, such a method was silently left out of the surface and its removal +reported "no breaking change"; every scanner bug found in the 2026-10 reviews +was that. Now an unfamiliar shape costs a look instead of a missed break: fix +the scanner and add a test, then rerun. A statement the pattern also matches +(`setup(x);` at class-member indent) is not a method and is not listed. + Only exported Go methods are surface. A lowercase method is unreachable outside the package, so a change to one breaks no caller. +**Go struct fields are surface too.** seclai-go's request/response types are +aliases (`type SetSlackTriggerConfigRequest = +generated.SetSlackTriggerConfigApiRequest`) of oapi-codegen structs, and +`generated` is a public package, so a field dropped by regeneration breaks +callers while every method signature is unchanged. `surface` diffs the exported +fields of exported structs in package `seclai` (read through the alias) and in +`generated/`, and the names of their other exported types, constants and +variables (an enum value the spec drops removes a generated constant): a removed +field, type, constant or variable is REMOVED; a changed field type, or a changed +declared type of a constant or variable, is BREAKING; an added one is ADDED. For +a non-struct type only the first line of its definition is compared (`string`, +an alias's target, `interface {`), so a change after the first line (a method of +a multi-line interface) is not seen. An alias to an exported struct in `seclai` +or `generated/` is compared by its fields, so re-pointing it at a struct with +the same fields is not seen either; if the old target is removed, its REMOVED +line still catches that. An alias to a struct in another package (`type File = +openapi_types.File`) is compared by its target's name only. Tags and comments +are ignored; in a nested struct only backtick tags are, the form oapi-codegen +writes. A struct it cannot read, or one embedding an unexported struct, is +UNREADABLE, as for a method head. Methods on generated types (union accessors +such as `AsValidationErrorLoc0`) are not read. It found one shipped break: +v1.8.0 renamed `generated.InsufficientCreditsDetail.Error` and +`generated.InsufficientCreditsResponse.Detail`. C# has the same blind spot for +model properties, and it is not covered. + An optional parameter inserted *mid-list* is graded breaking, not a warning: in C# a positional `M(id, token)` then tries to bind `token` to the new parameter. Use an overload rather than growing a parameter list. @@ -207,12 +269,22 @@ both revisions. Only seclai-csharp: everywhere else regeneration keeps models in step. Use `--since `, which limits the report to properties a schema **gained** since -that rev. Without it you get every long-standing coverage gap — 27 for -seclai-csharp, none of them actionable, which is how a gate gets muted. - -It cannot see models whose schema is declared inline rather than as a named -component; it prints how many are unchecked for that reason. Two of the six -models left stale in 2026-07 were in that blind spot, so treat the count as real. +that rev. Without it you get every long-standing coverage gap, which is how a +gate gets muted. + +A class is matched to its schema by name with the module prefix and the `Api` / +`Model` affixes dropped (`SlackTriggerConfigApiResponse` is +`SlackTriggerConfigResponse`), and by the +client's own methods: the class a method returns, or takes as its body, models +that operation's schema. That links `ContentFileUploadResponse` and picks which +`FileUploadResponse` a class is. Until 2026-11 an affixed class was never +matched, so `--since` did not report the Slack `alias` addition. A name that +still matches several schemas is listed as AMBIGUOUS and checked only for the +properties they all declare. + +It cannot see models with no named schema (an inline body, an SSE event, a +client-side options class); it names them as unchecked. Two of the six models +left stale in 2026-07 were in that blind spot, so read the list. ## `docexamples` — compile the README @@ -238,7 +310,7 @@ cover README snippets, and two shipped PRs contained examples that could not com | --- | --- | --- | --- | | seclai-python | generated + hand-written wrappers | yes | `make generate`, then black | | seclai-javascript | types generated, methods hand-written | yes | `npm run generate` | -| seclai-go | hand-written | yes | | +| seclai-go | types generated, methods hand-written | yes | `make generate`; `types.go` aliases the `generated/` structs | | seclai-csharp | hand-rolled from the start | **no** | no codegen library was suitable; still at near-full parity — sync it like the others | | seclai-cli | wraps `@seclai/sdk` | no | coverage question is command-to-SDK-method | | seclai-mcp | no client source | no | | @@ -271,7 +343,11 @@ For repos without a bundled spec, point at one: generator formats with ruff but the repo commits black, so raw output shows ~240 changed files that collapse to ~60 real ones. - javascript: `npm run generate` (no churn; types only). - - go / csharp: nothing to regenerate. + - go: `make generate` (`go generate ./...`, directives in `generated/gen.go`): + `cmd/specfix` converts the 3.1 spec to 3.0, then oapi-codegen v2.4.1 writes + `generated/seclai.gen.go`. To keep a field the spec removed, compiling and + marked deprecated, add it to `retainedRequestFields` in `cmd/specfix`. + - csharp: nothing to regenerate; hand-written models are what `models` checks. 4. **`parity`** to list what is missing. Implement every path, not just the ones that look interesting — binary/stream endpoints with no JSON schema are the ones that get skipped. @@ -350,7 +426,6 @@ local conventions (`searchDocs` / `search_docs` / `SearchDocs`). ## Release -The version comes from the merge commit message, read by `seclai/github-tag-action` -with `DEFAULT_BUMP: patch`. A sync that adds endpoints needs `#minor` in a commit -message (not the PR title), or it ships as a patch and the changelog heading -will not match the tag. +A sync that adds endpoints needs a `#minor` bump; where the keyword goes and how +the squash commit carries it is in the `seclai-changelog` skill, "Determine the +version". diff --git a/.claude/skills/seclai-sdk-sync/sdksync.py b/.claude/skills/seclai-sdk-sync/sdksync.py index 428f6a7..02b6529 100755 --- a/.claude/skills/seclai-sdk-sync/sdksync.py +++ b/.claude/skills/seclai-sdk-sync/sdksync.py @@ -53,8 +53,12 @@ # `[^)]*` class: a callback type or a `(default: 2000)` doc # comment would end it early, the method would drop out of the # surface and be reported REMOVED. -# sig_tail_re — matched straight after the closing `)` (the return type). -# sig_groups — head groups, then "params", then tail groups. +# The return type after the closing `)` is read by `_read_return_tail`, not a +# regex: `Promise<{ id: string }>` and Go's single, untupled `error` both +# defeated one. +# sig_groups — head groups, then "params", then "ret" when it comes after. +# type_params — a `<...>` type-parameter list may sit between the name and +# `(`; it is read and recorded with the return type. # line_comment — the line-comment marker; `public_surface` strips these, and # `/* */` too when it is `//`, from the whole file. Default `//`. # Endpoint helpers, used by `params` and `returns` (see `request_blocks`): @@ -68,7 +72,6 @@ LANGS = { "python": { "sig_re": r"\n (?:async )?def ([a-z]\w*)\(", - "sig_tail_re": r"\s*->\s*([^:]+):", "line_comment": "#", "sig_groups": ("name", "params", "ret"), "detect": "seclai/seclai.py", @@ -79,7 +82,7 @@ # calls. `api_key`, a plain property, was credited with GET /sources. "boundary_re": r"^[ \t]*(?:async )?def |^class ", "verb_re": r'"(GET|POST|PUT|PATCH|DELETE)"', - "return_re": r"\s*->\s*([^:\n]+):", + "return_after_params": True, "dict_at": [r"params\s*=\s*(?:_strip_none\()?\s*"], "dict_key_re": r'"([a-zA-Z_][a-zA-Z0-9_]*)"\s*:', "helper_re": r"^[ \t]+(?:async )?def (_[a-z0-9][a-z0-9_]*)\(", @@ -87,8 +90,14 @@ "skip": {"request", "request_raw", "stream", "paginate"}, }, "javascript": { - "sig_re": r"\n (?:async )?(\w+)\(", - "sig_tail_re": r"\s*:\s*([^{\n]+?)\s*\{", + # `async *name(` (an async generator, e.g. `runStreamingAgent`) and + # `name(` too: either one missing from the pattern left the method out + # of the surface, so removing it went unreported. The match stops before + # the `<` or `(`; `public_surface` reads a type-parameter list with + # `balanced_slice`, so parens and prettier's line wrapping inside it are + # fine, and records it with the return type (see `_split_type_params`). + "sig_re": r"\n (?:async )?\*?(\w+)(?=[<(])", + "type_params": True, "sig_groups": ("name", "params", "ret"), "detect": "src/client.ts", "sources": ["src/client.ts"], @@ -102,7 +111,7 @@ r"(?!(?:if|for|while|switch|catch|do|else|try|return|new|typeof|await)\b)" r"([a-zA-Z_][a-zA-Z0-9_]*)\s*[(<]"), "verb_re": r'"(GET|POST|PUT|PATCH|DELETE)"', - "return_re": r"\s*:\s*([^{\n]+?)\s*\{", + "return_after_params": True, "dict_at": [r"query:\s*"], "dict_key_re": r'["\']?([a-zA-Z_][a-zA-Z0-9_]*)["\']?\s*:', # Read a query literal member by member rather than with dict_key_re: @@ -125,7 +134,6 @@ # so a change to one breaks no caller. `alertConfigsBody` and # `modelAlertsBody` were graded BREAKING for that reason alone. "sig_re": r"func \(c \*Client\) ([A-Z]\w*)\(", - "sig_tail_re": r" \(([^)]*)\)", "sig_groups": ("name", "params", "ret"), "detect": "client.go", "sources": ["*.go"], @@ -135,8 +143,8 @@ # between two client methods are read as part of the earlier one. "boundary_re": r"^(?:func|type|var|const)\b", "verb_re": r"http\.Method(Get|Post|Put|Patch|Delete)", - # Go returns `(*T, error)`; take the first element of the tuple. - "return_re": r"\s*\(?\s*([^,{\n]+?)\s*(?:,\s*error)?\s*\)?\s*\{", + # Go returns `(*T, error)`; `block_return` takes the tuple's first element. + "return_after_params": True, "key_re": r'q\["([a-zA-Z_][a-zA-Z0-9_]*)"\]\s*=', # Go builds a query either by subscript assignment (key_re) or as a map # literal. Reading only the former made Search and SearchDocs — whose @@ -178,6 +186,10 @@ }, } +# The scanner each language's source needs (see `strip_comments`). +for _name, _cfg in LANGS.items(): + _cfg["lexer"] = _name + NOT_APPLICABLE = { "seclai-cli": "wraps @seclai/sdk; coverage is SDK-method-to-command, not spec-path", "seclai-mcp": "ships no client source", @@ -272,16 +284,79 @@ def _quoted_end(text: str, i: int) -> int | None: return None +def _norm(x: str | None) -> str: + """Whitespace-normalised text: every run of whitespace becomes one space.""" + return " ".join((x or "").split()) + + +def _norm_ts(x: str | None) -> str: + """`_norm` for a piece of TypeScript signature, with prettier's wrapped angle + brackets joined first: `Record<\n string,\n unknown,\n>` becomes + `Record`, as it is written on one line. + + Plain `_norm` turned the wrapped closer's line break into a space on each + side of the `>`, which `_bracket` then read as a comparison, so a later scan + of the normalised text lost its depth. It assumes a `<` ending a line and a + `>` starting one are brackets. That is wrong for a comparison broken across + lines in a parameter default (`a <\n b`): the join then miscounts depth, which + can only produce a false BREAKING, never a missed one. No SDK file has it.""" + x = re.sub(r"<[ \t]*\n\s*", "<", x or "") + x = re.sub(r",?\s*\n[ \t]*>", ">", x) + return _norm(x) + + +def _bracket(s: str, j: int) -> int: + """+1 if `s[j]` opens a bracket, -1 if it closes one, 0 otherwise. + + The one place that decides whether `<` and `>` are brackets, shared by every + depth-counting scanner here. They are, as generics (`Map>`), except: + `=>` and `->` (an arrow), `<=` and `>=`, `<-` (Go's `<-chan`, `chan<-`), a + shift `<<`, and a `<` or `>` with a space on both sides (`a > b` in a default + value). Counting those drove the depth negative or left it open, so the + commas after them stopped splitting and arity came out wrong. + + A `>` that starts its line is a closer even with a space on both sides: + prettier wraps a long generic as `Promise<\n A | B\n > {`, and reading + that `>` as a comparison left the return type open, so the method fell out + of the surface and removing it reported nothing. A comparison does not start + a line: prettier breaks a binary expression after its operator. + """ + c = s[j] + if c in "([{": + return 1 + if c in ")]}": + return -1 + if c not in "<>": + return 0 + prev = s[j - 1] if j else "" + nxt = s[j + 1] if j + 1 < len(s) else "" + if prev == " " and nxt == " " and not (c == ">" and _starts_line(s, j)): + return 0 + if c == "<": + return 0 if (nxt in "-=<" or prev == "<") else 1 + return 0 if (prev in "=-" or nxt == "=") else -1 + + +def _starts_line(s: str, j: int) -> bool: + """True when only spaces or tabs precede `s[j]` on its line.""" + k = j - 1 + while k >= 0 and s[k] in " \t": + k -= 1 + return k < 0 or s[k] == "\n" + + def balanced_slice(text: str, at: int, opener: str = "{", closer: str = "}") -> str | None: """Return the bracket-delimited literal starting at or after `at`. Regex cannot do this: query literals span lines and nest, and a non-greedy match silently truncates at the first inner `}`. Brackets inside a string - literal (`fmt?: "{id}"`, `sep = ")"`) are text, not structure. + literal (`fmt?: "{id}"`, `sep = ")"`) are text, not structure. For `<`/`>`, + which of them count is `_bracket`'s call: the `>` of `=>` does not. """ i = text.find(opener, at) if i == -1: return None + angle = opener == "<" depth, j = 0, i while j < len(text): ch = text[j] @@ -290,9 +365,9 @@ def balanced_slice(text: str, at: int, opener: str = "{", closer: str = "}") -> if end is not None: j = end continue - if ch == opener: + if ch == opener and (not angle or _bracket(text, j) == 1): depth += 1 - elif ch == closer: + elif ch == closer and (not angle or _bracket(text, j) == -1): depth -= 1 if depth == 0: return text[i + 1: j] @@ -300,22 +375,133 @@ def balanced_slice(text: str, at: int, opener: str = "{", closer: str = "}") -> return None -def strip_comments(text: str, line: str = "//") -> str: +def _triple_end(text: str, i: int) -> int | None: + """Index just past the Python triple-quoted string opening at `text[i]`.""" + q = text[i:i + 3] + j = i + 3 + while j < len(text): + if text[j] == "\\": + j += 2 + continue + if text.startswith(q, j): + return j + 3 + j += 1 + return None + + +def _template_end(text: str, i: int) -> int | None: + """Index just past the JavaScript template literal opening at `text[i]`. + It may span lines, and a `${...}` inside may hold strings and templates.""" + j, n = i + 1, len(text) + while j < n: + ch = text[j] + if ch == "\\": + j += 2 + continue + if ch == "`": + return j + 1 + if text.startswith("${", j): + depth, j = 1, j + 2 + while j < n and depth: + c = text[j] + if c == "`": + end = _template_end(text, j) + if end is None: + return None + j = end + continue + if c in "\"'": + end = _quoted_end(text, j) + j = end if end is not None else j + 1 + continue + depth += 1 if c == "{" else -1 if c == "}" else 0 + j += 1 + continue + j += 1 + return None + + +# After one of these characters or words a `/` starts a regex literal, not a division. +_REGEX_AFTER_CHARS = set("(,=:[!&|?{};+-*%<>~^") +_REGEX_AFTER_WORDS = {"return", "typeof", "case", "do", "else", "in", "of", "new", + "delete", "void", "throw", "yield", "await", "instanceof"} + + +def _regex_end(text: str, i: int, before: str) -> int | None: + """Index just past the JavaScript regex literal opening at `text[i]`, or None + when the `/` is a division. `before` is the code preceding it. + + Which one a `/` is depends on the grammar; this is the usual previous-token + heuristic. It is still a limit: a regex directly after `)` (`if (x) /re/.test(s)`) + or `]` is read as a division, and its body as code. A literal must close on + its own line, as the grammar requires. + """ + if text.startswith(("//", "/*"), i): + return None + tail = before.rstrip() + word = re.search(r"[A-Za-z_$][\w$]*$", tail) + if tail and tail[-1] not in _REGEX_AFTER_CHARS and not (word and word.group() in _REGEX_AFTER_WORDS): + return None + j, in_class = i + 1, False + while j < len(text): + c = text[j] + if c == "\\": + j += 2 + continue + if c == "\n": + return None + if in_class: + in_class = c != "]" + elif c == "[": + in_class = True + elif c == "/": + j += 1 + while j < len(text) and (text[j].isalnum() or text[j] in "_$"): + j += 1 + return j + j += 1 + return None + + +def strip_comments(text: str, line: str = "//", lexer: str | None = None, + drop_triple: bool = False) -> str: """`text` with `/* ... */` and `line`-to-end-of-line comments replaced by a space. String literals are left intact, so `"https://x"` is not a comment. Python passes `line="#"` and has no block comments; `/*` there would be an operator sequence, never a comment, so it is only honoured for `//`. + + `lexer` (a `LANGS` key) adds the literals that span lines or hold comment + markers, so a `/*` in one cannot swallow method heads up to the next `*/`: + Python triple-quoted strings (replaced by `""` with `drop_triple`, which is + how docstrings are kept out of request scanning), JavaScript template and + regex literals, Go raw strings. Without it, a literal must close on its own + line. C# verbatim strings are a known limit. """ out, j, n = [], 0, len(text) + recent = "" # the last code emitted, for the regex-literal heuristic while j < n: ch = text[j] - if ch in "\"'`": - end = _quoted_end(text, j) + end = None + if lexer == "python" and text.startswith(('"""', "'''"), j): + end = _triple_end(text, j) if end is not None: - out.append(text[j:end]) - j = end + out.append('""' if drop_triple else text[j:end]) + recent, j = '""', end continue + if ch == "`" and lexer == "javascript": + end = _template_end(text, j) + elif ch == "`" and lexer == "go": + close = text.find("`", j + 1) + end = None if close == -1 else close + 1 + elif ch in "\"'`": + end = _quoted_end(text, j) + elif ch == "/" and lexer == "javascript": + end = _regex_end(text, j, recent) + if end is not None: + out.append(text[j:end]) + recent, j = text[j:end], end + continue if line == "//" and text.startswith("/*", j): end = text.find("*/", j + 2) j = n if end == -1 else end + 2 @@ -327,6 +513,7 @@ def strip_comments(text: str, line: str = "//") -> str: out.append(" ") continue out.append(ch) + recent = (recent + ch)[-24:] j += 1 return "".join(out) @@ -336,12 +523,12 @@ def split_args(sig: str, seps: str = ",", keep_star: bool = False) -> list[str]: A naive `split(",")` turns `Dictionary q` into two entries, which over-counts arity for the query helpers and reports a phantom - signature change for `surface`. The `>` of a TypeScript arrow `=>` is not a - closing bracket: counting it would split `cb: (a: A, b: B) => void` after - the arrow. `seps` lets an object-literal type body split on `;` too. - Separators and brackets inside string literals are text. + signature change for `surface`. Which `<`/`>` are brackets is `_bracket`'s + call: the `>` of `cb: (a: A, b: B) => void`, Go's `<-chan` and the `>=` of a + default value are not. `seps` lets an object-literal type body split on `;` + too. Separators and brackets inside string literals are text. """ - out, depth, cur, prev = [], 0, [], "" + out, depth, cur = [], 0, [] j, n = 0, len(sig) while j < n: ch = sig[j] @@ -351,11 +538,7 @@ def split_args(sig: str, seps: str = ",", keep_star: bool = False) -> list[str]: cur.append(sig[j:end]) j = end continue - if ch in "<([{": - depth += 1 - elif ch in ")]}" or (ch == ">" and prev != "="): - depth -= 1 - prev = ch + depth += _bracket(sig, j) if ch in seps and depth == 0: out.append("".join(cur).strip()) cur = [] @@ -434,15 +617,22 @@ def block_call(body: str, verb_re: str) -> tuple[str, str] | None: return v.group(1).upper(), path -def strip_docs(text: str) -> str: - """Drop docstrings, block comments and whole-line comments. - - Helper detection reads through this: a docstring's `"GET"` and `"/sources"` - example made seclai-python's `_raise_for_openapi_response` look like the - request behind twenty methods that never issue one. +def code_only(text: str, cfg: dict) -> str: + """`text` without comments or Python docstrings, for request scanning. + + Every scan strips comments with `strip_comments` and the language's lexer: + `surface` and the TypeScript declaration readers (`ts_declarations`, + `_ts_object_members`, `_ts_typed_members`) and the JavaScript query-literal + reader `object_literal_keys` do too, so a multi-line template + literal holding `/*` is read the same way everywhere. A + docstring's `"GET"` and `"/sources"` example made seclai-python's + `_raise_for_openapi_response` look like the request behind twenty methods + that never issue one, and a trailing `// (default: 2000)` held a `)` that + ended a parameter list early. Triple-quoted strings are dropped (as `""`) + rather than kept, which is right for docstrings and harmless elsewhere: + no request path or query key is written in one. """ - text = re.sub(r'"""[\s\S]*?"""|\'\'\'[\s\S]*?\'\'\'|/\*[\s\S]*?\*/', "", text) - return re.sub(r"(?m)^[ \t]*(?://|#).*$", "", text) + return strip_comments(text, cfg.get("line_comment", "//"), cfg.get("lexer"), drop_triple=True) def endpoint_helpers(text: str, cfg: dict) -> dict[str, str]: @@ -454,16 +644,16 @@ def endpoint_helpers(text: str, cfg: dict) -> dict[str, str]: """ if "helper_re" not in cfg: return {} - ends = sorted({m.start() - for pat in (cfg["helper_re"], cfg["method_re"], cfg.get("boundary_re")) if pat - for m in re.finditer(pat, text, re.M)}) + # dict.fromkeys: the JavaScript boundary_re IS helper_re; scan it once. + patterns = dict.fromkeys(p for p in (cfg["helper_re"], cfg["method_re"], cfg.get("boundary_re")) if p) + ends = sorted({m.start() for pat in patterns for m in re.finditer(pat, text, re.M)}) out: dict[str, str] = {} for m in re.finditer(cfg["helper_re"], text, re.M): name = m.group(1) if name in cfg.get("skip", ()) or name in out: continue end = next((e for e in ends if e > m.start()), len(text)) - body = strip_docs(text[m.start():end]) + body = code_only(text[m.start():end], cfg) if block_call(body, cfg["verb_re"]): out[name] = body return out @@ -471,7 +661,7 @@ def endpoint_helpers(text: str, cfg: dict) -> dict[str, str]: def helper_called(body: str, cfg: dict, helpers: dict[str, str]) -> str | None: """Name of the first endpoint helper a method body calls, if any.""" - code = strip_docs(body) # a comment naming a helper is not a call to it + code = code_only(body, cfg) # a comment naming a helper is not a call to it first: tuple[int, str] | None = None for name in helpers: m = re.search(cfg["helper_call"].replace("{name}", re.escape(name)), code) @@ -487,7 +677,10 @@ def request_blocks(texts: list[str], cfg: dict): defining one is read through that helper — ONE hop. `body` stays the public method's, so its declared return type is what `returns` compares; `request_body` is where query keys are read. `helper` is None for own requests. + Every text is read without comments or docstrings (see `code_only`), so a + commented-out request is not a request and a `)` in a comment is not code. """ + texts = [code_only(t, cfg) for t in texts] shared: dict[str, str] = {} if cfg.get("helper_scope") == "package": for text in texts: @@ -523,7 +716,7 @@ def object_literal_keys(lit: str) -> tuple[set[str], bool]: """ keys: set[str] = set() ok = True - for member in split_args(strip_comments(lit)): + for member in split_args(strip_comments(lit, "//", "javascript")): # JS only (`dict_members`) m = re.match(r"""(?:"([^"]+)"|'([^']+)'|([A-Za-z_$][\w$]*))\s*:""", member) if m: keys.add(next(g for g in m.groups() if g)) @@ -571,8 +764,9 @@ def resolve(name: str, stack: tuple[str, ...]) -> tuple[set[str], bool]: def _calls(body: str, name: str) -> bool: - """A call to the free function `name(` — not `x.name(`, not `othername(`.""" - return re.search(rf"(? str: def block_return(body: str, cfg: dict) -> str | None: - """The declared return type of the method a block starts with. + """The declared return type of the method a block starts with; None when it + declares none or cannot be read (`read_block_return` tells them apart).""" + return read_block_return(body, cfg)[0] + + +def read_block_return(body: str, cfg: dict) -> tuple[str | None, bool]: + """(declared return type, unreadable) of the method a block starts with. + The block is known to be a method that makes a request, so anything short of + reading its return — or finding it has no annotation — is unreadable, and + `returns` must not count it as "nothing to check". Read by slicing the parameter list with paren matching rather than by regex: signatures wrap across lines and contain nested parens and generics, both of @@ -1075,22 +1278,23 @@ def block_return(body: str, cfg: dict) -> str | None: """ if "return_head_re" in cfg: m = re.match(cfg["return_head_re"], body, re.S) - return m.group(1).strip() if m else None + return (m.group(1).strip(), False) if m else (None, True) # Start at the paren the method anchor itself ends on. Taking the first `(` # in the block instead picks up Go's receiver — `func (c *Client) GetMe(` — # so the "return type" came back as the rest of the signature. anchor = re.match(cfg["method_re"], body, re.M) if not anchor: - return None - open_paren = body.find("(", anchor.end() - 1) - if open_paren == -1: - return None - params = balanced_slice(body, open_paren, "(", ")") - if params is None: - return None - rest = body[open_paren + len(params) + 2:] - m = re.match(cfg["return_re"], rest, re.S) - return m.group(1).strip() if m else None + return None, True + sliced = _after_params(body, body.find("(", anchor.end() - 1)) + if sliced is None: + return None, True + ret, _ = _read_return_tail(body, sliced[1], cfg["lexer"]) + if ret is None: + return None, True + if cfg["lexer"] == "go": + parts = split_args(ret) # `*T, error` -> `*T` + return (parts[0] if parts else ""), False + return (ret or None), False # no annotation declares nothing to check def cmd_returns(args) -> int: @@ -1102,7 +1306,7 @@ def cmd_returns(args) -> int: if not lang: die(f"cannot detect SDK language in {repo}") cfg = LANGS[lang] - if "return_re" not in cfg and "return_head_re" not in cfg: + if "return_after_params" not in cfg and "return_head_re" not in cfg: print(f"{repo.name} [{lang}] — return types are not declared; nothing to check.") return 0 files = source_files(repo, cfg) @@ -1124,12 +1328,15 @@ def cmd_returns(args) -> int: _schema_label(_json_schema(op, "response", code))) break - mismatched, renamed, untyped, n, via_helper = [], [], [], 0, 0 + mismatched, renamed, untyped, unreadable, n, via_helper = [], [], [], [], 0, 0 texts = [path.read_text(encoding="utf-8", errors="replace") for path in files] for mname, body, call, _request_body, helper in request_blocks(texts, cfg): if call not in want: continue - declared = block_return(body, cfg) + declared, unread = read_block_return(body, cfg) + if unread: + unreadable.append((mname, call)) + continue if declared is None: continue n += 1 @@ -1178,39 +1385,170 @@ def cmd_returns(args) -> int: for name, (verb, p), expect, declared in untyped: print(f" {name} ({verb} {p}): {declared}, spec declares {expect}") + if unreadable: + print(f"\nUNREADABLE ({len(unreadable)}) — request methods whose return type this " + "scan could not read; a shape mismatch there would go unreported:") + for name, (verb, p) in unreadable: + print(f" {name} ({verb} {p})") if mismatched: print(f"\n{len(mismatched)} return type(s) disagree with the spec in SHAPE.") return 1 + if unreadable: + print(f"\n{len(unreadable)} unreadable return type(s) — the result above is incomplete.") + return 1 print("\nno return type disagrees with the spec in shape") return 0 # ── surface ────────────────────────────────────────────────────────────────── +def _after_params(text: str, open_paren: int) -> tuple[str, int] | None: + """(parameter list, index just past its closing `)`) for the `(` at + `open_paren`, sliced by paren matching. Shared by `surface` and `returns`, so + the offset past the `)` is written once.""" + if open_paren < 0: + return None + params = balanced_slice(text, open_paren, "(", ")") + if params is None: + return None + return params, open_paren + len(params) + 2 + + +def _read_return_tail(text: str, i: int, lexer: str) -> tuple[str | None, bool]: + """(return type, unreadable). The type is None with unreadable False when + what follows is a statement, not a method; with unreadable True when the + scan never reached the body — the case `surface` must not stay quiet about. + + A scan, not a regex. The type ends at the `{` that opens the body, at depth + 0 — so `Promise<{ id: string }>` (its `{` is inside `<>`), an object type + literal right after `:` or a `|`, and Go's `interface{}` are read whole. The + regexes this replaced stopped at the first `{` and recorded `Promise<`, so a + change inside that type was invisible, and Go's matched only a `(...)` tuple, + so every method returning a lone `error` was missing from the surface. + """ + if lexer == "csharp": + return "", False # read from the head; see sig_groups + if lexer == "python": + m = re.compile(r"\s*(?:->\s*([^:]+?))?\s*:").match(text, i) + # Every match of Python's pattern is a `def`, so a miss is a misread. + return (_norm(m.group(1)), False) if m else (None, True) + j, n = i, len(text) + if lexer == "javascript": + m = re.compile(r"\s*:").match(text, j) + if m is None: + m = re.compile(r"\s*\{").match(text, j) + return ("", False) if m else (None, False) # `foo(x);` is a call + j = m.end() + start, depth = j, 0 + while j < n: + c = text[j] + if c in "\"'`": + end = _quoted_end(text, j) + if end is not None: + j = end + continue + if c in ";=" and depth == 0 and not text.startswith("=>", j): + return None, False # a statement or a field initialiser, not a method + if c == "{" and depth == 0: + so_far = text[start:j].rstrip() + # A TypeScript `{` opens a type literal, not the body, wherever a type + # can start: after `|`, `&`, `=>`, `:`, `,`, and in a conditional type + # after `?` or `extends` (`T extends { a: 1 } ? { b: 2 } : never`). + literal = (lexer == "go" and re.search(r"\b(?:interface|struct)$", so_far)) or ( + lexer == "javascript" and ( + not so_far or so_far.endswith(("|", "&", "=>", ":", ",", "?")) + or re.search(r"\bextends$", so_far))) + if not literal: + ret = (_norm_ts if lexer == "javascript" else _norm)(text[start:j]) + # A Go result tuple is recorded without its parens, `*Run, error`. + if lexer == "go" and ret.startswith("(") and balanced_slice(ret, 0, "(", ")") == ret[1:-1]: + ret = _norm(ret[1:-1]) + return ret, False + body = balanced_slice(text, j) + if body is None: + return None, True + j += len(body) + 2 + continue + depth += _bracket(text, j) + j += 1 + return None, True # ran off the end: the body was never found + + def public_surface(text: str, cfg: dict) -> dict[str, set[tuple[str, str]]]: """method name -> {(return type, parameter list)}. A set, because C# and Go - both allow overloads/variants under one name.""" - out: dict[str, set[tuple[str, str]]] = {} - order = cfg["sig_groups"] + both allow overloads/variants under one name. + + The parameter list is stored as its split, normalised members joined by + ", ", so a reformat — prettier or black wrapping a signature and adding a + trailing comma — is not a change. Python's bare `*` is kept: making a + parameter keyword-only is one. + """ + return scan_surface(text, cfg)[0] - def norm(x: str | None) -> str: - return " ".join((x or "").split()) - tail_re = re.compile(cfg.get("sig_tail_re", ""), re.S) +def scan_surface(text: str, cfg: dict) -> tuple[dict[str, set[tuple[str, str]]], list[str]]: + """(`public_surface`, `unreadable_heads`) from one pass over `text`.""" + out: dict[str, set[tuple[str, str]]] = {} + unreadable: set[str] = set() # Comments are not signature, and stripped before slicing so a `)` or an # apostrophe in one cannot end the parameter list early. - text = strip_comments(text, cfg.get("line_comment", "//")) + text = strip_comments(text, cfg.get("line_comment", "//"), cfg["lexer"]) for m in re.finditer(cfg["sig_re"], text, re.S): - open_paren = m.end() - 1 - params = balanced_slice(text, open_paren, "(", ")") - if params is None: - continue - tail = tail_re.match(text, open_paren + len(params) + 2) - if tail is None: - continue - parts = dict(zip(order, m.groups() + (params,) + tail.groups())) - out.setdefault(norm(parts["name"]), set()).add((norm(parts["ret"]), norm(parts["params"]))) - return out + status, name, entry = _read_head(text, m, cfg) + if status == "ok": + out.setdefault(name, set()).add(entry) + elif status == "unreadable": + unreadable.add(name) + return out, sorted(unreadable) + + +def unreadable_heads(text: str, cfg: dict) -> list[str]: + """Names of public method heads `public_surface` matched but could not read. + + Such a method is missing from the surface at that revision, so its removal + or change would report nothing. `surface` lists these and fails, which turns + any shape the scanner does not understand into a visible error rather than a + silent miss. A statement the pattern also matches (`foo(x);`) is not listed. + """ + return scan_surface(text, cfg)[1] + + +def _read_head(text: str, m: re.Match, cfg: dict) -> tuple[str, str, tuple[str, str] | None]: + """("ok", name, (return type, parameters)), ("skip", name, None) for a match + that is not a method, or ("unreadable", name, None).""" + order = cfg["sig_groups"] + name = _norm(dict(zip(order, m.groups()))["name"]) + # Most patterns end on the `(`; JavaScript's stops before it, so a + # type-parameter list can come first. + at = m.end() - 1 if text[m.end() - 1] == "(" else m.end() + tparams = None + if cfg.get("type_params") and text.startswith("<", at): + tp = balanced_slice(text, at, "<", ">") + if tp is None: + return "unreadable", name, None + # Split and rejoined like the parameters, so prettier's trailing comma + # on a wrapped list is not a change. + tparams = ", ".join(_norm_ts(x) for x in split_args(tp)) + at += len(tp) + 2 + while at < len(text) and text[at] in " \t\n": + at += 1 + if not text.startswith("(", at): + return ("unreadable" if tparams is not None else "skip"), name, None + sliced = _after_params(text, at) + if sliced is None: + return "unreadable", name, None + params, after = sliced + ret, unreadable = _read_return_tail(text, after, cfg["lexer"]) + if ret is None: + return ("unreadable" if unreadable else "skip"), name, None + if tparams is not None: + ret = _join_type_params(tparams, ret) + parts = dict(zip(order, m.groups() + (params,))) + if "ret" not in parts: + parts["ret"] = ret + norm = _norm_ts if cfg["lexer"] == "javascript" else _norm + canon = ", ".join(norm(p) for p in split_args(params, keep_star=True)) + return "ok", _norm(parts["name"]), (_norm(parts["ret"]), canon) @@ -1218,6 +1556,15 @@ def norm(x: str | None) -> str: _params = split_args +# A TypeScript parameter's head, `name?: `, and an object member's name and `?`. +_TS_PARAM_HEAD_RE = re.compile(r"\s*([\w$]+\s*\??\s*:\s*)") +_TS_MEMBER_NAME = r"(?:readonly\s+)?([\w$]+|\"[^\"]*\"|'[^']*')\s*(\?)?\s*" +# A property member, `name?: type`, with the type captured. +_TS_MEMBER_RE = re.compile(_TS_MEMBER_NAME + r":\s*(.*)$", re.S) +# Any named member: a property, or a method `name(...)`. +_TS_NAMED_MEMBER_RE = re.compile(_TS_MEMBER_NAME + r"[:(]") + + def _ts_inline_object(param: str) -> tuple[str, str, str] | None: """(prefix, body, suffix) when a TypeScript parameter's type is an inline object literal: `opts: { a?: string } = {}` -> ("opts:", "a?: string", "= {}"). @@ -1225,44 +1572,68 @@ def _ts_inline_object(param: str) -> tuple[str, str, str] | None: The `{` must follow the annotation colon directly, so a default value such as `x = {}` or a generic `Partial<{...}>` never qualifies. None otherwise. """ - m = re.match(r"\s*[\w$]+\s*\??\s*:\s*\{", param) - if not m: + m = _TS_PARAM_HEAD_RE.match(param) + if not m or not param.startswith("{", m.end()): return None - start = m.end() - 1 + start = m.end() body = balanced_slice(param, start) if body is None: return None end = start + len(body) + 1 - return (" ".join(param[:start].split()), body, " ".join(param[end + 1:].split())) + return (_norm(param[:start]), body, _norm(param[end + 1:])) def _ts_object_members(body: str) -> dict[str, tuple[bool, str]] | None: """member name -> (optional, normalised text) for an object-literal type body, split on depth-0 `;` or `,`. None when any member cannot be named (index signatures, call signatures, spread-like text) or a name repeats — the caller - then refuses to call the change compatible. Comments are dropped first: + then refuses to call the change compatible. Comments are dropped here even + though `public_surface` already stripped them: `classify_change` is also + called directly with raw signature text, and the bodies are a few lines, so + the second pass costs nothing worth a precondition. Comments are dropped first: seclai-javascript documents each member with `/** ... */`, which would otherwise hide the member's name, and a `,` or `;` inside one would split it. """ - parts = split_args(strip_comments(body), ";,") + parts = split_args(strip_comments(body, "//", "javascript"), ";,") out: dict[str, tuple[bool, str]] = {} for raw in parts: - text = " ".join(raw.split()) + text = _norm(raw) if not text: continue - m = re.match(r"(?:readonly\s+)?([\w$]+|\"[^\"]*\"|'[^']*')\s*(\?)?\s*[:(]", text) + m = _TS_NAMED_MEMBER_RE.match(text) if not m or m.group(1) in out: return None out[m.group(1)] = (bool(m.group(2)), text) return out -_TS_MEMBER_RE = re.compile(r"(?:readonly\s+)?([\w$]+|\"[^\"]*\"|'[^']*')\s*(\?)?\s*:\s*(.*)$", re.S) + +def _arrow_at_top(t: str) -> bool: + """`t` is a function type: it has an `=>` at bracket depth 0.""" + depth, j = 0, 0 + while j < len(t): + if t[j] in "\"'`": + end = _quoted_end(t, j) + if end is not None: + j = end + continue + if depth == 0 and t.startswith("=>", j): + return True + depth += _bracket(t, j) + j += 1 + return False def _ts_union(t: str) -> list[str]: - """Top-level `|` members of a type, whitespace-normalised and sorted.""" - return sorted(" ".join(x.split()) for x in split_args(t, "|")) + """Top-level `|` members of a type, whitespace-normalised and sorted. + + A function type is one member: in `(e: E) => T | undefined` the `|` belongs + to the return type, so splitting it there called a callback whose return + type changed "optional property widened to accept undefined". + """ + if _arrow_at_top(t): + return [_norm(t)] + return sorted(_norm(x) for x in split_args(t, "|")) def _ts_widened_to_undefined(old: str, new: str) -> bool: @@ -1314,9 +1685,8 @@ def _ts_wrapped_type(o: str, n: str) -> str | None: optionality and the default textually unchanged. Every old argument still fits both. None otherwise — including `Pick<>` and an `Omit<>` of another type.""" - ho = re.match(r"\s*([\w$]+\s*\??\s*:\s*)", o) - hn = re.match(r"\s*([\w$]+\s*\??\s*:\s*)", n) - if not ho or not hn or " ".join(ho.group(1).split()) != " ".join(hn.group(1).split()): + ho, hn = _TS_PARAM_HEAD_RE.match(o), _TS_PARAM_HEAD_RE.match(n) + if not ho or not hn or _norm(ho.group(1)) != _norm(hn.group(1)): return None rest_o, rest_n = o[ho.end():], n[hn.end():] w = re.match(r"(Omit|Partial)\s*<", rest_n) @@ -1336,8 +1706,7 @@ def _ts_wrapped_type(o: str, n: str) -> str | None: return None elif len(args) != 1: return None - norm = lambda x: " ".join(x.split()) # noqa: E731 - return kind if norm(rest_o) == norm(args[0] + after) else None + return kind if _norm(rest_o) == _norm(args[0] + after) else None _TS_COMPATIBLE_LABELS = { @@ -1398,10 +1767,9 @@ def _top_level_index(s: str, ch: str) -> int: if end is not None: j, prev = end, "" continue - if c in "<([{": - depth += 1 - elif c in ")]}" or (c == ">" and prev != "="): - depth -= 1 + b = _bracket(s, j) + if b: + depth += b elif c == ch and depth == 0: nxt = s[j + 1] if j + 1 < len(s) else "" if ch != "=" or (nxt not in "=>" and prev not in "=!<>"): @@ -1414,8 +1782,8 @@ def _top_level_index(s: str, ch: str) -> int: def _union_members(t: str) -> set[str]: m = re.fullmatch(r"\s*Optional\[(.*)\]\s*", t, re.S) if m: - return {" ".join(x.split()) for x in split_args(m.group(1), "|")} | {"None"} - return {" ".join(x.split()) for x in split_args(t, "|")} + return {_norm(x) for x in split_args(m.group(1), "|")} | {"None"} + return {_norm(x) for x in split_args(t, "|")} def _py_param(p: str) -> tuple[str, str, str | None] | None: @@ -1425,17 +1793,17 @@ def _py_param(p: str) -> tuple[str, str, str | None] | None: m = re.fullmatch(r"\s*([A-Za-z_]\w*)\s*(?::\s*(.+?))?\s*", head, re.S) if not m: return None - return m.group(1), " ".join((m.group(2) or "").split()), default + return m.group(1), _norm(m.group(2)), default def _ts_param(p: str) -> tuple[str, bool, str, str | None] | None: """(name, optional `?`, type, default) of one TypeScript parameter, or None.""" eq = _top_level_index(p, "=") - head, default = (p, None) if eq < 0 else (p[:eq], " ".join(p[eq + 1:].split())) + head, default = (p, None) if eq < 0 else (p[:eq], _norm(p[eq + 1:])) m = re.fullmatch(r"\s*([A-Za-z_$][\w$]*)\s*(\?)?\s*(?::\s*(.+?))?\s*", head, re.S) if not m: return None - return m.group(1), bool(m.group(2)), " ".join((m.group(3) or "").split()), default + return m.group(1), bool(m.group(2)), _norm(m.group(3)), default def _gained_default(lang: str, o: str, n: str) -> bool: @@ -1543,23 +1911,21 @@ def _gained_default_change(lang: str, old_par: str, new_par: str, def _ts_expr_end(text: str, i: int) -> int: """End of a type-alias expression starting at `i`: the first `;` at depth 0, or a blank line / new top-level declaration when the `;` is omitted.""" - depth, j, prev = 0, i, "" + depth, j = 0, i while j < len(text): c = text[j] if c in "\"'`": end = _quoted_end(text, j) if end is not None: - j, prev = end, "" + j = end continue - if c in "<([{": - depth += 1 - elif c in ")]}" or (c == ">" and prev != "="): - depth -= 1 + b = _bracket(text, j) + if b: + depth += b elif depth == 0 and (c == ";" or (c == "\n" and re.match( r"\n\s*(?:\n|(?:export\s+)?(?:interface|type|const|function|class|import)\b)", text[j:]))): return j - prev = c j += 1 return len(text) @@ -1576,7 +1942,7 @@ def put(name: str, decl: tuple | None) -> None: out[name] = decl for raw in texts: - text = strip_comments(raw) + text = strip_comments(raw, "//", "javascript") for m in re.finditer(r"(?m)^[ \t]*(?:export\s+)?(?:declare\s+)?interface\s+([\w$]+)\s*(<)?", text): brace = text.find("{", m.end()) @@ -1589,7 +1955,7 @@ def put(name: str, decl: tuple | None) -> None: if ext and not bases: put(m.group(1), None) continue - whole = " ".join(text[m.start():brace + len(body) + 2].split()) + whole = _norm(text[m.start():brace + len(body) + 2]) put(m.group(1), ("interface", bases, body, whole)) for m in re.finditer(r"(?m)^[ \t]*(?:export\s+)?(?:declare\s+)?type\s+([\w$]+)\s*(<)?", text): if m.group(2): @@ -1600,8 +1966,8 @@ def put(name: str, decl: tuple | None) -> None: continue start = m.end() + eq.end() end = _ts_expr_end(text, start) - expr = " ".join(text[start:end].split()) - put(m.group(1), ("alias", expr, " ".join(text[m.start():end].split()))) + expr = _norm(text[start:end]) + put(m.group(1), ("alias", expr, _norm(text[m.start():end]))) return out @@ -1609,14 +1975,14 @@ def _ts_typed_members(body: str) -> dict[str, tuple[bool, str]] | None: """member -> (optional, type) for an object type body; None if any member is not a plain `name?: type` property (methods, index signatures, ...).""" out: dict[str, tuple[bool, str]] = {} - for raw in split_args(strip_comments(body), ";,"): - text = " ".join(raw.split()) + for raw in split_args(strip_comments(body, "//", "javascript"), ";,"): + text = _norm(raw) if not text: continue m = _TS_MEMBER_RE.match(text) if not m or m.group(1) in out: return None - out[m.group(1).strip("\"'")] = (bool(m.group(2)), " ".join(m.group(3).split())) + out[m.group(1).strip("\"'")] = (bool(m.group(2)), _norm(m.group(3))) return out @@ -1630,7 +1996,7 @@ def _ts_string_keys(k: str) -> set[str] | None: def ts_resolve(expr: str, decls: dict, seen: tuple[str, ...] = () ) -> dict[str, tuple[bool, str]] | None: """Members of a TypeScript object type expression, or None if unresolvable.""" - e = " ".join(expr.split()) + e = _norm(expr) while e.startswith("(") and balanced_slice(e, 0, "(", ")") == e[1:-1]: e = e[1:-1].strip() if not e: @@ -1727,6 +2093,82 @@ def ts_named_options_change(o: str, n: str, old_decls: dict, new_decls: dict) -> return kinds or {"same"} +def _join_type_params(tparams: str, ret: str) -> str: + """A generic method's recorded return: `: Promise`. + + Kept with the return type rather than the parameters so that the optional- + parameter rules, which compare parameters by position, are unaffected. No + TypeScript type starts with `<...>:` (a generic function type is `(`), so + `_split_type_params` can always take it apart again.""" + return f"<{tparams}>: {ret}".rstrip() + + +def _unwrap(t: str, wrapper: str) -> str | None: + """`X` when `t` is exactly `wrapper` (`Promise`, `Array`), else None.""" + if not t.startswith(wrapper + "<"): + return None + inner = balanced_slice(t, len(wrapper), "<", ">") + if inner is None or len(wrapper) + len(inner) + 2 != len(t): + return None + return inner.strip() + + +def _array_element(t: str) -> str | None: + """`X` when `t` is an array of `X`: `X[]` for a name or a parenthesised + type, or `Array`. A union like `A | B[]` is not an array, so the element + must be a single name (with type arguments) or wrapped in parens.""" + inner = _unwrap(t, "Array") + if inner is not None: + return inner + if not t.endswith("[]"): + return None + el = t[:-2].strip() + if el.startswith("(") and balanced_slice(el, 0, "(", ")") == el[1:-1]: + return el[1:-1].strip() + m = re.fullmatch(r"[\w$.]+", el) or re.fullmatch(r"[\w$.]+(<.*>)", el) + if m and (m.lastindex is None or _unwrap(el, el[:el.index("<")]) is not None): + return el + return None + + +def _narrowed_from_unknown(old: str, new: str) -> bool: + """True when `new` only replaces the `unknown` in `old` with a concrete type, + keeping everything around it — the one return change a caller cannot + notice except as better types. + + `unknown` is the value the caller reads only when it is the whole return, + its array, or either inside `Promise<>`. The wrapper must survive: callers of + `unknown[]` call `.map`, and callers of `Promise` `await` or + `.then`, so `unknown[]` -> `Promise` breaks them. An `unknown` + anywhere else (a callback parameter, a constraint, `Record`) + stays BREAKING. + """ + if "unknown" in new: + return False + def concrete(x: str | None) -> bool: + return bool(x) and "unknown" not in x + if old == "unknown": + return True + if old == "unknown[]": + return concrete(_array_element(new)) + if old == "Promise": + return concrete(_unwrap(new, "Promise")) + if old == "Promise": + inner = _unwrap(new, "Promise") + return inner is not None and concrete(_array_element(inner)) + return False + + +def _split_type_params(ret: str) -> tuple[str | None, str]: + """(type parameters, return type) of a recorded return; None when the method + is not generic.""" + if ret.startswith("<"): + inner = balanced_slice(ret, 0, "<", ">") + if inner is not None and ret.startswith(":", len(inner) + 2): + return inner, ret[len(inner) + 3:].strip() + return None, ret + + def classify_change(lang: str, old: tuple[str, str], new: tuple[str, str], ts_decls: tuple[dict, dict] | None = None) -> tuple[str, str]: """(severity, label) for one signature change. severity is 'break' or 'warn'. @@ -1740,6 +2182,13 @@ def classify_change(lang: str, old: tuple[str, str], new: tuple[str, str], """ old_ret, old_par = old new_ret, new_par = new + if lang == "javascript": + # A type parameter or its constraint changing cannot be proven safe from + # text: `` -> `` can reject every old call. + old_tp, old_ret = _split_type_params(old_ret) + new_tp, new_ret = _split_type_params(new_ret) + if old_tp != new_tp: + return "break", "type parameters changed" if old_par != new_par: op, np_ = split_args(old_par), split_args(new_par) if lang == "javascript" and old_ret == new_ret: @@ -1776,7 +2225,13 @@ def ts_param_change(o: str, n: str) -> set[str] | None: return "warn", "optional parameters added — source compatible" return "break", "parameter list changed" if old_ret != new_ret: - if lang == "javascript" and "unknown" in old_ret and "unknown" not in new_ret: + # Only where `unknown` is the value the caller reads: the whole return, + # its array, or either inside `Promise<>`. Anywhere else it may be a + # parameter of a callback (`(x: unknown) => void`) or a constraint + # (``), where narrowing rejects calls that worked, so + # it stays BREAKING; a covariant case like `() => unknown` is a false + # alarm this accepts. + if lang == "javascript" and _narrowed_from_unknown(old_ret, new_ret): return "warn", "return narrowed from unknown — safe for consumers" if lang == "python": return "warn", "return annotation changed — not enforced at runtime, but check the value did not change" @@ -1803,6 +2258,201 @@ def _texts_at(repo: Path, rev: str | None, sub: str, suffix: str) -> list[str]: return out +def _go_package_texts(repo: Path, rev: str | None, sub: str) -> list[str]: + """The non-test `.go` files directly in `sub` (one Go package), at `rev` or + in the working tree. Listed at the rev itself, so a file deleted since the + tag still contributes its types to the baseline.""" + if rev is None: + d = repo / sub + return [f.read_text(encoding="utf-8", errors="replace") + for f in sorted(d.glob("*.go")) if not f.name.endswith("_test.go")] + prefix = "" if sub in ("", ".") else sub.rstrip("/") + "/" + ls = subprocess.run(["git", "-C", str(repo), "ls-tree", "--name-only", rev, "--", prefix or "."], + capture_output=True, text=True) + out = [] + for name in ls.stdout.split(): + if name.endswith(".go") and not name.endswith("_test.go") and "/" not in name[len(prefix):]: + p = subprocess.run(["git", "-C", str(repo), "show", f"{rev}:{name}"], + capture_output=True, text=True) + if p.returncode == 0: + out.append(p.stdout) + return out + + +_GO_TYPE_RE = re.compile(r"^type[ \t]+([A-Za-z_]\w*)(?:\[[^\]\n]*\])?[ \t]+", re.M) +_GO_FIELD_RE = re.compile(r"^([A-Za-z_]\w*(?:\s*,\s*[A-Za-z_]\w*)*)\s+(\S.*)$", re.S) +_GO_EMBED_RE = re.compile(r"^\*?([A-Za-z_]\w*\.)?([A-Za-z_]\w*)(?:\[.*\])?$", re.S) + + +def _go_struct_members(body: str) -> list[str] | None: + """A struct body (or `const`/`var` block) split into its declarations at + depth 0: a nested `struct { ... }` or `map[K]V` stays one member. A line + break splits only where Go would end the statement (after an identifier, + literal, `)`, `]` or `}`), so `"a" +` continues onto the next line. None + when it does not balance.""" + out, cur, depth, j = [], [], 0, 0 + while j < len(body): + ch = body[j] + if ch in "\"'`": + end = (body.find("`", j + 1) + 1 or None) if ch == "`" else _quoted_end(body, j) + if end is None: + return None + cur.append(body[j:end]) + j = end + continue + if ch in "([{": + depth += 1 + elif ch in ")]}": + depth -= 1 + if depth < 0: + return None + if depth == 0 and ch in "\n;": + member = "".join(cur) + tail = member.rstrip() + if (ch == ";" or not tail or tail[-1].isalnum() or tail[-1] in "_\"'`)]}" + or tail.endswith(("++", "--")) or re.search(r"(? tuple[dict[str, dict[str, str] | str], list[str]]: + """(declarations, unreadable) for one Go package's source. + + A declaration maps an EXPORTED name to {exported field: type} for a struct, + to the alias target string (`generated.X`) for `type A = B`, to "type T" + for any other type (`type Scope string` -> "type string"), and to + "const T" / "var T" for a package constant or variable, T its declared type + if any. A struct tag is dropped, and a nested struct's backtick tags too: + retagging changes the wire, not what compiles. An embedded field is named by its type, + as Go names it. A struct whose body does not balance, holds a member read as + neither a field nor an embedding, or embeds an unexported struct (whose + promoted fields this scan cannot list) is unreadable. + """ + text = strip_comments(text, "//", "go") + decls: dict[str, dict[str, str] | str] = {} + # A grouped `type ( ... )` block is not read; say so rather than drop it. + unreadable: list[str] = ["type ( ... ) block" + for _ in re.finditer(r"^type[ \t]*\(", text, re.M)] + for m in re.finditer(r"^(const|var)\b[ \t]*(\()?", text, re.M): + kind = m.group(1) + if m.group(2): + body = balanced_slice(text, m.end() - 1, "(", ")") + members = _go_struct_members(body) if body is not None else None + if members is None: + unreadable.append(f"{kind} ( ... ) block") + continue + else: + members = [text[m.end():].split("\n", 1)[0]] + last = "" + for mem in members: + left, eq, _ = mem.partition("=") + head = re.match(r"\s*([\w\s,]*?\w)(?:\s+([^,\s].*?))?\s*$", left) + if head is None: # no name before the type or `=` + unreadable.append(f"{kind} ( ... ) block" if m.group(2) else f"{kind} {left.strip()}") + break + names, typ = head.groups() + if kind == "const" and not eq and not typ: + typ = last # Go repeats the previous spec's type and value + last = typ or "" + for n in re.split(r"\s*,\s*", names): + if n[0].isupper(): + decls[n] = _norm(f"{kind} {typ or ''}") + for m in _GO_TYPE_RE.finditer(text): + name, rest = m.group(1), text[m.end():] + if not name[0].isupper(): + continue + if rest.startswith("="): + decls[name] = _norm(rest[1:].split("\n", 1)[0]) + continue + if not re.match(r"struct\s*\{", rest): + # `type Scope string`, interfaces, func types: the first line of the + # underlying type, so `string` -> `int` is a change. + decls[name] = _norm("type " + rest.split("\n", 1)[0]) + continue + body = balanced_slice(text, m.end()) + members = _go_struct_members(body) if body is not None else None + if members is None: + unreadable.append(f"type {name} struct") + continue + fields: dict[str, str] = {} + ok = True + for mem in members: + mem = re.sub(r"\s*`[^`]*`\s*$|\s*\"(?:[^\"\\]|\\.)*\"\s*$", "", mem) + e = _GO_EMBED_RE.match(mem) + f = _GO_FIELD_RE.match(mem) + if e and not f: + if not e.group(1) and e.group(2)[0].islower(): + ok = False + break + fields[e.group(2)] = "embedded " + _norm(mem) + elif f: + for fname in re.split(r"\s*,\s*", f.group(1)): + fields[fname] = _norm(re.sub(r"\s*`[^`]*`", "", f.group(2))) + else: + ok = False + break + if not ok: + unreadable.append(f"type {name} struct") + continue + decls[name] = {k: v for k, v in fields.items() if k[0].isupper()} + return decls, unreadable + + +def _go_kind(entry: str) -> str: + """What a `go_field_surface` entry declares: "type", "const", "var" or "field". + A field's type text cannot start with these keywords; "struct" alone is a type.""" + word = entry.split(" ", 1)[0] + return "type" if entry == "struct" or word == "type" else word if word in ("const", "var") else "field" + + +def go_field_surface(seclai_texts: list[str], generated_texts: list[str] + ) -> tuple[dict[str, str], list[str]]: + """(surface, unreadable) for package seclai (as `X`) and `generated/` (as + `generated.X`): `Type` -> "struct" and `Type.Field` -> field type for every + exported struct, `Name` -> its `go_struct_fields` value for every other + exported declaration, and "type = target" for any other alias. An alias to + an exported struct in either package (`type Req = generated.ReqApi`) is + listed with the fields it resolves to, since that is what its callers set. + """ + pk: dict[str, dict] = {} + unreadable: list[str] = [] + for prefix, texts in (("", seclai_texts), ("generated.", generated_texts)): + decls = {} + for t in texts: + d, u = go_struct_fields(t) + decls.update(d) + unreadable += [f"{n} in package {prefix.rstrip('.') or 'seclai'}" for n in u] + pk[prefix] = decls + + def resolve(prefix: str, name: str, seen: tuple = ()) -> dict | None: + d = pk[prefix].get(name) + if isinstance(d, dict) or d is None or name in seen: + return d if isinstance(d, dict) else None + if d.startswith("generated."): + return resolve("generated.", d[len("generated."):], seen + (name,)) + if "." in d or not re.fullmatch(r"\w+", d): + return None # another package, or not a plain name: not ours to read + return resolve(prefix, d, seen + (name,)) + + out: dict[str, str] = {} + for prefix, decls in pk.items(): + for name, d in decls.items(): + fields = resolve(prefix, name) + if fields is None: + out[prefix + name] = d if _go_kind(d) != "field" else f"type = {d}" + continue + out[prefix + name] = "struct" + for f, t in fields.items(): + out[f"{prefix}{name}.{f}"] = t + return out, unreadable + + def cmd_surface(args) -> int: """Diff the public API surface against a released tag. @@ -1847,7 +2497,8 @@ def cmd_surface(args) -> int: old_text = "\n".join(old_parts) new_text = "\n".join(f.read_text(encoding="utf-8", errors="replace") for f in files) - old, new = public_surface(old_text, cfg), public_surface(new_text, cfg) + (old, old_unread), (new, new_unread) = scan_surface(old_text, cfg), scan_surface(new_text, cfg) + unreadable = [(args.rev, n) for n in old_unread] + [("now", n) for n in new_unread] ts_decls = None if lang == "javascript": ts_decls = (ts_declarations(_texts_at(repo, args.rev, "src", ".ts")), @@ -1856,8 +2507,34 @@ def cmd_surface(args) -> int: added = sorted(set(new) - set(old)) changed = [k for k in sorted(set(old) & set(new)) if old[k] - new[k]] - print(f"{repo.name} [{lang}] — {len(old)} public method(s) at {args.rev}, {len(new)} now") - + # Go: exported struct fields too. The request/response types are aliases of + # oapi-codegen structs in the public `generated` package, so removing or + # retyping a field breaks callers while every method signature is unchanged. + old_f, new_f = {}, {} + if lang == "go": + old_f, old_fu = go_field_surface(_go_package_texts(repo, args.rev, "."), + _go_package_texts(repo, args.rev, "generated")) + new_f, new_fu = go_field_surface(_go_package_texts(repo, None, "."), + _go_package_texts(repo, None, "generated")) + unreadable += [(args.rev, n) for n in old_fu] + [("now", n) for n in new_fu] + gone_types = {k for k in old_f if old_f[k] == "struct" and k not in new_f} + for k in sorted(set(old_f) - set(new_f)): + if k in gone_types or k.rsplit(".", 1)[0] not in gone_types: + removed.append(f"{k} ({_go_kind(old_f[k])})") + added += [f"{k} ({_go_kind(new_f[k])})" for k in sorted(set(new_f) - set(old_f)) + if _go_kind(new_f[k]) != "field" or k.rsplit(".", 1)[0] in old_f] + + fields_note = (f"; {sum(v == 'struct' for v in old_f.values())} exported struct type(s) " + f"at {args.rev}, {sum(v == 'struct' for v in new_f.values())} now") if lang == "go" else "" + print(f"{repo.name} [{lang}] — {len(old)} public method(s) at {args.rev}, {len(new)} now{fields_note}") + + if unreadable: + print(f"\nUNREADABLE ({len(unreadable)}) — public method heads, or Go declarations, " + "this scan could not read; they are missing from the surface, so a change " + "to them would go " + "unreported. Fix the scanner (and add a test) before trusting this result:") + for rev, n in unreadable: + print(f" {n} (at {rev})") if removed: print(f"\nREMOVED ({len(removed)}) — every caller breaks:") for k in removed: @@ -1870,14 +2547,19 @@ def cmd_surface(args) -> int: if len(gone) == 1 and len(now) == 1: sev, label = classify_change(lang, gone[0], now[0], ts_decls) (breaks if sev == "break" else warns).append((k, label, gone, now)) + for k in sorted(set(old_f) & set(new_f)): + if old_f[k] != new_f[k]: + what = ("field type changed" if _go_kind(old_f[k]) == _go_kind(new_f[k]) == "field" + else "declaration changed") + breaks.append((k, what, [(old_f[k], None)], [(new_f[k], None)])) def show(rows: list) -> None: for k, label, gone, now in rows: print(f" {k} — {label}") for ret, params in gone: - print(f" was: {ret} ({params[:100]})") + print(f" was: {ret}" + ("" if params is None else f" ({params[:100]})")) for ret, params in now: - print(f" now: {ret} ({params[:100]})") + print(f" now: {ret}" + ("" if params is None else f" ({params[:100]})")) if breaks: print(f"\nBREAKING ({len(breaks)}) — a released signature no longer works:") @@ -1893,6 +2575,9 @@ def show(rows: list) -> None: print(f"\n{len(removed) + len(breaks)} breaking change(s). Keep the old form and add " "alongside it, or bump the major.") return 1 + if unreadable: + print(f"\n{len(unreadable)} unreadable declaration(s) — the result above is incomplete.") + return 1 print("\nno breaking change — every released signature still works") return 0 @@ -1940,6 +2625,21 @@ def git_location(spec: Path, fallback: Path) -> tuple[Path, str]: die(f"{full} resolves outside its own work tree {root}; --since cannot read it") +def affix_free(name: str) -> str: + """A schema name with its module qualification and `Api`/`Model` affixes + dropped, as seclai-csharp names its hand-written models: + `routers__api__agents__UpdateAgentRequest` -> `UpdateAgentRequest`, + `SlackTriggerConfigApiResponse` -> `SlackTriggerConfigResponse`, + `CloudDriveResponseModel` -> `CloudDriveResponse`. Only a trailing `Model` + and an `Api` before the last word go, so `EmbeddingModelListResponse`, + `ApiVersionResponse` and `UpdateApiVersionRequest` keep theirs. `models` + matches classes by it. + """ + name = short_label(name) + name = re.sub(r"(?<=[a-z0-9])Model$", "", name) + return re.sub(r"(?<=[a-z0-9])Api(?=[A-Z][a-z0-9]*$)", "", name) + + def spec_schema_index(spec: dict) -> dict[str, list[tuple[str, set[str]]]]: """Short schema name -> [(full name, property names)]. @@ -1949,11 +2649,91 @@ def spec_schema_index(spec: dict) -> dict[str, list[tuple[str, set[str]]]]: """ index: dict[str, list[tuple[str, set[str]]]] = {} for full, sch in (spec.get("components", {}).get("schemas") or {}).items(): - short = full.rsplit("__", 1)[-1] - index.setdefault(short, []).append((full, set(sch.get("properties") or {}))) + index.setdefault(short_label(full), []).append((full, set(sch.get("properties") or {}))) return index +def _element(label: str) -> tuple[bool, str]: + m = re.match(r"^array<(.+)>$", label) + return (True, m.group(1)) if m else (False, label) + + +def operation_models(repo: Path, lang: str, spec: dict, classes: set[str]) -> dict[str, dict[str, set[str]]]: + """Model class -> {full schema name -> client methods} from the client itself. + + A method that returns class X for an operation whose response is schema S, + or takes X as a parameter where the operation's request body is S, says X + models S. That is the client's own declaration, so it settles which of two + same-named schemas (`routers__api__sources__FileUploadResponse` and + `routers__api__contents__FileUploadResponse`) a class is, and links a class + named nothing like its schema (`ContentFileUploadResponse`). A list on one + side and an object on the other links nothing: that is a `returns` finding. + """ + cfg = LANGS.get(lang) + if not cfg or "return_head_re" not in cfg: + return {} + ops = {} + for raw, item in spec.get("paths", {}).items(): + for verb, op in item.items(): + if verb in ("get", "post", "put", "patch", "delete"): + ops[(verb.upper(), normalise(raw))] = op + links: dict[str, dict[str, set[str]]] = {} + + def link(cls: str, label: str, mname: str) -> None: + if cls in classes and "<" not in label and label not in ("object", "none", "unknown"): + links.setdefault(cls, {}).setdefault(label, set()).add(mname) + + texts = [f.read_text(encoding="utf-8", errors="replace") for f in source_files(repo, cfg)] + for mname, body, call, _request_body, _helper in request_blocks(texts, cfg): + op = ops.get(call) + if op is None: + continue + declared, unread = read_block_return(body, cfg) + if declared and not unread: + got_arr, got = _element(normalise_type(declared, lang)) + for code in ("200", "201", "202"): + if code in op.get("responses", {}): + want_arr, want = _element(_schema_label(_json_schema(op, "response", code))) + if got_arr == want_arr: + link(got, want, mname) + break + req = _schema_label(_json_schema(op, "request")) + anchor = re.search(cfg["method_re"], body) + sliced = _after_params(body, body.find("(", anchor.end() - 1)) if anchor else None + if sliced and req not in ("none", "object"): + types = {a.split()[0] for a in split_args(sliced[0]) if a.split()} + for t in types & classes: + link(t, req, mname) + return links + + +def match_models(classes: set[str], spec: dict, + links: dict[str, dict[str, set[str]]]) -> dict[str, list[str]]: + """Model class -> the full schema names it may model ([] when none). + + By name first: the exact short name, else the `affix_free` spelling, + ignoring case (`HttpValidationError` is `HTTPValidationError`). A + class the client links to operations (`operation_models`) is narrowed to + the named candidates it is used for; when it is used for none of them, its + links replace them. More than one result is ambiguous and is reported so; + picking one would check a model against a schema it may not be. + """ + fulls = list((spec.get("components", {}).get("schemas") or {})) + exact: dict[str, list[str]] = {} + loose: dict[str, list[str]] = {} + for full in fulls: + exact.setdefault(short_label(full), []).append(full) + loose.setdefault(affix_free(full).lower(), []).append(full) + out = {} + for cls in classes: + named = exact.get(cls) or loose.get(affix_free(cls).lower()) or [] + used = set(links.get(cls, {})) + if used: + named = sorted(used & set(named)) or sorted(used) + out[cls] = named + return out + + def cmd_models(args) -> int: repo = Path(args.repo).resolve() if repo.name in NOT_APPLICABLE: @@ -1970,7 +2750,8 @@ def cmd_models(args) -> int: return 0 spec = load_spec(args.rev, args.spec, repo) - index = spec_schema_index(spec) + props = {full: set(sch.get("properties") or {}) + for full, sch in (spec.get("components", {}).get("schemas") or {}).items()} # `--since` narrows the report to properties the schema GAINED since a rev. # Without it the C# models show 27 findings, every one a coverage gap that @@ -1980,63 +2761,77 @@ def cmd_models(args) -> int: gained: dict[str, set[str]] | None = None if args.since: spec_repo, rel = git_location(Path(args.spec), repo) - before = spec_schema_index(load_spec(args.since, rel, spec_repo)) + old = load_spec(args.since, rel, spec_repo) + was_full = {f: set(s.get("properties") or {}) + for f, s in (old.get("components", {}).get("schemas") or {}).items()} + was_short = spec_schema_index(old) gained = {} - for short, cands in index.items(): - now = set.union(*(p for _, p in cands)) - was = set.union(*(p for _, p in before[short])) if short in before else set() + for full, now in props.items(): + # A schema that only gained or lost its module qualification since + # the rev is the same schema: compare it with its old short name. + was = was_full.get(full) + if was is None: + was = set().union(*(p for _, p in was_short.get(short_label(full), []))) if now - was: - gained[short] = now - was + gained[full] = now - was files = sorted(repo.glob(cfg["glob"])) scope = f", {len(gained)} schema(s) changed since {args.since}" if gained is not None else "" - print(f"{repo.name} [{lang}] — {len(files)} model file(s), {len(index)} spec schema names{scope}") + print(f"{repo.name} [{lang}] — {len(files)} model file(s), {len(props)} spec schemas{scope}") - missing, extra, unmatched = [], [], [] + models: dict[str, set[str]] = {} for f in files: text = f.read_text(encoding="utf-8", errors="replace") m = re.search(cfg["class_re"], text) - if not m: - continue - cls = m.group(1) - have = set(re.findall(cfg["prop_re"], text)) - cands = index.get(cls) + if m: + models[m.group(1)] = set(re.findall(cfg["prop_re"], text)) + matches = match_models(set(models), spec, operation_models(repo, lang, spec, set(models))) + + missing, extra, unmatched, ambiguous = [], [], [], [] + for cls in sorted(models): + have, cands = models[cls], matches[cls] if not cands: unmatched.append(cls) continue + if len(cands) > 1: + ambiguous.append((cls, cands)) # With more than one candidate, only properties common to all of them are # unambiguously missing. A gate that guesses is a gate that gets muted. - want = set.intersection(*(p for _, p in cands)) + want = set.intersection(*(props[c] for c in cands)) if gained is not None: - want &= gained.get(cls, set()) + want &= set().union(*(gained.get(c, set()) for c in cands)) gone = want - have if gone: - missing.append((cls, sorted(gone), [n for n, _ in cands])) + missing.append((cls, sorted(gone), cands)) if len(cands) == 1 and gained is None: - surplus = have - cands[0][1] + surplus = have - props[cands[0]] if surplus: extra.append((cls, sorted(surplus))) if missing: print(f"\nMISSING PROPERTIES ({len(missing)}) — the spec declares these and the model does not:") - for cls, props, full in missing: + for cls, gone, full in missing: where = f" [{', '.join(full)}]" if full != [cls] else "" - print(f" {cls}{where}\n {', '.join(props)}") + print(f" {cls}{where}\n {', '.join(gone)}") if extra: print(f"\nWARN — EXTRA PROPERTIES ({len(extra)}) — in the model, not in the schema:") - for cls, props in extra: - print(f" {cls}: {', '.join(props)}") - if unmatched and not args.quiet_unmatched and gained is None: - print(f"\nWARN — NOT IN SPEC ({len(unmatched)}) — model classes with no matching schema:") - print(" " + ", ".join(sorted(unmatched))) + for cls, surplus in extra: + print(f" {cls}: {', '.join(surplus)}") + if ambiguous: + print(f"\nWARN — AMBIGUOUS ({len(ambiguous)}) — the class matches several schemas and " + "nothing narrows it to one; only properties all of them declare are checked:") + for cls, cands in ambiguous: + print(f" {cls}: {', '.join(cands)}") if unmatched: # Not a finding: the spec declares some request bodies inline rather than # as a named component, so CreateSourceRequest and UpdateSourceRequest — # both of which WERE left stale in 2026-07 — have nothing to match - # against. Say so, so the count is never read as full coverage. + # against. Say so, and name them, so the count is never read as full + # coverage. print(f"\n note: {len(unmatched)} model class(es) have no named schema " - f"and are unchecked (inline request bodies land here).") + f"and are unchecked (inline request bodies land here)" + + ("." if args.quiet_unmatched else ":\n " + ", ".join(unmatched))) if missing: print(f"\n{len(missing)} model(s) behind the spec.") @@ -2095,7 +2890,7 @@ def main() -> int: help="only report properties the schema gained since this spec rev " "(the sync gate; without it every long-standing gap is listed)") p.add_argument("--quiet-unmatched", action="store_true", - help="suppress the model-classes-with-no-schema report") + help="count the model classes with no schema without naming them") p.set_defaults(func=cmd_models) p = sub.add_parser("spec-diff", help="paths/schemas added, removed or changed between revisions") diff --git a/CHANGELOG.md b/CHANGELOG.md index 4476bc1..62c2828 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,27 @@ # Changelog +## [1.9.0] - 2026-11-05 + +### Changed + +- Sync the bundled OpenAPI spec and regenerate the types. Existing responses gain properties: `result_reported_at` on `AgentRunResponse`, set once the run's answer is in `output`, and `alias` on `SlackTriggerConfigResponse` +- Move `SeclaiApiVersion.Latest` to `2026-11-04`. Nothing is sent unless `apiVersion` is set, so a client that passes `SeclaiApiVersion.Latest` now opts into `2026-11-04`. From `2026-11-02`, `updateAgentDefinition()`, and `updateAgent()` when it sets `sampling_config`, refuse a definition or sampling rule that references a metadata key no trigger declares (with a 400 and a 422 respectively), where older versions declare the key as optional; from `2026-11-04`, `createEvaluationResult()` rejects `retry_triggered: true`, `updateAgent()` rejects `evaluation_mode` `eval_and_retry`, and `createEvaluationCriteria()` and `updateEvaluationCriteria()` reject a `step_id` that does not name a `display_result` or `streaming_result` step, each with a 400 +- Note API changes that apply on every API version, whatever `apiVersion` is set: a run started with `runAgent()`, `runStreamingAgent()` or the helpers built on them is refused with a 400 when its `metadata` sends a custom key the trigger does not declare, or lacks a required one, so declare custom keys with `setMetadataKeys()` before sending them; `updateAgentDefinition()` rejects a definition holding a `gate` or `retry` step (use `if_else`, or `repeat_until` to run steps again until a condition holds); the `agent_eval_fail` rollback trigger is accepted and never fires, and the default rollback triggers no longer include it; `max_retries` and `retry_on_failure` on `updateAgent()` are accepted and have no effect; and `evaluation_mode` on `AgentSummaryResponse` defaults to `sample_and_flag` instead of `eval_and_retry` +- Check the `runAgent()` response in `runAgentAndPoll()` before polling, so a run that ended at once is returned without waiting one `pollIntervalMs`. After an abort through `signal` during the start request, that run is now returned instead of throwing `Polling aborted.`, and with `includeStepOutputs` it comes back without step outputs. When `timeoutMs` elapses during the start request, that run is also returned, where `SeclaiStreamingError` was thrown +- Mark `input` on `SetSlackTriggerConfigRequest` deprecated. The API ignores it: a Slack run's input is the text after `/seclai `, or the message the shortcut was used on. It stays on the type so existing calls still compile +- Mark `max_retries` and `retry_on_failure` on `UpdateAgentRequest`, and `retry_count` and `retry_triggered` on `CreateEvaluationResultRequest`, deprecated. They have no effect, and from `2026-11-04` `retry_triggered: true` returns a 400. They stay on the types so existing calls still compile + +### Added + +- Add `getMetadataKeys()` and `setMetadataKeys()` to read and replace the run metadata keys an agent trigger declares, which steps reference as `{{metadata.}}`. A key's `required` defaults to `false` and can be omitted +- Add `getKnowledgeBaseMetadataKeyStats()` to see which item-metadata keys a knowledge base's content carries, with item counts and top values for all time and the last 30 and 7 days +- Accept `alias` on `setSlackTriggerConfig()`, the name people type after `/seclai` to run the agent +- Add the `returnOnResult` option to `runAgentAndPoll()` and `runStreamingAgentAndWait()`, to return as soon as a run has `result_reported_at` set (from `runAgent()` or a poll), or a `result` event arrives, without waiting for later steps. The run may still be in progress and can still fail +- Add the `throwOnIncomplete` option to `runStreamingAgentAndWait()`, to throw `SeclaiStreamingError`, with the latest snapshot in `run`, when the stream ends or breaks before `done`, or the server answers with plain JSON for a run that has not finished, instead of returning a run that has not finished or throwing the fetch error. It will become the default in the next major version +- Add `run` to `SeclaiStreamingError`: the latest snapshot of the unfinished run when `runStreamingAgentAndWait()` or `runAgentAndPoll()` gives up, so a reported answer in `output` is not lost +- Add the `SeclaiApiVersion.V2026_11_02` and `SeclaiApiVersion.V2026_11_04` constants, so those versions are accepted as `apiVersion` +- Add the type exports `MetadataOrigin`, `MetadataKeyDeclaration`, `MetadataKeyDeclarationInput`, `SetMetadataKeysRequest`, `DeclarableMetadataKey`, `MetadataKeysResponse`, `KnowledgeBaseMetadataKeyStatsResponse`, `MetadataKeyWindowResponse`, `MetadataKeyCountResponse` and `MetadataValueCountResponse` + ## [1.8.0] - 2026-10-31 ### Changed @@ -270,6 +292,7 @@ _Stable release. No functional changes since 0.0.1._ _Initial release._ +[1.9.0]: https://github.com/seclai/seclai-javascript/releases/tag/1.9.0 [1.8.0]: https://github.com/seclai/seclai-javascript/releases/tag/1.8.0 [1.7.0]: https://github.com/seclai/seclai-javascript/releases/tag/1.7.0 [1.6.1]: https://github.com/seclai/seclai-javascript/releases/tag/1.6.1 diff --git a/CLAUDE.md b/CLAUDE.md index 4d144a0..af13c02 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,5 +1,13 @@ # seclai-javascript +## Changing and reviewing code + +The rules for writing a change (the cold-review gate on your own diff before you +report) and for producing review findings are shared by every SDK repo. They are +vendored from `sdk-tools` with the review-triage skill: + +@.claude/skills/seclai-sdk-review-triage/rules.md + ## Git and PRs - **Never commit or push.** No `git commit`, `git push`, `gh pr create`, no merging, not after finishing a task, not after addressing review comments, not after checks pass. The user commits all work themselves. Leave the changes in the working tree and say the work is ready. Do not offer to commit either. diff --git a/README.md b/README.md index 66ed44f..6230a25 100644 --- a/README.md +++ b/README.md @@ -127,7 +127,7 @@ Leave `apiVersion` unset and the header is omitted, so the account's pinned baseline applies and responses keep their current shapes. Upgrading this package alone never changes the wire contract. -Known versions are on `SeclaiApiVersion` (`V2026_07_01` through `V2026_10_28`, +Known versions are on `SeclaiApiVersion` (`V2026_07_01` through `V2026_11_04`, plus `Default` and `Latest`), imported from `@seclai/sdk`. A version this release was **not** built against throws at construction: a newer version can reshape responses, and this client would decode them incorrectly rather than reject them. @@ -190,6 +190,9 @@ a request that used to succeed, or (from `2026-10-21`) which fields can be returns `null` `score` and `passed` for one. - `2026-10-28` changes what an `if_else` or `switch` step's `output` contains. Read the arm it took from the step's `decision`, which is set on every version. +- `2026-11-02` refuses an agent write that references a metadata key no trigger + declares, where older versions declare the key for you. +- `2026-11-04` rejects with a 400 evaluation settings that were accepted before. | Version | What it changes | | --- | --- | @@ -205,6 +208,8 @@ a request that used to succeed, or (from `2026-10-21`) which fields can be | `2026-10-19` | `createSource()` rejects `polling_max_items` on any source type but an RSS feed with a 400. Below it the value is stored | | `2026-10-21` | An evaluation result whose evaluator returned no verdict (`error`) is left out of `failed` and of `pass_rate` / `failure_rate` in `getNonManualEvaluationSummary()`, and `getEvaluationCriteriaSummary()` averages `average_score` over passed and failed results only. `testDraftEvaluation()` returns `score` and `passed` as `null` for an `error` result. Below it the summary counts `error` results in `failed` and the rates, `average_score` includes their stored scores, and the test draft returns `score` 0.5 and `passed: false`. The summary's `error` count is returned on every version | | `2026-10-28` | An `if_else` or `switch` step's `output` is the result of the arm it took, or its input when that arm is empty or it took none. Below it the `output` is a `{"branch": ...}` marker naming the arm, with no `attachments`. The step's `decision` names the arm on every version | +| `2026-11-02` | `updateAgentDefinition()`, and `updateAgent()` when it sets `sampling_config`, refuse a definition or sampling rule that references a metadata key no trigger declares, with a 400 and a 422 respectively. Declare the key first with `setMetadataKeys()`. An `agent_definition` import through `createAgent()` or `updateAgent()` is not refused and still declares the keys it reads. Below it each such key is declared on the trigger as optional | +| `2026-11-04` | `createEvaluationResult()` rejects `retry_triggered: true` and `updateAgent()` rejects `evaluation_mode` `eval_and_retry` with a 400: evaluations never retry a step. `createEvaluationCriteria()`, and `updateEvaluationCriteria()` when it changes `step_id`, reject a `step_id` that does not name a `display_result` or `streaming_result` step with a 400. Below it `eval_and_retry` is stored as `sample_and_flag` | **`Latest` moves with the SDK.** `SeclaiApiVersion.Latest` is the newest version the installed release knows, so upgrading the package can opt a client that @@ -212,8 +217,10 @@ passes it into every version added since — `1.6.0` moved it from `2026-07-27` `2026-10-03`, across the `2026-09-30` output change, and `1.7.0` moves it to `2026-10-21`, across the `2026-10-05` status change, the `2026-10-13` and `2026-10-19` source-setting rejections, the `2026-10-16` input change and the -`2026-10-21` evaluation change, and `1.8.0` moves it to `2026-10-28`, -across the `if_else` / `switch` step-output change. Pass a +`2026-10-21` evaluation change, `1.8.0` moves it to `2026-10-28`, +across the `if_else` / `switch` step-output change, and `1.9.0` moves it to +`2026-11-04`, across the `2026-11-02` undeclared-metadata-key refusal and the +`2026-11-04` evaluation-setting rejections. Pass a dated constant such as `SeclaiApiVersion.V2026_07_27` to keep behaviour fixed across upgrades. @@ -354,13 +361,25 @@ await client.deleteAgentRun("run_id"); The SDK provides two streaming patterns over the SSE `/runs/stream` endpoint: -**Block until done** — returns the final `done` payload or throws on timeout: +**Block until done** — returns the run from the `done` event. If `timeoutMs` +elapses first, it throws `SeclaiStreamingError`, whose `run` holds the latest +snapshot: the `result` snapshot if one arrived, else the `init` snapshot. An +abort through `signal` rethrows the abort error. +By default, a stream that ends before `done` returns the latest snapshot when +its status is not `pending`, so the run returned may not have finished: check +`status` and `result_reported_at`. A stream that breaks throws the fetch error. +Pass `throwOnIncomplete: true` to throw `SeclaiStreamingError`, with the snapshot +in `run`, in both cases instead, and also when the server answers with plain +JSON for a run that has not finished. It is recommended, and becomes the default in +the next major version: + + ```ts const result = await client.runStreamingAgentAndWait( "agent_id", { input: "Hello", metadata: {} }, - { timeoutMs: 60_000 }, + { timeoutMs: 60_000, throwOnIncomplete: true }, ); ``` @@ -377,6 +396,40 @@ for await (const event of client.runStreamingAgent( } ``` +When the server answers with plain JSON instead of an SSE stream, +`runStreamingAgent` yields only a `done` event, so read the answer from it. + +A `result` event carries the run's answer in `output` once `result_reported_at` +is set. It comes before `done`, and for an agent that reports its result early it +can arrive while later steps still run. Pass `returnOnResult: true` to +`runStreamingAgentAndWait` to return that `result` snapshot without waiting for +`done`. Such a run may still be in progress: its `status` is not terminal and the +run can still fail, so check `result_reported_at`, not `status`: + + +```ts +const run = await client.runStreamingAgentAndWait("agent_id", { input: "Hello" }, { returnOnResult: true }); +if (run.result_reported_at) console.log(run.output); +``` + +On a timeout, or with `throwOnIncomplete` on a cut stream, read the answer, if +one was reported, from the error's `run`: + +```ts +import { SeclaiStreamingError } from "@seclai/sdk"; + +try { + const run = await client.runStreamingAgentAndWait("agent_id", { input: "Hello" }, { throwOnIncomplete: true }); + console.log(run.output); +} catch (err) { + if (err instanceof SeclaiStreamingError && err.run?.result_reported_at) { + console.log("Answer, run not finished:", err.run.output); + } else { + throw err; + } +} +``` + Pass `include_reasoning: true` to also receive `stream_reasoning` events carrying the model's reasoning while it thinks. They are sent only for a `streaming_result` step whose author enabled `show_reasoning`, and reasoning is @@ -404,6 +457,23 @@ const result = await client.runAgentAndPoll( ); ``` +A run's answer can be ready before its last step finishes: once +`result_reported_at` is set on `getAgentRun()`, `output` holds the answer. Pass +`returnOnResult: true` to return as soon as a run has it set, the one +`runAgent()` returns included. With `includeStepOutputs`, when that run would end +the poll and `signal` is not aborted, it is fetched again at once, because the +`runAgent()` response carries no step outputs; after an abort it comes back +without step outputs. Such a run may still be in progress: its `status` +is not terminal and the run can still fail, so check `result_reported_at`, not +`status`. If `timeoutMs` elapses first, it throws `SeclaiStreamingError` with the +latest run it has (the `runAgent()` response if no poll ran) in `run`. + + +```ts +const run = await client.runAgentAndPoll("agent_id", { input: "Hello" }, { returnOnResult: true }); +if (run.result_reported_at) console.log(run.output); +``` + ### Agent input uploads @@ -465,11 +535,15 @@ are listed in the run's `email_attachment_issues` metadata. By default the run f ### Agent Slack triggers Configure an agent's `SLACK_COMMAND` trigger, which people run from Slack with -`/seclai` or the message shortcut. Omitted fields are left unchanged. +`/seclai ` or the message shortcut. Omitted fields are left unchanged. +`alias` is generated from the agent's name when not set; the text after the alias, +or the message the shortcut was used on, is the run's input. `input` is +deprecated and ignored by the API. ```ts const slack = await client.setSlackTriggerConfig("agent_id", "trigger_id", { + alias: "support", // people run `/seclai support` scope: "selected_channels", // or "bot_channels" channel_ids: ["C0123456789"], activation: "everywhere", // or "channel_opt_in" @@ -505,6 +579,33 @@ const { accepted_attachments } = await client.setAcceptedAttachments("agent_id", console.log(accepted_attachments.map((slot) => `${slot.name}: ${slot.min}..${slot.max ?? "any"}`)); ``` +### Trigger metadata keys + +Declare the run metadata keys a trigger provides, which steps reference as +`{{metadata.}}`. `system` and `source` keys are written by the platform; +`custom` keys are sent by the caller or set on the trigger, and only a `custom` +key can be `required` (default `false`): a caller-started run without it is +refused. A run is also refused with a 400 when its `metadata` sends a custom key +the trigger does not declare, on every API version, so declare custom keys with +`setMetadataKeys()` before sending them in a run's `metadata`. The call replaces +the whole declaration, and is refused with a 400 when the agent references a key +no trigger of its branch declares. + + +```ts +const agent = await client.getAgent("agent_id"); +const current = await client.getMetadataKeys("agent_id", agent.trigger_id!); +console.log(current.declarable_keys.map((k) => `${k.key} (${k.origin})`)); + +const updated = await client.setMetadataKeys("agent_id", agent.trigger_id!, { + metadata_keys: [ + { key: "customer_id", origin: "custom", required: true }, + { key: "region", origin: "custom" }, + ], +}); +console.log(updated.warnings); // agents whose `call_agent` mapping no longer fits +``` + ### Agent email governance ```ts @@ -585,6 +686,17 @@ await client.updateKnowledgeBase("kb_id", { name: "Renamed KB" }); await client.deleteKnowledgeBase("kb_id"); ``` +See which item-metadata keys a knowledge base's content carries, for all time +and the last 30 and 7 days: + + +```ts +const stats = await client.getKnowledgeBaseMetadataKeyStats("kb_id"); +for (const w of stats.windows) { + console.log(w.window, w.total_items, w.keys.map((k) => `${k.key}: ${k.items}`)); +} +``` + ### Memory banks ```ts diff --git a/openapi/seclai.openapi.json b/openapi/seclai.openapi.json index 4220dcc..bee860b 100644 --- a/openapi/seclai.openapi.json +++ b/openapi/seclai.openapi.json @@ -277,7 +277,7 @@ }, "definition": { "additionalProperties": true, - "description": "The agent definition containing name, description, tags, and step workflow tree. Step types include prompt_call, retrieval, regex_replace, gate, retry, evaluate_step, extract_data, extract_content, add_chat_turn, load_chat_history, add_memory, search_memory, load_memory, streaming_result, send_email, webhook_call, call_agent, write_metadata, write_content_attachment, load_content_attachment, load_content, display_result, merge, for_each, if_else, switch, and others.", + "description": "The agent definition containing name, description, tags, and step workflow tree. Step types include prompt_call, retrieval, regex_replace, evaluate_step, extract_data, extract_content, add_chat_turn, load_chat_history, add_memory, search_memory, load_memory, streaming_result, send_email, webhook_call, call_agent, write_metadata, write_content_attachment, load_content_attachment, load_content, display_result, merge, for_each, if_else, switch, and others.", "title": "Definition", "type": "object" }, @@ -492,7 +492,7 @@ "title": "Evaluation Criteria" }, "export_version": { - "description": "Schema version of the export format (currently \"5\").", + "description": "Schema version of the export format (currently \"6\").", "title": "Export Version", "type": "string" }, @@ -720,7 +720,7 @@ "type": "null" } ], - "description": "Metadata to make available for string substitution expressions in agent tasks.", + "description": "Metadata for string substitution in agent tasks. Each key must be one the agent's trigger declares (or sets); a run that sends another custom key, or lacks a required one, is refused with HTTP 400.", "title": "Metadata" }, "priority": { @@ -806,7 +806,7 @@ "type": "null" } ], - "description": "Result of the governance input evaluation: safe, blocked, skipped, or timed_out.", + "description": "Result of the governance input evaluation: pending, safe, blocked, skipped, timed_out, or error.", "title": "Governance Input Status" }, "governance_input_wait_ms": { @@ -830,7 +830,7 @@ "type": "null" } ], - "description": "Cumulative milliseconds the run was parked waiting for a human decision on a human_in_the_loop step. Subtracted from active duration in run-detail and duration-stats responses.", + "description": "Sum over the run's human_in_the_loop steps of each one's wait for a decision, in milliseconds. Waits that overlap are each counted in full, so the sum can exceed the run's duration.", "title": "Hitl Wait Ms" }, "input": { @@ -886,6 +886,19 @@ "title": "Priority", "type": "boolean" }, + "result_reported_at": { + "anyOf": [ + { + "format": "date-time", + "type": "string" + }, + { + "type": "null" + } + ], + "description": "When the run reported success: for an agent whose `report_success` is `on_result`, once its result step published, or once its run started if it has no result step; else when the run completed. From then on `output` is the run's answer and never changes; stop polling here if you only need the answer. `status` still follows every step, so a run can report success and later fail. Null until the run reports, for a run that failed before reporting, and for runs that ended before this field existed.", + "title": "Result Reported At" + }, "run_id": { "description": "Unique identifier for the agent run.", "title": "Run Id", @@ -944,7 +957,7 @@ "type": "null" } ], - "description": "Cumulative milliseconds the run was parked on standard-mode wait steps. Subtracted from active duration in run-detail and duration-stats responses, exactly like hitl_wait_ms. Priority waits block inline and are not counted here.", + "description": "Sum over the run's standard-mode wait steps of each one's wait, in milliseconds. Waits that overlap are each counted in full. Priority waits block inline and are not counted here.", "title": "Wait Ms" } }, @@ -1175,7 +1188,7 @@ "type": "null" } ], - "description": "Metadata to make available for string substitution expressions in agent tasks.", + "description": "Metadata for string substitution in agent tasks. Each key must be one the agent's trigger declares (or sets); a run that sends another custom key, or lacks a required one, is refused with HTTP 400.", "title": "Metadata" }, "replay_of_run_id": { @@ -1389,8 +1402,8 @@ "title": "Disabled Reason" }, "evaluation_mode": { - "default": "eval_and_retry", - "description": "Evaluation mode: output_expectation, eval_and_retry, or sample_and_flag.", + "default": "sample_and_flag", + "description": "Evaluation mode: output_expectation or sample_and_flag.", "title": "Evaluation Mode", "type": "string" }, @@ -1440,7 +1453,7 @@ }, "max_retries": { "default": 3, - "description": "Max retries for eval_and_retry mode.", + "description": "Deprecated: has no effect.", "title": "Max Retries", "type": "integer" }, @@ -1503,7 +1516,7 @@ "type": "null" } ], - "description": "Failure signals that trigger rollback. Defaults to agent_eval_fail, governance_block, agent_run_failed when null.", + "description": "Failure signals that trigger rollback. Defaults to governance_block, agent_run_failed when null.", "title": "Prompt Model Auto Rollback Triggers" }, "prompt_model_auto_upgrade_strategy": { @@ -1514,7 +1527,7 @@ }, "retry_on_failure": { "default": true, - "description": "Whether to retry on evaluation failure.", + "description": "Deprecated: has no effect.", "title": "Retry On Failure", "type": "boolean" }, @@ -3412,7 +3425,7 @@ "type": "object" }, "CreateEvaluationCriteriaRequest": { - "description": "Request body for creating an evaluation criteria.\n\nThe evaluation mode, retry settings, and sample frequency are set at the\nagent level, not per-criteria.", + "description": "Request body for creating an evaluation criteria.\n\nstep_id names the display_result or streaming_result step to score. The\nevaluation mode and sample frequency are set at the\nagent level, not per-criteria.", "properties": { "description": { "anyOf": [ @@ -3522,12 +3535,14 @@ }, "retry_count": { "default": 0, + "description": "Deprecated: has no effect on any count.", "minimum": 0.0, "title": "Retry Count", "type": "integer" }, "retry_triggered": { "default": false, + "description": "Retired; true returns 400 from Seclai-Version 2026-11-04.", "title": "Retry Triggered", "type": "boolean" }, @@ -4127,6 +4142,29 @@ "title": "CreateSourceBody", "type": "object" }, + "DeclarableMetadataKey": { + "properties": { + "always_present": { + "description": "Whether every run carries this key.", + "title": "Always Present", + "type": "boolean" + }, + "key": { + "title": "Key", + "type": "string" + }, + "origin": { + "$ref": "#/components/schemas/MetadataOrigin" + } + }, + "required": [ + "key", + "origin", + "always_present" + ], + "title": "DeclarableMetadataKey", + "type": "object" + }, "DmarcFailingSourceResponse": { "properties": { "failed_count": { @@ -5062,7 +5100,7 @@ "type": "boolean" }, "evaluation_mode": { - "description": "Runtime behavior mode. output_expectation (manual validation), eval_and_retry (every run + retry), sample_and_flag (sampled monitoring).", + "description": "Runtime behavior mode. output_expectation (manual validation), sample_and_flag (sampled monitoring).", "title": "Evaluation Mode", "type": "string" }, @@ -5106,6 +5144,7 @@ "type": "string" }, "max_retries": { + "description": "Deprecated: has no effect.", "title": "Max Retries", "type": "integer" }, @@ -5124,6 +5163,7 @@ "type": "object" }, "retry_on_failure": { + "description": "Deprecated: has no effect.", "title": "Retry On Failure", "type": "boolean" }, @@ -5253,10 +5293,12 @@ "type": "string" }, "retry_count": { + "description": "Deprecated.", "title": "Retry Count", "type": "integer" }, "retry_triggered": { + "description": "Deprecated.", "title": "Retry Triggered", "type": "boolean" }, @@ -5439,10 +5481,12 @@ "type": "string" }, "retry_count": { + "description": "Deprecated.", "title": "Retry Count", "type": "integer" }, "retry_triggered": { + "description": "Deprecated.", "title": "Retry Triggered", "type": "boolean" }, @@ -6118,7 +6162,7 @@ "title": "Step Id" }, "step_type": { - "description": "The step type to generate config for (e.g. 'regex_replace', 'gate', 'text', 'prompt_call', 'retrieval').", + "description": "The step type to generate config for (e.g. 'regex_replace', 'if_else', 'text', 'prompt_call', 'retrieval').", "title": "Step Type", "type": "string" }, @@ -6726,6 +6770,27 @@ "title": "KnowledgeBaseListResponseModel", "type": "object" }, + "KnowledgeBaseMetadataKeyStatsResponse": { + "properties": { + "knowledge_base_id": { + "title": "Knowledge Base Id", + "type": "string" + }, + "windows": { + "items": { + "$ref": "#/components/schemas/MetadataKeyWindowResponse" + }, + "title": "Windows", + "type": "array" + } + }, + "required": [ + "knowledge_base_id", + "windows" + ], + "title": "KnowledgeBaseMetadataKeyStats", + "type": "object" + }, "KnowledgeBaseResponseModel": { "description": "Response model for a single knowledge base.", "properties": { @@ -7278,6 +7343,198 @@ "title": "MemoryBank", "type": "object" }, + "MetadataKeyCountResponse": { + "properties": { + "distinct_values": { + "description": "How many different values it takes.", + "title": "Distinct Values", + "type": "integer" + }, + "items": { + "description": "Items in the window carrying the key.", + "title": "Items", + "type": "integer" + }, + "key": { + "description": "The item-metadata key.", + "title": "Key", + "type": "string" + }, + "top_values": { + "description": "Its 5 most frequent values, most frequent first.", + "items": { + "$ref": "#/components/schemas/MetadataValueCountResponse" + }, + "title": "Top Values", + "type": "array" + } + }, + "required": [ + "key", + "items", + "distinct_values", + "top_values" + ], + "title": "MetadataKeyCountResponse", + "type": "object" + }, + "MetadataKeyDeclaration": { + "additionalProperties": false, + "properties": { + "key": { + "description": "Steps reference the key as ``{{metadata.}}``.", + "maxLength": 255, + "minLength": 1, + "title": "Key", + "type": "string" + }, + "origin": { + "$ref": "#/components/schemas/MetadataOrigin", + "description": "``system`` and ``source`` keys are written by the platform; ``custom`` keys are sent by the caller or set on the trigger." + }, + "required": { + "default": false, + "description": "A caller-started run without this key is refused. Only a ``custom`` key can be required.", + "title": "Required", + "type": "boolean" + } + }, + "required": [ + "key", + "origin" + ], + "title": "MetadataKeyDeclaration", + "type": "object" + }, + "MetadataKeyWindowResponse": { + "properties": { + "keys": { + "description": "Keys seen in the window, most frequent first.", + "items": { + "$ref": "#/components/schemas/MetadataKeyCountResponse" + }, + "title": "Keys", + "type": "array" + }, + "since": { + "anyOf": [ + { + "format": "date-time", + "type": "string" + }, + { + "type": "null" + } + ], + "description": "The window's start; null for all time.", + "title": "Since" + }, + "total_items": { + "description": "Items in the window.", + "title": "Total Items", + "type": "integer" + }, + "window": { + "enum": [ + "all_time", + "last_30_days", + "last_7_days" + ], + "title": "Window", + "type": "string" + } + }, + "required": [ + "window", + "since", + "total_items", + "keys" + ], + "title": "MetadataKeyWindowResponse", + "type": "object" + }, + "MetadataKeysResponse": { + "properties": { + "accepts_any_source_key": { + "description": "True for content triggers, which may declare any item-metadata key.", + "title": "Accepts Any Source Key", + "type": "boolean" + }, + "agent_id": { + "format": "uuid", + "title": "Agent Id", + "type": "string" + }, + "declarable_keys": { + "description": "The system and source keys this trigger's type can declare.", + "items": { + "$ref": "#/components/schemas/DeclarableMetadataKey" + }, + "title": "Declarable Keys", + "type": "array" + }, + "metadata_keys": { + "description": "The keys the agent's steps may reference. Static trigger metadata and Slack form fields are declared custom keys too, and are not repeated here.", + "items": { + "$ref": "#/components/schemas/MetadataKeyDeclaration" + }, + "title": "Metadata Keys", + "type": "array" + }, + "trigger_id": { + "format": "uuid", + "title": "Trigger Id", + "type": "string" + }, + "warnings": { + "description": "Set by a PUT that changes the declaration: agents whose `call_agent` mapping no longer fits it. Always empty on GET.", + "items": { + "type": "string" + }, + "title": "Warnings", + "type": "array" + } + }, + "required": [ + "trigger_id", + "agent_id", + "metadata_keys", + "declarable_keys", + "accepts_any_source_key" + ], + "title": "MetadataKeysResponse", + "type": "object" + }, + "MetadataOrigin": { + "description": "Who writes a run metadata key (docs/architecture/run-metadata.md, Q2).", + "enum": [ + "system", + "source", + "custom" + ], + "title": "MetadataOrigin", + "type": "string" + }, + "MetadataValueCountResponse": { + "properties": { + "items": { + "description": "Items carrying this value.", + "title": "Items", + "type": "integer" + }, + "value": { + "description": "The value as text, cut to 100 characters; an object is shown as {\u2026} and a list as [\u2026].", + "title": "Value", + "type": "string" + } + }, + "required": [ + "value", + "items" + ], + "title": "MetadataValueCountResponse", + "type": "object" + }, "ModalityRateResponse": { "description": "Per-modality rate for an LLM that prices image/audio/video output\n(or input) at a rate distinct from the default text rate.\n\nExample: Gemini 3.1 Flash Image charges $3/1M output tokens for\ntext but $60/1M output tokens for generated images. The image rate\nsurfaces here with ``modality=\"image\"`` and ``output_credits_per_1000_tokens``\nset; the default text rate stays on the parent model fields.", "properties": { @@ -7969,6 +8226,23 @@ "title": "SetAutoBlockModeRequest", "type": "object" }, + "SetMetadataKeysRequest": { + "properties": { + "metadata_keys": { + "description": "The whole declaration; replaces the trigger's current one.", + "items": { + "$ref": "#/components/schemas/MetadataKeyDeclaration" + }, + "title": "Metadata Keys", + "type": "array" + } + }, + "required": [ + "metadata_keys" + ], + "title": "SetMetadataKeysRequest", + "type": "object" + }, "SetSlackTriggerConfigApiRequest": { "description": "Omitted fields are left unchanged.", "properties": { @@ -7983,6 +8257,18 @@ ], "description": "`everywhere` in scope, or `channel_opt_in`." }, + "alias": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "description": "The name people type after `/seclai` to run this agent: lowercase letters, digits and hyphens, starting with a letter, unique in the Slack workspace. Generated from the agent's name when not set or empty.", + "title": "Alias" + }, "allow_dms": { "anyOf": [ { @@ -8022,7 +8308,7 @@ "type": "null" } ], - "description": "Form fields `{key, label, type, required, hint, choices, max_length}`. `type`: short_text, long_text, number, single_choice, multi_choice, yes_no, date, slack_user or slack_channel. Each value reaches the run as `{{metadata.}}`.", + "description": "Form fields `{key, label, type, required, hint, choices, max_length}`. `type`: short_text, long_text, number, single_choice, multi_choice, yes_no, date, slack_user or slack_channel. Each value reaches the run as `{{metadata.}}`, not in the run's input, which is the text after `/seclai ` or the message the shortcut was used on, and its files.", "title": "Form" }, "include_invoker_email": { @@ -8036,18 +8322,6 @@ ], "title": "Include Invoker Email" }, - "input": { - "anyOf": [ - { - "type": "string" - }, - { - "type": "null" - } - ], - "description": "The trigger's input template. Each `{{metadata.}}` in it must be a form field key or a platform key; it is checked together with `form`.", - "title": "Input" - }, "max_turns": { "anyOf": [ { @@ -8225,6 +8499,18 @@ "title": "Agent Id", "type": "string" }, + "alias": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "description": "The name people type after `/seclai` to run this agent.", + "title": "Alias" + }, "allow_dms": { "title": "Allow Dms", "type": "boolean" @@ -9817,7 +10103,7 @@ "type": "object" }, "UpdateEvaluationCriteriaRequest": { - "description": "Request body for updating an evaluation criteria.\n\nRetry settings and sample frequency are set at the agent level.", + "description": "Request body for updating an evaluation criteria.\n\nThe evaluation mode and sample frequency are set at the agent level.", "properties": { "description": { "anyOf": [ @@ -11115,7 +11401,7 @@ "type": "null" } ], - "description": "Evaluation mode: 'output_expectation', 'eval_and_retry', or 'sample_and_flag'.", + "description": "Evaluation mode: 'output_expectation' or 'sample_and_flag'. From Seclai-Version 2026-11-04 'eval_and_retry' returns 400; earlier versions store it as 'sample_and_flag'.", "title": "Evaluation Mode" }, "max_credits_per_period": { @@ -11153,7 +11439,7 @@ "type": "null" } ], - "description": "Max retries for eval_and_retry mode (1-10).", + "description": "Deprecated: accepted and has no effect.", "title": "Max Retries" }, "max_runs_per_day": { @@ -11231,7 +11517,7 @@ "type": "null" } ], - "description": "Failure signals that trigger rollback: agent_eval_fail, governance_flag, governance_block, agent_run_failed.", + "description": "Failure signals that trigger rollback: governance_flag, governance_block, agent_run_failed. agent_eval_fail is accepted and never fires; in the list, a run with a failed evaluation is never picked as the rollback target.", "title": "Prompt Model Auto Rollback Triggers" }, "prompt_model_auto_upgrade_strategy": { @@ -11254,7 +11540,7 @@ "type": "null" } ], - "description": "Whether to retry on evaluation failure.", + "description": "Deprecated: accepted and has no effect.", "title": "Retry On Failure" }, "sampling_config": { @@ -14955,7 +15241,7 @@ ] }, "patch": { - "description": "Update an evaluation criteria.\n\nAccepts a partial update \u2013 only the fields included in the request body\nare modified. Use this to toggle enabled/disabled, change the\nevaluation prompt, adjust retries, or rename the criteria.", + "description": "Update an evaluation criteria.\n\nAccepts a partial update \u2013 only the fields included in the request body\nare modified. Use this to toggle enabled/disabled, change the\nevaluation prompt, or rename the criteria. From Seclai-Version 2026-11-04 a\nchanged `step_id` must name a `display_result` or `streaming_result` step.", "operationId": "update_evaluation_criteria_api_agents_evaluation_criteria__criteria_id__patch", "parameters": [ { @@ -15285,7 +15571,7 @@ ] }, "post": { - "description": "Record an evaluation result for a criteria.\n\nUse this endpoint to push results from external test harnesses, CI/CD\npipelines, or custom evaluation logic. Each result is linked to an\nagent run and optionally a specific step run.", + "description": "Record an evaluation result for a criteria.\n\nUse this endpoint to push results from external test harnesses, CI/CD\npipelines, or custom evaluation logic. Each result is linked to an\nagent run and optionally a specific step run. From Seclai-Version 2026-11-04\n`retry_triggered: true` returns 400: evaluations never retry a step.", "operationId": "create_evaluation_result_api_agents_evaluation_criteria__criteria_id__results_post", "parameters": [ { @@ -15429,7 +15715,7 @@ }, "/agents/evaluation-results/non-manual-summary": { "get": { - "description": "Get an evaluation summary for API key clients.\n\nReturns aggregated pass/fail/flagged counts and pass rates for each evaluation mode (eval_and_retry, sample_and_flag).\n\nWith ``2026-10-21`` or later, ``error`` results (the evaluator returned no\nverdict) are left out of ``failed``, and the rates are over passed and\nfailed results. Earlier versions count them in ``failed`` and rate over\nevery result. ``error`` is returned on every version.\n\nThe ``agent_id`` scoping parameter is part of the ``2026-07-27`` changeset\n(opt in via the ``Seclai-Version`` header). Clients on the legacy baseline\nalways receive the account-wide rollup; ``agent_id`` is ignored for them so\nthe endpoint's behavior is frozen. With ``2026-07-27`` or later, the summary\nis scoped to ``agent_id`` when supplied (and returns 404 if it doesn't exist).", + "description": "Get an evaluation summary for API key clients.\n\nReturns aggregated pass/fail/flagged counts and pass rates for each automatic evaluation mode.\n\nWith ``2026-10-21`` or later, ``error`` results (the evaluator returned no\nverdict) are left out of ``failed``, and the rates are over passed and\nfailed results. Earlier versions count them in ``failed`` and rate over\nevery result. ``error`` is returned on every version.\n\nThe ``agent_id`` scoping parameter is part of the ``2026-07-27`` changeset\n(opt in via the ``Seclai-Version`` header). Clients on the legacy baseline\nalways receive the account-wide rollup; ``agent_id`` is ignored for them so\nthe endpoint's behavior is frozen. With ``2026-07-27`` or later, the summary\nis scoped to ``agent_id`` when supplied (and returns 404 if it doesn't exist).", "operationId": "get_non_manual_evaluation_summary_api_agents_evaluation_results_non_manual_summary_get", "parameters": [ { @@ -16309,7 +16595,7 @@ ] }, "put": { - "description": "Update an agent's name, description, evaluation settings, model lifecycle settings and run limits.\n\nEvaluation settings: `evaluation_mode` ('output_expectation', 'eval_and_retry', 'sample_and_flag'), `default_evaluation_tier` ('fast', 'balanced', 'thorough'), `max_retries`, `retry_on_failure`, `sampling_config`.\n\nModel lifecycle settings: `prompt_model_auto_upgrade_strategy` ('none', 'early_adopter', 'middle_of_road', 'cautious_adopter'), `prompt_model_auto_rollback_enabled`, `prompt_model_auto_rollback_triggers` (list of 'agent_eval_fail', 'governance_flag', 'governance_block', 'agent_run_failed').\n\nRun limits (each must be greater than 0; pass the paired `set_: true` with the value omitted to remove one): `max_credits_per_run`, `max_credits_per_period`, `max_runs_per_hour`, `max_runs_per_day`, `max_runs_total`. A runs-per-hour or per-day value above the plan's run quota is rejected with 400.\n\nReplacing the workflow from an export:\n- Pass `agent_definition` with the JSON shape produced by `GET /agents/{id}/export`. Update only touches the workflow + agent metadata \u2014 alert_configs, evaluation_criteria, governance_policies, schedules, and solution links from the imported file are NOT applied (use the dedicated endpoints for those, or `POST /agents` to import as a new agent).\n- `entity_remap: {source_uuid: target_uuid}` substitutes workflow entity refs before save (same shape as `POST /agents`).\n\nAt least one field must be provided.\n\nAuth & scoping:\n- Requires `X-API-Key` header or OAuth Bearer token. You can only update agents belonging to your account.", + "description": "Update an agent's name, description, evaluation settings, model lifecycle settings and run limits.\n\nEvaluation settings: `evaluation_mode` ('output_expectation', 'sample_and_flag'; see the field), `default_evaluation_tier` ('fast', 'balanced', 'thorough'), `sampling_config`. `max_retries` and `retry_on_failure` are accepted and have no effect.\n\nModel lifecycle settings: `prompt_model_auto_upgrade_strategy` ('none', 'early_adopter', 'middle_of_road', 'cautious_adopter'), `prompt_model_auto_rollback_enabled`, `prompt_model_auto_rollback_triggers` (list of 'governance_flag', 'governance_block', 'agent_run_failed'; 'agent_eval_fail' never fires, and in the list keeps a run with a failed evaluation from being a rollback target).\n\nRun limits (each must be greater than 0; pass the paired `set_: true` with the value omitted to remove one): `max_credits_per_run`, `max_credits_per_period`, `max_runs_per_hour`, `max_runs_per_day`, `max_runs_total`. A runs-per-hour or per-day value above the plan's run quota is rejected with 400.\n\nReplacing the workflow from an export:\n- Pass `agent_definition` with the JSON shape produced by `GET /agents/{id}/export`. Update only touches the workflow + agent metadata \u2014 alert_configs, evaluation_criteria, governance_policies, schedules, and solution links from the imported file are NOT applied (use the dedicated endpoints for those, or `POST /agents` to import as a new agent).\n- `entity_remap: {source_uuid: target_uuid}` substitutes workflow entity refs before save (same shape as `POST /agents`).\n\nAt least one field must be provided.\n\nAuth & scoping:\n- Requires `X-API-Key` header or OAuth Bearer token. You can only update agents belonging to your account.", "operationId": "update_agent_api_agents__agent_id__put", "parameters": [ { @@ -16887,7 +17173,7 @@ }, "/agents/{agent_id}/definition": { "get": { - "description": "Fetch the current agent definition from the main branch.\n\nThe response includes `change_id` which must be provided when updating the definition (optimistic locking).\n\nThe definition contains the agent's step workflow. Available step types:\n- `prompt_call`: Call an LLM with a prompt template\n- `retrieval`: Search a knowledge base\n- `regex_replace`: Reshape text via ordered regex find/replace rules\n- `gate`: Evaluate conditions, stop or continue child execution\n- `retry`: Re-execute from a target ancestor step (for quality-control loops; pair with a `gate` step for conditional retrying. Fields: `target_step_id` (ancestor step ID), `max_retries` (1\u201310))\n- `evaluate_step`: Score a selected previous step output and emit JSON with `score`, `passed`, and `pass_threshold` (fields: `target_step_id`, `evaluation_prompt`, `pass_threshold`, optional `evaluation_tier`, optional `expectation_config`)\n- `extract_data`: Progressively read and analyze large input\n- `extract_content`: Extract structured data (JSON, HTML, XML)\n- `add_chat_turn` / `load_chat_history`: Record a turn or load running history from a conversation memory bank\n- `add_memory` / `search_memory` / `load_memory`: Write, semantic-search, or load entries on a general memory bank\n- `send_email`: Send email with step output\n- `webhook_call`: POST data to an external URL\n- `write_aws_s3_object`: Write output to S3\n- `call_agent`: Invoke another agent\n- `write_metadata`: Write a value to content metadata (for filtering/gates; content-triggered agents only. Fields: `metadata_key`, `content`)\n- `write_content_attachment`: Write a file-backed attachment to content (optionally indexed for retrieval; content-triggered agents only. Fields: `attachment_key`, `content`, `content_type`, `indexed`)\n- `load_content_attachment`: Load a previously written attachment (content-triggered agents only. Fields: `attachment_key`)\n- `load_content`: Load the full text body of a source document (typically used with content-triggered agents; can also load by explicit `content_version_id`. Fields: `content_version_id` optional)\n- `streaming_result`: Stream LLM tokens in real-time via SSE (must be a direct child of `prompt_call`; requires `dynamic_input` or `template_input` trigger; `priority: true` enables real-time streaming). To stream **and** return a schema-validated payload, add a sibling branch `extract_content` (`expected_format: \"json\"` + `json_schema`) \u2192 `display_result` under the same `prompt_call` \u2014 a `display_result` takes precedence over the stream regardless of which finishes first, so its validated output becomes the run's result while the tokens act as a progress channel\n- `display_result`: Show output to the user\n- `join`: Merge parallel branches\n- `merge`: Combine multiple inputs into a single templated output\n- `text`: Static text literal\n- `for_each`: Iterate a body over a list of items (body lives in `body[]`)\n- `if_else`: Conditional dispatch. Evaluates `conditions` (same shape as `gate`) and runs `then_steps` on match, otherwise the optional `else_steps`. The chosen branch's output flows to the if_else step's own `child_steps` (post-branch continuation chain). **`display_result` and `streaming_result` are not allowed inside `then_steps` / `else_steps`** \u2014 end each branch with a content-producing step (e.g. `text`, `prompt_call`) and place the single `display_result` in `child_steps`.\n- `switch`: Single-discriminator dispatch. Renders `discriminator` (default `{{input}}`) and routes to the first matching `cases[]` entry (equality by default; pass a list in `match` for `$in` semantics) or to `else_steps` when nothing matches. The chosen case's output flows to the switch step's own `child_steps`. **`display_result` and `streaming_result` are not allowed inside `cases[].steps` or `else_steps`** \u2014 end each case with a content-producing step and place the single `display_result` in `child_steps`.\n\nAuth & scoping:\n- Requires `X-API-Key` header or OAuth Bearer token. You can only access agents belonging to your account.", + "description": "Fetch the current agent definition from the main branch.\n\nThe response includes `change_id` which must be provided when updating the definition (optimistic locking).\n\nThe definition contains the agent's step workflow. Available step types:\n- `prompt_call`: Call an LLM with a prompt template\n- `retrieval`: Search a knowledge base\n- `regex_replace`: Reshape text via ordered regex find/replace rules\n- `repeat_until`: Run a body (`body[]`) again until its `conditions` match the iteration's verdict or `max_iterations` (1\u20131000, default 3) is reached. Fields: `conditions`, `match`, optional `verdict_template`, `output` (`last` or `all`), `fail_when_max_reached`\n- `evaluate_step`: Score a selected previous step output and emit JSON with `score`, `passed`, and `pass_threshold` (fields: `target_step_id`, `evaluation_prompt`, `pass_threshold`, optional `evaluation_tier`, optional `expectation_config`)\n- `extract_data`: Progressively read and analyze large input\n- `extract_content`: Extract structured data (JSON, HTML, XML)\n- `add_chat_turn` / `load_chat_history`: Record a turn or load running history from a conversation memory bank\n- `add_memory` / `search_memory` / `load_memory`: Write, semantic-search, or load entries on a general memory bank\n- `send_email`: Send email with step output\n- `webhook_call`: POST data to an external URL\n- `write_aws_s3_object`: Write output to S3\n- `call_agent`: Invoke another agent\n- `write_metadata`: Write a value to content metadata (for filtering/gates; content-triggered agents only. Fields: `metadata_key`, `content`)\n- `write_content_attachment`: Write a file-backed attachment to content (optionally indexed for retrieval; content-triggered agents only. Fields: `attachment_key`, `content`, `content_type`, `indexed`)\n- `load_content_attachment`: Load a previously written attachment (content-triggered agents only. Fields: `attachment_key`)\n- `load_content`: Load the full text body of a source document (typically used with content-triggered agents; can also load by explicit `content_version_id`. Fields: `content_version_id` optional)\n- `streaming_result`: Stream LLM tokens in real-time via SSE (must be a direct child of `prompt_call`; requires `dynamic_input` or `template_input` trigger; `priority: true` enables real-time streaming). An agent has at most one `streaming_result` or `display_result`; saving refuses a second\n- `display_result`: Show output to the user\n- `join`: Merge parallel branches\n- `merge`: Combine multiple inputs into a single templated output\n- `text`: Static text literal\n- `for_each`: Iterate a body over a list of items (body lives in `body[]`)\n- `if_else`: Conditional dispatch. Evaluates `conditions` and runs `then_steps` on match, otherwise the optional `else_steps`. The chosen branch's output flows to the if_else step's own `child_steps` (post-branch continuation chain). **`display_result` and `streaming_result` are not allowed inside `then_steps` / `else_steps`** \u2014 end each branch with a content-producing step (e.g. `text`, `prompt_call`) and place the single `display_result` in `child_steps`.\n- `switch`: Single-discriminator dispatch. Renders `discriminator` (default `{{input}}`) and routes to the first matching `cases[]` entry (equality by default; pass a list in `match` for `$in` semantics) or to `else_steps` when nothing matches. The chosen case's output flows to the switch step's own `child_steps`. **`display_result` and `streaming_result` are not allowed inside `cases[].steps` or `else_steps`** \u2014 end each case with a content-producing step and place the single `display_result` in `child_steps`.\n\nAuth & scoping:\n- Requires `X-API-Key` header or OAuth Bearer token. You can only access agents belonging to your account.", "operationId": "get_agent_definition_api_agents__agent_id__definition_get", "parameters": [ { @@ -16952,7 +17238,7 @@ ] }, "put": { - "description": "Update the agent's definition on the main branch.\n\nUses **optimistic locking**: provide `expected_change_id` from the last `GET /agents/{agent_id}/definition`. Returns `409 Conflict` if the definition was modified since your last read.\n\nThe definition contains the agent's step workflow. Step types include `prompt_call`, `retrieval`, `regex_replace`, `gate`, `retry`, `evaluate_step`, `extract_data`, `extract_content`, `add_chat_turn`, `load_chat_history`, `add_memory`, `search_memory`, `load_memory`, `streaming_result`, `send_email`, `webhook_call`, `write_aws_s3_object`, `call_agent`, `write_metadata`, `write_content_attachment`, `load_content_attachment`, `load_content`, `display_result`, `join`, `merge`, `text`, `for_each`, `if_else`, and `switch`. Non-composite step types (`display_result`, `join`, `retry`, `streaming_result`) cannot contain child steps.\n\n**Retry steps** re-execute from a target ancestor step for quality-control loops. Configure with `target_step_id` (ancestor step ID) and `max_retries` (1\u201310). Best practice: place a `gate` step before the retry to make retries conditional.\n\n**if_else** runs `then_steps` when its `conditions` (same shape as `gate`) match, otherwise its optional `else_steps`. Either branch's output flows to the if_else step's own `child_steps` (the post-branch continuation chain).\n\n**switch** dispatches on a `discriminator` template (default `{{input}}`) to the first matching case (equality by default; pass a list in `match` for `$in` semantics) or to `else_steps` when no case matches. Cases own their own `steps` subtrees; the chosen branch's output flows to the switch step's own `child_steps`.\n\nAuth & scoping:\n- Requires `X-API-Key` header or OAuth Bearer token. You can only update agents belonging to your account.", + "description": "Update the agent's definition on the main branch.\n\nUses **optimistic locking**: provide `expected_change_id` from the last `GET /agents/{agent_id}/definition`. Returns `409 Conflict` if the definition was modified since your last read.\n\nThe definition contains the agent's step workflow. Step types include `prompt_call`, `retrieval`, `regex_replace`, `evaluate_step`, `extract_data`, `extract_content`, `add_chat_turn`, `load_chat_history`, `add_memory`, `search_memory`, `load_memory`, `streaming_result`, `send_email`, `webhook_call`, `write_aws_s3_object`, `call_agent`, `write_metadata`, `write_content_attachment`, `load_content_attachment`, `load_content`, `display_result`, `join`, `merge`, `text`, `for_each`, `repeat_until`, `if_else`, and `switch`. Non-composite step types (`display_result`, `join`, `streaming_result`) cannot contain child steps. A definition holding a `gate` or `retry` step is rejected: use `if_else`, or `repeat_until` to run steps again until a condition holds.\n\n**repeat_until** runs its `body[]`, then tests its `conditions` against the iteration's verdict (`verdict_template`, else the iteration's value); it repeats until they match or `max_iterations` is reached.\n\n**if_else** runs `then_steps` when its `conditions` match, otherwise its optional `else_steps`. Either branch's output flows to the if_else step's own `child_steps` (the post-branch continuation chain).\n\n**switch** dispatches on a `discriminator` template (default `{{input}}`) to the first matching case (equality by default; pass a list in `match` for `$in` semantics) or to `else_steps` when no case matches. Cases own their own `steps` subtrees; the chosen branch's output flows to the switch step's own `child_steps`.\n\nAuth & scoping:\n- Requires `X-API-Key` header or OAuth Bearer token. You can only update agents belonging to your account.", "operationId": "update_agent_definition_api_agents__agent_id__definition_put", "parameters": [ { @@ -17255,7 +17541,7 @@ ] }, "post": { - "description": "Create new step evaluation settings for an agent.\n\nThe evaluation mode, retry settings, and sample frequency are inherited\nfrom the agent and stored on the criteria row for historical reference.", + "description": "Create new step evaluation settings for an agent.\n\nFrom Seclai-Version 2026-11-04, `step_id` must name a `display_result` or\n`streaming_result` step, the run's published result; another step returns 400.\nThe evaluation mode and sample frequency are the agent's.", "operationId": "create_evaluation_criteria_api_agents__agent_id__evaluation_criteria_post", "parameters": [ { @@ -17727,7 +18013,7 @@ }, "/agents/{agent_id}/export": { "get": { - "description": "Export an agent definition as a portable JSON snapshot.\n\nThe response contains the full definition, trigger configuration with schedules, alert configs, evaluation criteria, agent-scoped governance policies, and a resolved dependency manifest that maps every referenced external entity UUID to its human-readable name.\n\nResponse shape:\n- `export_version`: schema version (currently `\"5\"`)\n- `exported_at`: ISO-8601 timestamp\n- `agent`: name, description, schema_version, definition, timestamps\n- `trigger`: trigger type, input template, schedules\n- `alert_configs`: alert type, thresholds, recipients\n- `evaluation_criteria`: evaluation settings per step\n- `governance_policies`: agent-scoped governance policies\n- `dependencies`: knowledge_bases, memory_banks, source_connections, agents, users\n\nQuery params:\n- `download` (default true): when true, sets `Content-Disposition: attachment` so clients treat the response as a file download.\n\nAuth & scoping:\n- Requires `X-API-Key`. You can only export agents belonging to your account.", + "description": "Export an agent definition as a portable JSON snapshot.\n\nThe response contains the full definition, trigger configuration with schedules, alert configs, evaluation criteria, agent-scoped governance policies, and a resolved dependency manifest that maps every referenced external entity UUID to its human-readable name.\n\nResponse shape:\n- `export_version`: schema version (currently `\"6\"`)\n- `exported_at`: ISO-8601 timestamp\n- `agent`: name, description, schema_version, definition, timestamps\n- `trigger`: trigger type, input template, schedules\n- `alert_configs`: alert type, thresholds, recipients\n- `evaluation_criteria`: evaluation settings per step\n- `governance_policies`: agent-scoped governance policies\n- `dependencies`: knowledge_bases, memory_banks, source_connections, agents, users\n\nQuery params:\n- `download` (default true): when true, sets `Content-Disposition: attachment` so clients treat the response as a file download.\n\nAuth & scoping:\n- Requires `X-API-Key`. You can only export agents belonging to your account.", "operationId": "export_agent_api_agents__agent_id__export_get", "parameters": [ { @@ -17992,7 +18278,7 @@ ] }, "post": { - "description": "Start an agent run.\n\nAn *agent* is an automated workflow that can monitor content from your sources, process it with AI, and trigger actions. This endpoint creates a new run and returns a `run_id` you can poll to retrieve status and output.\n\nWhen to use:\n- Use this endpoint for request/response style integrations where polling is acceptable.\n- Use `POST /agents/{agent_id}/runs/stream` if you need real-time progress via SSE.\n\nKey fields:\n- `input`: text input for agents with a `dynamic_input` trigger.\n- `input_upload_id` / `input_upload_ids`: reference one or more files previously uploaded via `POST /agents/{agent_id}/upload-input`. Send them **with** `input` to pair prompt text with the files (the text leads, each file follows under its heading); only the two upload fields are mutually exclusive with each other.\n- `priority`: set true for latency-sensitive, user-facing work. For agents with a `streaming_result` step, set `priority=true` to enable real-time token streaming; otherwise the run still proceeds, but without live token streaming.\n- `metadata`: a JSON object that becomes available to agent steps for string substitution.\n\nAfter starting:\n- Poll `GET /agents/runs/{run_id}` until `status` is `completed` or `failed`.\n- Use `include_step_outputs=true` to include per-step outputs, timing, and credits.\n\nAuth & scoping:\n- Requires `X-API-Key` header or OAuth Bearer token. All resources are scoped to the caller's account.", + "description": "Start an agent run.\n\nAn *agent* is an automated workflow that can monitor content from your sources, process it with AI, and trigger actions. This endpoint creates a new run and returns a `run_id` you can poll to retrieve status and output.\n\nWhen to use:\n- Use this endpoint for request/response style integrations where polling is acceptable.\n- Use `POST /agents/{agent_id}/runs/stream` if you need real-time progress via SSE.\n\nKey fields:\n- `input`: text input for agents with a `dynamic_input` trigger.\n- `input_upload_id` / `input_upload_ids`: reference one or more files previously uploaded via `POST /agents/{agent_id}/upload-input`. Send them **with** `input` to pair prompt text with the files (the text leads, each file follows under its heading); only the two upload fields are mutually exclusive with each other.\n- `priority`: set true for latency-sensitive, user-facing work. For agents with a `streaming_result` step, set `priority=true` to enable real-time token streaming; otherwise the run still proceeds, but without live token streaming.\n- `metadata`: a JSON object for string substitution. Each key must be one the trigger declares (`GET /agents/{agent_id}/triggers/{trigger_id}/metadata-keys`); a run that sends another custom key, or lacks a required one, is refused with HTTP 400.\n\nAfter starting:\n- Poll `GET /agents/runs/{run_id}` until `result_reported_at` is set (the run's answer is in `output`) or `status` is `failed`; poll on until `status` is `completed` or `failed` if you need to know that every step finished.\n- Use `include_step_outputs=true` to include per-step outputs, timing, and credits.\n\nAuth & scoping:\n- Requires `X-API-Key` header or OAuth Bearer token. All resources are scoped to the caller's account.", "operationId": "run_agent_api_agents__agent_id__runs_post", "parameters": [ { @@ -18079,7 +18365,7 @@ }, "/agents/{agent_id}/runs/stream": { "post": { - "description": "Start a **priority** agent run and stream run events using Server-Sent Events (SSE).\n\nThis is the best option for interactive UIs where you want progress updates as the run executes.\n\nHow it works:\n- The first `init` event contains an `AgentRunResponse` snapshot, including the `run_id`.\n- Subsequent events are forwarded from the run event stream (status changes, step events, etc).\n- If the agent contains a `streaming_result` step, `stream_token` events deliver individual LLM tokens (with a `token` field) and a `stream_end` event signals completion.\n- With `include_reasoning=true`, `stream_reasoning` events (with a `text` field) carry the model's reasoning while it thinks, when the `streaming_result` step's author enabled `show_reasoning`. Reasoning is never part of `output`.\n- The final `done` event contains the terminal snapshot (including `output` and `credits` when available).\n\nInput options (for `dynamic_input` triggers):\n- `input`: text input passed directly.\n- `input_upload_id` / `input_upload_ids`: reference one or more files uploaded via `POST /agents/{agent_id}/upload-input`. Combine either with `input` to send prompt text alongside the files; only the two upload fields are mutually exclusive with each other.\n\nClient guidance:\n- Keep the connection open and handle keepalive comments.\n- On `timeout` or `error`, the payload includes `run_id` so clients can resume by polling `GET /agents/runs/{run_id}`.\n- `stream_token` events carry the model's raw output. When the agent pairs `streaming_result` with a sibling `extract_content`/`display_result` branch, that branch's `display_result` takes precedence over the stream, so the `done` snapshot's `output` is the validated payload and the tokens are a progress channel \u2014 render tokens live, but read the result from `done`. Check `done.status` first: the streamed text is already the run's output when the stream ends, so a run that FAILED its validation step still carries that raw text in `output`.\n\nAuth & scoping:\n- Requires `X-API-Key` header or OAuth Bearer token. All resources are scoped to the caller's account.", + "description": "Start a **priority** agent run and stream run events using Server-Sent Events (SSE).\n\nThis is the best option for interactive UIs where you want progress updates as the run executes.\n\nHow it works:\n- The first `init` event contains an `AgentRunResponse` snapshot, including the `run_id`.\n- Subsequent events are forwarded from the run event stream (status changes, step events, etc).\n- If the agent contains a `streaming_result` step, `stream_token` events deliver individual LLM tokens (with a `token` field) and a `stream_end` event signals completion.\n- With `include_reasoning=true`, `stream_reasoning` events (with a `text` field) carry the model's reasoning while it thinks, when the `streaming_result` step's author enabled `show_reasoning`. Reasoning is never part of `output`.\n- A `result` event contains the snapshot once the run has reported its result (`result_reported_at` is set and `output` is the answer).\n- The final `done` event contains the terminal snapshot (including `output` and `credits` when available).\n\nInput options (for `dynamic_input` triggers):\n- `input`: text input passed directly.\n- `input_upload_id` / `input_upload_ids`: reference one or more files uploaded via `POST /agents/{agent_id}/upload-input`. Combine either with `input` to send prompt text alongside the files; only the two upload fields are mutually exclusive with each other.\n\nClient guidance:\n- Keep the connection open and handle keepalive comments.\n- On `timeout` or `error`, the payload includes `run_id` so clients can resume by polling `GET /agents/runs/{run_id}`.\n- `stream_token` events carry the model's raw output: render them live, but take the answer from the `result` event. `done.status` says whether the whole run succeeded, which can still fail after its result.\n\nAuth & scoping:\n- Requires `X-API-Key` header or OAuth Bearer token. All resources are scoped to the caller's account.", "operationId": "run_streaming_agent_api_agents__agent_id__runs_stream_post", "parameters": [ { @@ -18121,7 +18407,7 @@ } } }, - "description": "Streams agent run events via Server-Sent Events (SSE); run is always created as priority.\n\nSSE events:\n- `event: init` \u2014 `data` is an `AgentRunResponse` snapshot (includes `run_id`).\n- `event: done` \u2014 `data` is the final `AgentRunResponse` snapshot (includes `output`, `credits`, etc).\n- Other events (e.g. `status`, step events) are forwarded from the run event stream.\n- On `timeout` / `error`, the payload includes `run_id` so clients can fetch status via `GET /agents/runs/{run_id}`." + "description": "Streams agent run events via Server-Sent Events (SSE); run is always created as priority.\n\nSSE events:\n- `event: init` \u2014 `data` is an `AgentRunResponse` snapshot (includes `run_id`).\n- `event: result` \u2014 `data` is an `AgentRunResponse` snapshot once the run reported success (see `result_reported_at`): its `output` is the run's answer. Sent before `done`; for an agent whose `report_success` is `on_result` it can arrive while later steps still run.\n- `event: done` \u2014 `data` is the final `AgentRunResponse` snapshot (includes `output`, `credits`, etc).\n- Other events (e.g. `status`, step events) are forwarded from the run event stream.\n- On `timeout` / `error`, the payload includes `run_id` so clients can fetch status via `GET /agents/runs/{run_id}`." }, "402": { "content": { @@ -18443,6 +18729,170 @@ ] } }, + "/agents/{agent_id}/triggers/{trigger_id}/metadata-keys": { + "get": { + "description": "A trigger declares the run metadata keys its agent's steps may reference, each `required` or optional. `system` and `source` keys are written by the platform and are never required; `custom` keys are sent by the caller. A run you start is refused with HTTP 400 when it lacks a required key or sends a custom key the trigger does not declare. `trigger_id` is the one `GET /agents/{agent_id}` returns.\n\nAuth & scoping: requires `X-API-Key` header or OAuth Bearer token; the trigger must belong to an agent in the key's account.", + "operationId": "get_metadata_keys_api_api_agents__agent_id__triggers__trigger_id__metadata_keys_get", + "parameters": [ + { + "in": "path", + "name": "agent_id", + "required": true, + "schema": { + "format": "uuid", + "title": "Agent Id", + "type": "string" + } + }, + { + "in": "path", + "name": "trigger_id", + "required": true, + "schema": { + "format": "uuid", + "title": "Trigger Id", + "type": "string" + } + }, + { + "$ref": "#/components/parameters/X-Account-Id" + }, + { + "$ref": "#/components/parameters/Seclai-Version" + } + ], + "responses": { + "200": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/MetadataKeysResponse" + } + } + }, + "description": "Successful Response" + }, + "422": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + }, + "description": "Validation Error" + }, + "503": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ServiceUnavailableError" + } + } + }, + "description": "A dependency is unavailable. `database_unavailable` is temporary (for example routine maintenance) and carries a `Retry-After` header, after which the request can be retried unchanged. `vector_store_unavailable` means the vector store backing an embeddings operation is not configured; it carries no `Retry-After` because it does not clear without an operator.", + "headers": { + "Retry-After": { + "description": "Seconds to wait before retrying. Sent only with `database_unavailable`.", + "schema": { + "type": "integer" + } + } + } + } + }, + "summary": "Get the metadata keys a trigger declares", + "tags": [ + "agents" + ] + }, + "put": { + "description": "A trigger declares the run metadata keys its agent's steps may reference, each `required` or optional. `system` and `source` keys are written by the platform and are never required; `custom` keys are sent by the caller. A run you start is refused with HTTP 400 when it lacks a required key or sends a custom key the trigger does not declare. `trigger_id` is the one `GET /agents/{agent_id}` returns.\n\nAuth & scoping: requires `X-API-Key` header or OAuth Bearer token; the trigger must belong to an agent in the key's account.\n\nRejected with HTTP 400 when the agent references a key no trigger of its branch then declares.", + "operationId": "set_metadata_keys_api_api_agents__agent_id__triggers__trigger_id__metadata_keys_put", + "parameters": [ + { + "in": "path", + "name": "agent_id", + "required": true, + "schema": { + "format": "uuid", + "title": "Agent Id", + "type": "string" + } + }, + { + "in": "path", + "name": "trigger_id", + "required": true, + "schema": { + "format": "uuid", + "title": "Trigger Id", + "type": "string" + } + }, + { + "$ref": "#/components/parameters/X-Account-Id" + }, + { + "$ref": "#/components/parameters/Seclai-Version" + } + ], + "requestBody": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/SetMetadataKeysRequest" + } + } + }, + "required": true + }, + "responses": { + "200": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/MetadataKeysResponse" + } + } + }, + "description": "Successful Response" + }, + "422": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + }, + "description": "Validation Error" + }, + "503": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ServiceUnavailableError" + } + } + }, + "description": "A dependency is unavailable. `database_unavailable` is temporary (for example routine maintenance) and carries a `Retry-After` header, after which the request can be retried unchanged. `vector_store_unavailable` means the vector store backing an embeddings operation is not configured; it carries no `Retry-After` because it does not clear without an operator.", + "headers": { + "Retry-After": { + "description": "Seconds to wait before retrying. Sent only with `database_unavailable`.", + "schema": { + "type": "integer" + } + } + } + } + }, + "summary": "Declare the metadata keys a trigger accepts", + "tags": [ + "agents" + ] + } + }, "/agents/{agent_id}/triggers/{trigger_id}/slack-config": { "put": { "description": "Set the Slack connection, form, channels, visibility and conversation settings of an agent's SLACK_COMMAND trigger, which people run from Slack with `/seclai` or the message shortcut. Omitted fields are left unchanged.\n\nAuth & scoping: requires `X-API-Key` header or OAuth Bearer token; the trigger must belong to an agent in the key's account.", @@ -22982,6 +23432,73 @@ ] } }, + "/knowledge_bases/{knowledge_base_id}/metadata-key-stats": { + "get": { + "description": "How often each item-metadata key appears on the knowledge base's content items, for declaring a content trigger's metadata keys. An item is counted once, at its latest version; the total includes items with no metadata. Only keys a content run carries are listed: a null value counts as absent, and reserved keys and the keys the platform sets after the item's own are left out. Each window counts the items whose latest version was created or linked in it.\n\nAuth & scoping:\n- Requires `X-API-Key` header or OAuth Bearer token. You can only access knowledge bases belonging to your account.", + "operationId": "get_knowledge_base_metadata_key_stats_api_knowledge_bases__knowledge_base_id__metadata_key_stats_get", + "parameters": [ + { + "in": "path", + "name": "knowledge_base_id", + "required": true, + "schema": { + "title": "Knowledge Base Id", + "type": "string" + } + }, + { + "$ref": "#/components/parameters/X-Account-Id" + }, + { + "$ref": "#/components/parameters/Seclai-Version" + } + ], + "responses": { + "200": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/KnowledgeBaseMetadataKeyStatsResponse" + } + } + }, + "description": "Successful Response" + }, + "422": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + }, + "description": "Validation Error" + }, + "503": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ServiceUnavailableError" + } + } + }, + "description": "A dependency is unavailable. `database_unavailable` is temporary (for example routine maintenance) and carries a `Retry-After` header, after which the request can be retried unchanged. `vector_store_unavailable` means the vector store backing an embeddings operation is not configured; it carries no `Retry-After` because it does not clear without an operator.", + "headers": { + "Retry-After": { + "description": "Seconds to wait before retrying. Sent only with `database_unavailable`.", + "schema": { + "type": "integer" + } + } + } + } + }, + "summary": "Get Knowledge Base Metadata Key Stats", + "tags": [ + "knowledge_bases" + ] + } + }, "/me": { "get": { "description": "Returns the authenticated user's personal account ID and a list of organisations they belong to. Each organisation entry includes the organisation's own id, display name, and account_id. Useful for CLI tooling that needs to let the user pick an org context.", @@ -28897,8 +29414,10 @@ "2026-10-18", "2026-10-19", "2026-10-21", - "2026-10-28" + "2026-10-28", + "2026-11-02", + "2026-11-04" ], - "latest": "2026-10-28" + "latest": "2026-11-04" } } diff --git a/src/client.ts b/src/client.ts index 51cb2d2..bfadb8a 100644 --- a/src/client.ts +++ b/src/client.ts @@ -182,6 +182,9 @@ import type { ListAgentExamplesOptions, SetSlackTriggerConfigRequest, SlackTriggerConfigResponse, + SetMetadataKeysRequest, + MetadataKeysResponse, + KnowledgeBaseMetadataKeyStatsResponse, ListSolutionsOptions, ListMemoryBanksOptions, ListEvaluationResultsOptions, @@ -1195,16 +1198,67 @@ export class Seclai { * takes no `include_reasoning`, and one passed anyway is not sent: this method * returns no `stream_reasoning` events. * + * A `result` event carries the run's answer in `output` once + * `result_reported_at` is set, and can arrive while later steps still run. + * Pass `returnOnResult: true` to return that `result` snapshot as soon as it + * arrives, without waiting for `done`. Such a run may still be in progress: + * its `status` is not terminal and the run can still fail, so check + * `result_reported_at`, not `status`. + * + * If `timeoutMs` elapses, this throws {@link SeclaiStreamingError} with the + * latest snapshot in `run`: the `result` snapshot if one arrived, else the + * `init` snapshot. An abort from `signal` rethrows the abort error. + * + * By default, a stream that ends before `done` returns the latest snapshot + * when its status is not `pending`, so the run returned may not have + * finished: check `status` and `result_reported_at`. A stream that breaks + * throws the fetch error. Pass `throwOnIncomplete: true` to throw + * {@link SeclaiStreamingError} with the snapshot in `run` in both cases + * instead; it is recommended, and becomes the default in the next major + * version. + * * @param agentId - Agent identifier. * @param body - Streaming run request payload, without `include_reasoning`. - * @param opts - Timeout and abort signal options. - * @returns Final agent run payload from the `done` event. - * @throws {@link SeclaiStreamingError} If the stream ends before a `done` event. + * @param opts - Timeout, abort signal and `returnOnResult` options. + * @returns The run from the `done` event; with `returnOnResult`, the + * `result` snapshot; without `throwOnIncomplete`, possibly an unfinished + * snapshot (see above). + * @throws {@link SeclaiStreamingError} If `timeoutMs` elapses, or, with + * `throwOnIncomplete`, the stream ends or breaks, before the run finished. + * Its `run` holds the latest snapshot. + * + * @example + * ```ts + * try { + * const run = await client.runStreamingAgentAndWait("agent-id", { input: "Hello" }, { throwOnIncomplete: true }); + * console.log(run.output); + * } catch (err) { + * if (err instanceof SeclaiStreamingError && err.run?.result_reported_at) { + * console.log("Answer (run not finished):", err.run.output); + * } else throw err; + * } + * ``` */ async runStreamingAgentAndWait( agentId: string, body: Omit, - opts?: { timeoutMs?: number; signal?: AbortSignal }, + opts?: { + timeoutMs?: number; + signal?: AbortSignal; + /** + * Return the `result` snapshot as soon as a `result` event arrives (its + * answer is in `output`), without waiting for `done` (default: false). + */ + returnOnResult?: boolean; + /** + * Throw {@link SeclaiStreamingError}, with the latest snapshot in `run`, + * whenever the stream ends or breaks before `done`, or the server answers + * with plain JSON for a run that has not finished, instead of returning + * an unfinished run or throwing the fetch error (default: false). This + * becomes the default in the next major version. + */ + throwOnIncomplete?: boolean; + }, ): Promise { // Not sent even from untyped callers: the reasoning events are discarded here. const { include_reasoning: _ignored, ...runBody } = body as AgentRunStreamRequest; @@ -1228,6 +1282,7 @@ export class Seclai { const signal = anySignal([opts?.signal, timeoutController.signal]); let lastSeen: AgentRunResponse | undefined; + let reported: AgentRunResponse | undefined; try { const init: RequestInit = { method: "POST", headers, body: JSON.stringify(runBody) }; @@ -1263,7 +1318,16 @@ export class Seclai { // If server returned JSON instead of SSE if (isJson) { - return (await response.json()) as AgentRunResponse; + const run = (await response.json()) as AgentRunResponse; + const status = (run as any).status; + const finished = + status === "completed" || status === "failed" || status === "cancelled" || + Boolean(opts?.returnOnResult && run.result_reported_at); + if (opts?.throwOnIncomplete && !finished) { + const state = run.result_reported_at ? "A result had been reported; see `run`." : "No result had been reported."; + throw new SeclaiStreamingError(`The server answered with JSON for a run that has not finished. ${state}`, run.run_id, run); + } + return run; } if (!response.body) { @@ -1279,34 +1343,60 @@ export class Seclai { const parser = createSseParser((msg) => { if (!msg.data) return; - if (msg.event === "init" || msg.event === "done") { + if (msg.event === "init" || msg.event === "result" || msg.event === "done") { try { const parsed = JSON.parse(msg.data) as AgentRunResponse; lastSeen = parsed; + if (msg.event === "result") reported = parsed; if (msg.event === "done") final = parsed; } catch { /* ignore malformed JSON */ } } }); - while (!final) { - const { value, done } = await reader.read(); - if (done) break; - if (value) parser.feed(decoder.decode(value, { stream: true })); + let readFailure: string | undefined; + try { + while (!final && !(opts?.returnOnResult && reported)) { + const { value, done } = await reader.read(); + if (done) break; + if (value) parser.feed(decoder.decode(value, { stream: true })); + } + } catch (err) { + // The caller asked to stop: keep the abort error. Without + // throwOnIncomplete, a broken read throws the fetch error, as released. + if (opts?.signal?.aborted || (!opts?.throwOnIncomplete && !timedOut)) throw err; + readFailure = err instanceof Error ? err.message : String(err); } + // Flush the decoder, and dispatch an event whose complete lines were read + // but whose terminating blank line has not arrived. Text after the last + // newline is not parsed. + parser.feed(decoder.decode()); parser.end(); if (final) return final; - if (lastSeen && (lastSeen as any).status && (lastSeen as any).status !== "pending") { + if (opts?.returnOnResult && reported) { + reader.cancel().catch(() => {}); + return reported; + } + // Released behaviour without throwOnIncomplete: the stream ended early. + if (!opts?.throwOnIncomplete && !timedOut && lastSeen?.status && lastSeen.status !== "pending") { return lastSeen; } - throw new SeclaiStreamingError("Stream ended before receiving a 'done' event.", lastSeen?.run_id); + const run = reported ?? lastSeen; + const state = reported ? "A result had been reported; see `run`." : "No result had been reported."; + const reason = timedOut + ? `Timed out after ${timeoutMs}ms before the streaming agent run finished.` + : readFailure !== undefined + ? `Stream failed before the run finished: ${readFailure}.` + : "Stream ended before receiving a 'done' event."; + throw new SeclaiStreamingError(`${reason} ${state}`, run?.run_id, run); } catch (err) { - if (timedOut) { + if (timedOut && !opts?.signal?.aborted && !(err instanceof SeclaiStreamingError)) { throw new SeclaiStreamingError( `Timed out after ${timeoutMs}ms waiting for streaming agent run to complete.`, lastSeen?.run_id, + reported ?? lastSeen, ); } throw err; @@ -1329,6 +1419,10 @@ export class Seclai { * @example * ```ts * for await (const event of client.runStreamingAgent("agent-id", { input: "Hello!" })) { + * // `result` carries the answer in `output`, possibly before later steps finish. + * if (event.event === "result") { + * console.log("Answer:", (event.data as AgentRunResponse).output); + * } * if (event.event === "done") { * console.log("Final:", event.data); * } @@ -1456,11 +1550,38 @@ export class Seclai { * * This is useful in environments where SSE streaming is unavailable. * + * A run's answer can be ready before its last step finishes: once + * `result_reported_at` is set, `output` holds the answer. Pass + * `returnOnResult: true` to return as soon as a run has it set, the run + * `runAgent()` returns included. With `includeStepOutputs`, when that run + * would end the poll and `signal` is not aborted, it is fetched again at + * once for its step outputs. + * Such a run may still be in progress: its `status` is not terminal and the + * run can still fail, so check `result_reported_at`, not `status`. + * + * When `timeoutMs` elapses first, this throws {@link SeclaiStreamingError} + * with the latest run it has (the `runAgent()` response if no poll ran) in + * `run`. If that run has `result_reported_at` + * set, its `output` is the answer. + * + * `signal` is checked at the top of each poll, after the check for a run + * that ends the poll. A run that has ended the poll by then, whether the + * start run or a polled one, is returned even after an abort; the start run + * then comes back without step outputs, even when `includeStepOutputs` is + * set. Otherwise an abort throws `SeclaiError("Polling aborted.")`, which is + * not an `AbortError` and does not carry `signal.reason`. + * * @param agentId - Agent identifier. * @param body - Run request payload. * @param opts - Polling configuration and abort signal. - * @returns The terminal agent run. - * @throws {@link SeclaiStreamingError} On timeout. + * @returns The terminal agent run, or, with `returnOnResult`, the first run + * with `result_reported_at` set. + * @throws {@link SeclaiStreamingError} If `timeoutMs` elapses before the run + * finished. Its `run` holds the latest run it has (the `runAgent()` + * response if no poll ran). + * @throws {@link SeclaiError} `"Polling aborted."` If `signal` is aborted and, + * at the next poll's check, the run has not ended the poll. It is not an + * `AbortError` and does not carry `signal.reason`. */ async runAgentAndPoll( agentId: string, @@ -1472,6 +1593,12 @@ export class Seclai { timeoutMs?: number; /** Include per-step outputs in the final result. */ includeStepOutputs?: boolean; + /** + * Return as soon as a run, the `runAgent()` response included, has + * `result_reported_at` set (its answer is in `output`), without waiting + * for later steps to finish (default: false). + */ + returnOnResult?: boolean; signal?: AbortSignal; }, ): Promise { @@ -1483,21 +1610,36 @@ export class Seclai { const runId = (run as any).id ?? (run as any).run_id; if (!runId) throw new SeclaiError("Agent run response did not contain an id."); + // The initial response is checked too: a fast run can already be done. It + // carries no step outputs, so when they are asked for and it would end the + // poll, fetch the run first. + let current = run; + if (opts?.includeStepOutputs) { + const status = (run as any).status; + if (status === "completed" || status === "failed" || status === "cancelled" || + (opts.returnOnResult && run.result_reported_at)) { + // After an abort, skip the fetch: the loop returns the start response. + if (!opts.signal?.aborted) current = await this.getAgentRun(runId, { includeStepOutputs: true }); + } + } while (true) { + const status = (current as any).status; + if (status === "completed" || status === "failed" || status === "cancelled") { + return current; + } + if (opts?.returnOnResult && current.result_reported_at) return current; + if (opts?.signal?.aborted) throw new SeclaiError("Polling aborted."); if (Date.now() - startTime > timeout) { - throw new SeclaiStreamingError(`Polling timed out after ${timeout}ms.`, runId); + const state = current.result_reported_at ? "A result had been reported; see `run`." : "No result had been reported."; + throw new SeclaiStreamingError(`Polling timed out after ${timeout}ms. ${state}`, runId, current); } await new Promise((r) => setTimeout(r, pollInterval)); - const current = await this.getAgentRun(runId, + current = await this.getAgentRun(runId, opts?.includeStepOutputs ? { includeStepOutputs: true } : undefined, ); - const status = (current as any).status; - if (status === "completed" || status === "failed" || status === "cancelled") { - return current; - } } } @@ -1929,8 +2071,9 @@ export class Seclai { * Configure an agent's `SLACK_COMMAND` trigger, which people run from Slack * with `/seclai` or the message shortcut. * - * Sets the Slack connection, form, channels, reply visibility and conversation - * settings. Omitted fields are left unchanged. + * Sets the Slack connection, `alias` (the name people type after `/seclai`), + * form, channels, reply visibility and conversation settings. Omitted fields + * are left unchanged. `input` is deprecated and ignored by the API. * * @param agentId - Agent identifier. * @param triggerId - Trigger identifier. @@ -1970,6 +2113,46 @@ export class Seclai { })) as AcceptedAttachmentsResponse; } + /** + * Get the run metadata keys a trigger declares. + * + * Steps may reference a declared key as `{{metadata.}}`. The response + * also lists the system and source keys the trigger's type can declare. + * + * @param agentId - Agent identifier. + * @param triggerId - Trigger identifier, as `getAgent()` returns it. + * @returns The trigger's declared and declarable metadata keys. + */ + async getMetadataKeys(agentId: string, triggerId: string): Promise { + return (await this.request( + "GET", + `/agents/${agentId}/triggers/${triggerId}/metadata-keys`, + )) as MetadataKeysResponse; + } + + /** + * Replace the run metadata keys a trigger declares. + * + * The body is the whole declaration and replaces the current one. The API + * answers 400 when the agent references a key that no trigger of its branch + * declares. + * + * @param agentId - Agent identifier. + * @param triggerId - Trigger identifier, as `getAgent()` returns it. + * @param body - The full declaration; each key's `required` defaults to `false`. + * @returns The trigger's metadata keys, with `warnings` naming agents whose + * `call_agent` mapping no longer fits. + */ + async setMetadataKeys( + agentId: string, + triggerId: string, + body: SetMetadataKeysRequest, + ): Promise { + return (await this.request("PUT", `/agents/${agentId}/triggers/${triggerId}/metadata-keys`, { + json: body, + })) as MetadataKeysResponse; + } + // ═══════════════════════════════════════════════════════════════════════════ // Agent Email Governance // ═══════════════════════════════════════════════════════════════════════════ @@ -2157,6 +2340,22 @@ export class Seclai { return (await this.request("GET", `/knowledge_bases/${knowledgeBaseId}`)) as KnowledgeBaseResponse; } + /** + * Get the item-metadata keys seen in a knowledge base's content. + * + * Reports, for all time and the last 30 and 7 days, each key's item count, + * number of distinct values and five most frequent values. + * + * @param knowledgeBaseId - Knowledge base identifier. + * @returns The key statistics per window. + */ + async getKnowledgeBaseMetadataKeyStats(knowledgeBaseId: string): Promise { + return (await this.request( + "GET", + `/knowledge_bases/${knowledgeBaseId}/metadata-key-stats`, + )) as KnowledgeBaseMetadataKeyStatsResponse; + } + /** * Update a knowledge base. * diff --git a/src/errors.ts b/src/errors.ts index 36c1fb2..5d901f1 100644 --- a/src/errors.ts +++ b/src/errors.ts @@ -7,6 +7,8 @@ * handles every SDK error. Narrow to more specific subclasses for finer handling. */ +import type { AgentRunResponse } from "./types"; + /** Base error class for the Seclai SDK. */ export class SeclaiError extends Error { constructor(message: string) { @@ -79,14 +81,24 @@ export class SeclaiAPIValidationError extends SeclaiAPIStatusError { } } -/** Thrown when a streaming operation fails (e.g. stream ends unexpectedly). */ +/** + * Thrown when a streaming operation fails (e.g. stream ends unexpectedly), or + * when a run helper stops waiting before the run finished. + */ export class SeclaiStreamingError extends SeclaiError { /** The run ID associated with the failed stream, when available. */ public readonly runId: string | undefined; + /** + * Set by `runStreamingAgentAndWait()` and `runAgentAndPoll()` to the latest + * run they had, if any (when it has `result_reported_at` set, its `output` + * is the answer). The run may still be in progress, and can still fail. + */ + public readonly run: AgentRunResponse | undefined; - constructor(message: string, runId?: string) { + constructor(message: string, runId?: string, run?: AgentRunResponse) { super(message); this.name = "SeclaiStreamingError"; this.runId = runId; + this.run = run; } } diff --git a/src/index.ts b/src/index.ts index 5d0a673..fc1863c 100644 --- a/src/index.ts +++ b/src/index.ts @@ -121,6 +121,14 @@ export type { SlackTriggerScope, SlackReplyVisibility, + // Agent Trigger Metadata Keys + MetadataOrigin, + MetadataKeyDeclaration, + MetadataKeyDeclarationInput, + SetMetadataKeysRequest, + DeclarableMetadataKey, + MetadataKeysResponse, + // Agent Email Governance AgentEmailOptOutResponse, AgentEmailOptOutListResponse, @@ -217,6 +225,10 @@ export type { KnowledgeBaseResponse, CreateKnowledgeBaseBody, UpdateKnowledgeBaseBody, + KnowledgeBaseMetadataKeyStatsResponse, + MetadataKeyWindowResponse, + MetadataKeyCountResponse, + MetadataValueCountResponse, // Memory Banks MemoryBankListResponse, diff --git a/src/openapi.ts b/src/openapi.ts index 4e4835e..ae1b367 100644 --- a/src/openapi.ts +++ b/src/openapi.ts @@ -196,7 +196,8 @@ export interface paths { * * Accepts a partial update – only the fields included in the request body * are modified. Use this to toggle enabled/disabled, change the - * evaluation prompt, adjust retries, or rename the criteria. + * evaluation prompt, or rename the criteria. From Seclai-Version 2026-11-04 a + * changed `step_id` must name a `display_result` or `streaming_result` step. */ patch: operations["update_evaluation_criteria_api_agents_evaluation_criteria__criteria_id__patch"]; trace?: never; @@ -247,7 +248,8 @@ export interface paths { * * Use this endpoint to push results from external test harnesses, CI/CD * pipelines, or custom evaluation logic. Each result is linked to an - * agent run and optionally a specific step run. + * agent run and optionally a specific step run. From Seclai-Version 2026-11-04 + * `retry_triggered: true` returns 400: evaluations never retry a step. */ post: operations["create_evaluation_result_api_agents_evaluation_criteria__criteria_id__results_post"]; delete?: never; @@ -292,7 +294,7 @@ export interface paths { * Get Non Manual Evaluation Summary * @description Get an evaluation summary for API key clients. * - * Returns aggregated pass/fail/flagged counts and pass rates for each evaluation mode (eval_and_retry, sample_and_flag). + * Returns aggregated pass/fail/flagged counts and pass rates for each automatic evaluation mode. * * With ``2026-10-21`` or later, ``error`` results (the evaluator returned no * verdict) are left out of ``failed``, and the rates are over passed and @@ -530,9 +532,9 @@ export interface paths { * Update agent metadata * @description Update an agent's name, description, evaluation settings, model lifecycle settings and run limits. * - * Evaluation settings: `evaluation_mode` ('output_expectation', 'eval_and_retry', 'sample_and_flag'), `default_evaluation_tier` ('fast', 'balanced', 'thorough'), `max_retries`, `retry_on_failure`, `sampling_config`. + * Evaluation settings: `evaluation_mode` ('output_expectation', 'sample_and_flag'; see the field), `default_evaluation_tier` ('fast', 'balanced', 'thorough'), `sampling_config`. `max_retries` and `retry_on_failure` are accepted and have no effect. * - * Model lifecycle settings: `prompt_model_auto_upgrade_strategy` ('none', 'early_adopter', 'middle_of_road', 'cautious_adopter'), `prompt_model_auto_rollback_enabled`, `prompt_model_auto_rollback_triggers` (list of 'agent_eval_fail', 'governance_flag', 'governance_block', 'agent_run_failed'). + * Model lifecycle settings: `prompt_model_auto_upgrade_strategy` ('none', 'early_adopter', 'middle_of_road', 'cautious_adopter'), `prompt_model_auto_rollback_enabled`, `prompt_model_auto_rollback_triggers` (list of 'governance_flag', 'governance_block', 'agent_run_failed'; 'agent_eval_fail' never fires, and in the list keeps a run with a failed evaluation from being a rollback target). * * Run limits (each must be greater than 0; pass the paired `set_: true` with the value omitted to remove one): `max_credits_per_run`, `max_credits_per_period`, `max_runs_per_hour`, `max_runs_per_day`, `max_runs_total`. A runs-per-hour or per-day value above the plan's run quota is rejected with 400. * @@ -722,8 +724,7 @@ export interface paths { * - `prompt_call`: Call an LLM with a prompt template * - `retrieval`: Search a knowledge base * - `regex_replace`: Reshape text via ordered regex find/replace rules - * - `gate`: Evaluate conditions, stop or continue child execution - * - `retry`: Re-execute from a target ancestor step (for quality-control loops; pair with a `gate` step for conditional retrying. Fields: `target_step_id` (ancestor step ID), `max_retries` (1–10)) + * - `repeat_until`: Run a body (`body[]`) again until its `conditions` match the iteration's verdict or `max_iterations` (1–1000, default 3) is reached. Fields: `conditions`, `match`, optional `verdict_template`, `output` (`last` or `all`), `fail_when_max_reached` * - `evaluate_step`: Score a selected previous step output and emit JSON with `score`, `passed`, and `pass_threshold` (fields: `target_step_id`, `evaluation_prompt`, `pass_threshold`, optional `evaluation_tier`, optional `expectation_config`) * - `extract_data`: Progressively read and analyze large input * - `extract_content`: Extract structured data (JSON, HTML, XML) @@ -737,13 +738,13 @@ export interface paths { * - `write_content_attachment`: Write a file-backed attachment to content (optionally indexed for retrieval; content-triggered agents only. Fields: `attachment_key`, `content`, `content_type`, `indexed`) * - `load_content_attachment`: Load a previously written attachment (content-triggered agents only. Fields: `attachment_key`) * - `load_content`: Load the full text body of a source document (typically used with content-triggered agents; can also load by explicit `content_version_id`. Fields: `content_version_id` optional) - * - `streaming_result`: Stream LLM tokens in real-time via SSE (must be a direct child of `prompt_call`; requires `dynamic_input` or `template_input` trigger; `priority: true` enables real-time streaming). To stream **and** return a schema-validated payload, add a sibling branch `extract_content` (`expected_format: "json"` + `json_schema`) → `display_result` under the same `prompt_call` — a `display_result` takes precedence over the stream regardless of which finishes first, so its validated output becomes the run's result while the tokens act as a progress channel + * - `streaming_result`: Stream LLM tokens in real-time via SSE (must be a direct child of `prompt_call`; requires `dynamic_input` or `template_input` trigger; `priority: true` enables real-time streaming). An agent has at most one `streaming_result` or `display_result`; saving refuses a second * - `display_result`: Show output to the user * - `join`: Merge parallel branches * - `merge`: Combine multiple inputs into a single templated output * - `text`: Static text literal * - `for_each`: Iterate a body over a list of items (body lives in `body[]`) - * - `if_else`: Conditional dispatch. Evaluates `conditions` (same shape as `gate`) and runs `then_steps` on match, otherwise the optional `else_steps`. The chosen branch's output flows to the if_else step's own `child_steps` (post-branch continuation chain). **`display_result` and `streaming_result` are not allowed inside `then_steps` / `else_steps`** — end each branch with a content-producing step (e.g. `text`, `prompt_call`) and place the single `display_result` in `child_steps`. + * - `if_else`: Conditional dispatch. Evaluates `conditions` and runs `then_steps` on match, otherwise the optional `else_steps`. The chosen branch's output flows to the if_else step's own `child_steps` (post-branch continuation chain). **`display_result` and `streaming_result` are not allowed inside `then_steps` / `else_steps`** — end each branch with a content-producing step (e.g. `text`, `prompt_call`) and place the single `display_result` in `child_steps`. * - `switch`: Single-discriminator dispatch. Renders `discriminator` (default `{{input}}`) and routes to the first matching `cases[]` entry (equality by default; pass a list in `match` for `$in` semantics) or to `else_steps` when nothing matches. The chosen case's output flows to the switch step's own `child_steps`. **`display_result` and `streaming_result` are not allowed inside `cases[].steps` or `else_steps`** — end each case with a content-producing step and place the single `display_result` in `child_steps`. * * Auth & scoping: @@ -756,11 +757,11 @@ export interface paths { * * Uses **optimistic locking**: provide `expected_change_id` from the last `GET /agents/{agent_id}/definition`. Returns `409 Conflict` if the definition was modified since your last read. * - * The definition contains the agent's step workflow. Step types include `prompt_call`, `retrieval`, `regex_replace`, `gate`, `retry`, `evaluate_step`, `extract_data`, `extract_content`, `add_chat_turn`, `load_chat_history`, `add_memory`, `search_memory`, `load_memory`, `streaming_result`, `send_email`, `webhook_call`, `write_aws_s3_object`, `call_agent`, `write_metadata`, `write_content_attachment`, `load_content_attachment`, `load_content`, `display_result`, `join`, `merge`, `text`, `for_each`, `if_else`, and `switch`. Non-composite step types (`display_result`, `join`, `retry`, `streaming_result`) cannot contain child steps. + * The definition contains the agent's step workflow. Step types include `prompt_call`, `retrieval`, `regex_replace`, `evaluate_step`, `extract_data`, `extract_content`, `add_chat_turn`, `load_chat_history`, `add_memory`, `search_memory`, `load_memory`, `streaming_result`, `send_email`, `webhook_call`, `write_aws_s3_object`, `call_agent`, `write_metadata`, `write_content_attachment`, `load_content_attachment`, `load_content`, `display_result`, `join`, `merge`, `text`, `for_each`, `repeat_until`, `if_else`, and `switch`. Non-composite step types (`display_result`, `join`, `streaming_result`) cannot contain child steps. A definition holding a `gate` or `retry` step is rejected: use `if_else`, or `repeat_until` to run steps again until a condition holds. * - * **Retry steps** re-execute from a target ancestor step for quality-control loops. Configure with `target_step_id` (ancestor step ID) and `max_retries` (1–10). Best practice: place a `gate` step before the retry to make retries conditional. + * **repeat_until** runs its `body[]`, then tests its `conditions` against the iteration's verdict (`verdict_template`, else the iteration's value); it repeats until they match or `max_iterations` is reached. * - * **if_else** runs `then_steps` when its `conditions` (same shape as `gate`) match, otherwise its optional `else_steps`. Either branch's output flows to the if_else step's own `child_steps` (the post-branch continuation chain). + * **if_else** runs `then_steps` when its `conditions` match, otherwise its optional `else_steps`. Either branch's output flows to the if_else step's own `child_steps` (the post-branch continuation chain). * * **switch** dispatches on a `discriminator` template (default `{{input}}`) to the first matching case (equality by default; pass a list in `match` for `$in` semantics) or to `else_steps` when no case matches. Cases own their own `steps` subtrees; the chosen branch's output flows to the switch step's own `child_steps`. * @@ -850,8 +851,9 @@ export interface paths { * Create Evaluation Criteria * @description Create new step evaluation settings for an agent. * - * The evaluation mode, retry settings, and sample frequency are inherited - * from the agent and stored on the criteria row for historical reference. + * From Seclai-Version 2026-11-04, `step_id` must name a `display_result` or + * `streaming_result` step, the run's published result; another step returns 400. + * The evaluation mode and sample frequency are the agent's. */ post: operations["create_evaluation_criteria_api_agents__agent_id__evaluation_criteria_post"]; delete?: never; @@ -947,7 +949,7 @@ export interface paths { * The response contains the full definition, trigger configuration with schedules, alert configs, evaluation criteria, agent-scoped governance policies, and a resolved dependency manifest that maps every referenced external entity UUID to its human-readable name. * * Response shape: - * - `export_version`: schema version (currently `"5"`) + * - `export_version`: schema version (currently `"6"`) * - `exported_at`: ISO-8601 timestamp * - `agent`: name, description, schema_version, definition, timestamps * - `trigger`: trigger type, input template, schedules @@ -1033,10 +1035,10 @@ export interface paths { * - `input`: text input for agents with a `dynamic_input` trigger. * - `input_upload_id` / `input_upload_ids`: reference one or more files previously uploaded via `POST /agents/{agent_id}/upload-input`. Send them **with** `input` to pair prompt text with the files (the text leads, each file follows under its heading); only the two upload fields are mutually exclusive with each other. * - `priority`: set true for latency-sensitive, user-facing work. For agents with a `streaming_result` step, set `priority=true` to enable real-time token streaming; otherwise the run still proceeds, but without live token streaming. - * - `metadata`: a JSON object that becomes available to agent steps for string substitution. + * - `metadata`: a JSON object for string substitution. Each key must be one the trigger declares (`GET /agents/{agent_id}/triggers/{trigger_id}/metadata-keys`); a run that sends another custom key, or lacks a required one, is refused with HTTP 400. * * After starting: - * - Poll `GET /agents/runs/{run_id}` until `status` is `completed` or `failed`. + * - Poll `GET /agents/runs/{run_id}` until `result_reported_at` is set (the run's answer is in `output`) or `status` is `failed`; poll on until `status` is `completed` or `failed` if you need to know that every step finished. * - Use `include_step_outputs=true` to include per-step outputs, timing, and credits. * * Auth & scoping: @@ -1069,6 +1071,7 @@ export interface paths { * - Subsequent events are forwarded from the run event stream (status changes, step events, etc). * - If the agent contains a `streaming_result` step, `stream_token` events deliver individual LLM tokens (with a `token` field) and a `stream_end` event signals completion. * - With `include_reasoning=true`, `stream_reasoning` events (with a `text` field) carry the model's reasoning while it thinks, when the `streaming_result` step's author enabled `show_reasoning`. Reasoning is never part of `output`. + * - A `result` event contains the snapshot once the run has reported its result (`result_reported_at` is set and `output` is the answer). * - The final `done` event contains the terminal snapshot (including `output` and `credits` when available). * * Input options (for `dynamic_input` triggers): @@ -1078,7 +1081,7 @@ export interface paths { * Client guidance: * - Keep the connection open and handle keepalive comments. * - On `timeout` or `error`, the payload includes `run_id` so clients can resume by polling `GET /agents/runs/{run_id}`. - * - `stream_token` events carry the model's raw output. When the agent pairs `streaming_result` with a sibling `extract_content`/`display_result` branch, that branch's `display_result` takes precedence over the stream, so the `done` snapshot's `output` is the validated payload and the tokens are a progress channel — render tokens live, but read the result from `done`. Check `done.status` first: the streamed text is already the run's output when the stream ends, so a run that FAILED its validation step still carries that raw text in `output`. + * - `stream_token` events carry the model's raw output: render them live, but take the answer from the `result` event. `done.status` says whether the whole run succeeded, which can still fail after its result. * * Auth & scoping: * - Requires `X-API-Key` header or OAuth Bearer token. All resources are scoped to the caller's account. @@ -1165,6 +1168,36 @@ export interface paths { patch?: never; trace?: never; }; + "/agents/{agent_id}/triggers/{trigger_id}/metadata-keys": { + parameters: { + query?: never; + header?: never; + path?: never; + cookie?: never; + }; + /** + * Get the metadata keys a trigger declares + * @description A trigger declares the run metadata keys its agent's steps may reference, each `required` or optional. `system` and `source` keys are written by the platform and are never required; `custom` keys are sent by the caller. A run you start is refused with HTTP 400 when it lacks a required key or sends a custom key the trigger does not declare. `trigger_id` is the one `GET /agents/{agent_id}` returns. + * + * Auth & scoping: requires `X-API-Key` header or OAuth Bearer token; the trigger must belong to an agent in the key's account. + */ + get: operations["get_metadata_keys_api_api_agents__agent_id__triggers__trigger_id__metadata_keys_get"]; + /** + * Declare the metadata keys a trigger accepts + * @description A trigger declares the run metadata keys its agent's steps may reference, each `required` or optional. `system` and `source` keys are written by the platform and are never required; `custom` keys are sent by the caller. A run you start is refused with HTTP 400 when it lacks a required key or sends a custom key the trigger does not declare. `trigger_id` is the one `GET /agents/{agent_id}` returns. + * + * Auth & scoping: requires `X-API-Key` header or OAuth Bearer token; the trigger must belong to an agent in the key's account. + * + * Rejected with HTTP 400 when the agent references a key no trigger of its branch then declares. + */ + put: operations["set_metadata_keys_api_api_agents__agent_id__triggers__trigger_id__metadata_keys_put"]; + post?: never; + delete?: never; + options?: never; + head?: never; + patch?: never; + trace?: never; + }; "/agents/{agent_id}/triggers/{trigger_id}/slack-config": { parameters: { query?: never; @@ -2467,6 +2500,29 @@ export interface paths { patch?: never; trace?: never; }; + "/knowledge_bases/{knowledge_base_id}/metadata-key-stats": { + parameters: { + query?: never; + header?: never; + path?: never; + cookie?: never; + }; + /** + * Get Knowledge Base Metadata Key Stats + * @description How often each item-metadata key appears on the knowledge base's content items, for declaring a content trigger's metadata keys. An item is counted once, at its latest version; the total includes items with no metadata. Only keys a content run carries are listed: a null value counts as absent, and reserved keys and the keys the platform sets after the item's own are left out. Each window counts the items whose latest version was created or linked in it. + * + * Auth & scoping: + * - Requires `X-API-Key` header or OAuth Bearer token. You can only access knowledge bases belonging to your account. + */ + get: operations["get_knowledge_base_metadata_key_stats_api_knowledge_bases__knowledge_base_id__metadata_key_stats_get"]; + put?: never; + post?: never; + delete?: never; + options?: never; + head?: never; + patch?: never; + trace?: never; + }; "/me": { parameters: { query?: never; @@ -4144,7 +4200,7 @@ export interface components { change_id: string; /** * Definition - * @description The agent definition containing name, description, tags, and step workflow tree. Step types include prompt_call, retrieval, regex_replace, gate, retry, evaluate_step, extract_data, extract_content, add_chat_turn, load_chat_history, add_memory, search_memory, load_memory, streaming_result, send_email, webhook_call, call_agent, write_metadata, write_content_attachment, load_content_attachment, load_content, display_result, merge, for_each, if_else, switch, and others. + * @description The agent definition containing name, description, tags, and step workflow tree. Step types include prompt_call, retrieval, regex_replace, evaluate_step, extract_data, extract_content, add_chat_turn, load_chat_history, add_memory, search_memory, load_memory, streaming_result, send_email, webhook_call, call_agent, write_metadata, write_content_attachment, load_content_attachment, load_content, display_result, merge, for_each, if_else, switch, and others. */ definition: { [key: string]: unknown; @@ -4247,7 +4303,7 @@ export interface components { }[] | null; /** * Export Version - * @description Schema version of the export format (currently "5"). + * @description Schema version of the export format (currently "6"). */ export_version: string; /** @@ -4355,7 +4411,7 @@ export interface components { input_upload_ids?: string[] | null; /** * Metadata - * @description Metadata to make available for string substitution expressions in agent tasks. + * @description Metadata for string substitution in agent tasks. Each key must be one the agent's trigger declares (or sets); a run that sends another custom key, or lacks a required one, is refused with HTTP 400. */ metadata?: { [key: string]: components["schemas"]["JsonValue"]; @@ -4406,7 +4462,7 @@ export interface components { flagged_policies?: components["schemas"]["routers__api__agents__GovernancePolicyRefResponse"][]; /** * Governance Input Status - * @description Result of the governance input evaluation: safe, blocked, skipped, or timed_out. + * @description Result of the governance input evaluation: pending, safe, blocked, skipped, timed_out, or error. */ governance_input_status?: string | null; /** @@ -4416,7 +4472,7 @@ export interface components { governance_input_wait_ms?: number | null; /** * Hitl Wait Ms - * @description Cumulative milliseconds the run was parked waiting for a human decision on a human_in_the_loop step. Subtracted from active duration in run-detail and duration-stats responses. + * @description Sum over the run's human_in_the_loop steps of each one's wait for a decision, in milliseconds. Waits that overlap are each counted in full, so the sum can exceed the run's duration. */ hitl_wait_ms?: number | null; /** @@ -4444,6 +4500,11 @@ export interface components { * @description Indicates if the run was treated as a priority execution. */ priority: boolean; + /** + * Result Reported At + * @description When the run reported success: for an agent whose `report_success` is `on_result`, once its result step published, or once its run started if it has no result step; else when the run completed. From then on `output` is the run's answer and never changes; stop polling here if you only need the answer. `status` still follows every step, so a run can report success and later fail. Null until the run reports, for a run that failed before reporting, and for runs that ended before this field existed. + */ + result_reported_at?: string | null; /** * Run Id * @description Unique identifier for the agent run. @@ -4468,7 +4529,7 @@ export interface components { trace_purged_at?: string | null; /** * Wait Ms - * @description Cumulative milliseconds the run was parked on standard-mode wait steps. Subtracted from active duration in run-detail and duration-stats responses, exactly like hitl_wait_ms. Priority waits block inline and are not counted here. + * @description Sum over the run's standard-mode wait steps of each one's wait, in milliseconds. Waits that overlap are each counted in full. Priority waits block inline and are not counted here. */ wait_ms?: number | null; }; @@ -4567,7 +4628,7 @@ export interface components { input_upload_ids?: string[] | null; /** * Metadata - * @description Metadata to make available for string substitution expressions in agent tasks. + * @description Metadata for string substitution in agent tasks. Each key must be one the agent's trigger declares (or sets); a run that sends another custom key, or lacks a required one, is refused with HTTP 400. */ metadata?: { [key: string]: components["schemas"]["JsonValue"]; @@ -4690,8 +4751,8 @@ export interface components { disabled_reason?: string | null; /** * Evaluation Mode - * @description Evaluation mode: output_expectation, eval_and_retry, or sample_and_flag. - * @default eval_and_retry + * @description Evaluation mode: output_expectation or sample_and_flag. + * @default sample_and_flag */ evaluation_mode: string; /** @@ -4716,7 +4777,7 @@ export interface components { max_credits_per_run?: number | null; /** * Max Retries - * @description Max retries for eval_and_retry mode. + * @description Deprecated: has no effect. * @default 3 */ max_retries: number; @@ -4748,7 +4809,7 @@ export interface components { prompt_model_auto_rollback_enabled: boolean; /** * Prompt Model Auto Rollback Triggers - * @description Failure signals that trigger rollback. Defaults to agent_eval_fail, governance_block, agent_run_failed when null. + * @description Failure signals that trigger rollback. Defaults to governance_block, agent_run_failed when null. */ prompt_model_auto_rollback_triggers?: string[] | null; /** @@ -4759,7 +4820,7 @@ export interface components { prompt_model_auto_upgrade_strategy: string; /** * Retry On Failure - * @description Whether to retry on evaluation failure. + * @description Deprecated: has no effect. * @default true */ retry_on_failure: boolean; @@ -5709,7 +5770,8 @@ export interface components { * CreateEvaluationCriteriaRequest * @description Request body for creating an evaluation criteria. * - * The evaluation mode, retry settings, and sample frequency are set at the + * step_id names the display_result or streaming_result step to score. The + * evaluation mode and sample frequency are set at the * agent level, not per-criteria. */ CreateEvaluationCriteriaRequest: { @@ -5758,11 +5820,13 @@ export interface components { flagged: boolean; /** * Retry Count + * @description Deprecated: has no effect on any count. * @default 0 */ retry_count: number; /** * Retry Triggered + * @description Retired; true returns 400 from Seclai-Version 2026-11-04. * @default false */ retry_triggered: boolean; @@ -6027,6 +6091,17 @@ export interface components { */ url_id?: string | null; }; + /** DeclarableMetadataKey */ + DeclarableMetadataKey: { + /** + * Always Present + * @description Whether every run carries this key. + */ + always_present: boolean; + /** Key */ + key: string; + origin: components["schemas"]["MetadataOrigin"]; + }; /** DmarcFailingSourceResponse */ DmarcFailingSourceResponse: { /** Failed Count */ @@ -6501,7 +6576,7 @@ export interface components { enabled: boolean; /** * Evaluation Mode - * @description Runtime behavior mode. output_expectation (manual validation), eval_and_retry (every run + retry), sample_and_flag (sampled monitoring). + * @description Runtime behavior mode. output_expectation (manual validation), sample_and_flag (sampled monitoring). */ evaluation_mode: string; /** Evaluation Prompt */ @@ -6517,7 +6592,10 @@ export interface components { * Format: uuid */ id: string; - /** Max Retries */ + /** + * Max Retries + * @description Deprecated: has no effect. + */ max_retries: number; /** * Pass Threshold @@ -6528,7 +6606,10 @@ export interface components { result_summary: { [key: string]: number; }; - /** Retry On Failure */ + /** + * Retry On Failure + * @description Deprecated: has no effect. + */ retry_on_failure: boolean; /** Step Id */ step_id: string | null; @@ -6587,9 +6668,15 @@ export interface components { * Format: uuid */ id: string; - /** Retry Count */ + /** + * Retry Count + * @description Deprecated. + */ retry_count: number; - /** Retry Triggered */ + /** + * Retry Triggered + * @description Deprecated. + */ retry_triggered: boolean; /** * Score @@ -6677,9 +6764,15 @@ export interface components { * Format: uuid */ id: string; - /** Retry Count */ + /** + * Retry Count + * @description Deprecated. + */ retry_count: number; - /** Retry Triggered */ + /** + * Retry Triggered + * @description Deprecated. + */ retry_triggered: boolean; /** * Score @@ -6977,7 +7070,7 @@ export interface components { step_id?: string | null; /** * Step Type - * @description The step type to generate config for (e.g. 'regex_replace', 'gate', 'text', 'prompt_call', 'retrieval'). + * @description The step type to generate config for (e.g. 'regex_replace', 'if_else', 'text', 'prompt_call', 'retrieval'). */ step_type: string; /** @@ -7322,6 +7415,13 @@ export interface components { */ total: number; }; + /** KnowledgeBaseMetadataKeyStats */ + KnowledgeBaseMetadataKeyStatsResponse: { + /** Knowledge Base Id */ + knowledge_base_id: string; + /** Windows */ + windows: components["schemas"]["MetadataKeyWindowResponse"][]; + }; /** * KnowledgeBase * @description Response model for a single knowledge base. @@ -7626,6 +7726,120 @@ export interface components { */ updated_at: string; }; + /** MetadataKeyCountResponse */ + MetadataKeyCountResponse: { + /** + * Distinct Values + * @description How many different values it takes. + */ + distinct_values: number; + /** + * Items + * @description Items in the window carrying the key. + */ + items: number; + /** + * Key + * @description The item-metadata key. + */ + key: string; + /** + * Top Values + * @description Its 5 most frequent values, most frequent first. + */ + top_values: components["schemas"]["MetadataValueCountResponse"][]; + }; + /** MetadataKeyDeclaration */ + MetadataKeyDeclaration: { + /** + * Key + * @description Steps reference the key as ``{{metadata.}}``. + */ + key: string; + /** @description ``system`` and ``source`` keys are written by the platform; ``custom`` keys are sent by the caller or set on the trigger. */ + origin: components["schemas"]["MetadataOrigin"]; + /** + * Required + * @description A caller-started run without this key is refused. Only a ``custom`` key can be required. + * @default false + */ + required: boolean; + }; + /** MetadataKeyWindowResponse */ + MetadataKeyWindowResponse: { + /** + * Keys + * @description Keys seen in the window, most frequent first. + */ + keys: components["schemas"]["MetadataKeyCountResponse"][]; + /** + * Since + * @description The window's start; null for all time. + */ + since: string | null; + /** + * Total Items + * @description Items in the window. + */ + total_items: number; + /** + * Window + * @enum {string} + */ + window: "all_time" | "last_30_days" | "last_7_days"; + }; + /** MetadataKeysResponse */ + MetadataKeysResponse: { + /** + * Accepts Any Source Key + * @description True for content triggers, which may declare any item-metadata key. + */ + accepts_any_source_key: boolean; + /** + * Agent Id + * Format: uuid + */ + agent_id: string; + /** + * Declarable Keys + * @description The system and source keys this trigger's type can declare. + */ + declarable_keys: components["schemas"]["DeclarableMetadataKey"][]; + /** + * Metadata Keys + * @description The keys the agent's steps may reference. Static trigger metadata and Slack form fields are declared custom keys too, and are not repeated here. + */ + metadata_keys: components["schemas"]["MetadataKeyDeclaration"][]; + /** + * Trigger Id + * Format: uuid + */ + trigger_id: string; + /** + * Warnings + * @description Set by a PUT that changes the declaration: agents whose `call_agent` mapping no longer fits it. Always empty on GET. + */ + warnings?: string[]; + }; + /** + * MetadataOrigin + * @description Who writes a run metadata key (docs/architecture/run-metadata.md, Q2). + * @enum {string} + */ + MetadataOrigin: "system" | "source" | "custom"; + /** MetadataValueCountResponse */ + MetadataValueCountResponse: { + /** + * Items + * @description Items carrying this value. + */ + items: number; + /** + * Value + * @description The value as text, cut to 100 characters; an object is shown as {…} and a list as […]. + */ + value: string; + }; /** * ModalityRateResponse * @description Per-modality rate for an LLM that prices image/audio/video output @@ -7979,6 +8193,14 @@ export interface components { /** Mode */ mode: string; }; + /** SetMetadataKeysRequest */ + SetMetadataKeysRequest: { + /** + * Metadata Keys + * @description The whole declaration; replaces the trigger's current one. + */ + metadata_keys: components["schemas"]["MetadataKeyDeclaration"][]; + }; /** * SetSlackTriggerConfigApiRequest * @description Omitted fields are left unchanged. @@ -7986,6 +8208,11 @@ export interface components { SetSlackTriggerConfigApiRequest: { /** @description `everywhere` in scope, or `channel_opt_in`. */ activation?: components["schemas"]["SlackTriggerActivation"] | null; + /** + * Alias + * @description The name people type after `/seclai` to run this agent: lowercase letters, digits and hyphens, starting with a letter, unique in the Slack workspace. Generated from the agent's name when not set or empty. + */ + alias?: string | null; /** * Allow Dms * @description Runnable in the app's own DMs. @@ -7995,18 +8222,13 @@ export interface components { channel_ids?: string[] | null; /** * Form - * @description Form fields `{key, label, type, required, hint, choices, max_length}`. `type`: short_text, long_text, number, single_choice, multi_choice, yes_no, date, slack_user or slack_channel. Each value reaches the run as `{{metadata.}}`. + * @description Form fields `{key, label, type, required, hint, choices, max_length}`. `type`: short_text, long_text, number, single_choice, multi_choice, yes_no, date, slack_user or slack_channel. Each value reaches the run as `{{metadata.}}`, not in the run's input, which is the text after `/seclai ` or the message the shortcut was used on, and its files. */ form?: { [key: string]: unknown; }[] | null; /** Include Invoker Email */ include_invoker_email?: boolean | null; - /** - * Input - * @description The trigger's input template. Each `{{metadata.}}` in it must be a form field key or a platform key; it is checked together with `form`. - */ - input?: string | null; /** Max Turns */ max_turns?: number | null; /** @@ -8074,6 +8296,11 @@ export interface components { * Format: uuid */ agent_id: string; + /** + * Alias + * @description The name people type after `/seclai` to run this agent. + */ + alias?: string | null; /** Allow Dms */ allow_dms: boolean; /** Channel Ids */ @@ -8870,7 +9097,7 @@ export interface components { * UpdateEvaluationCriteriaRequest * @description Request body for updating an evaluation criteria. * - * Retry settings and sample frequency are set at the agent level. + * The evaluation mode and sample frequency are set at the agent level. */ UpdateEvaluationCriteriaRequest: { /** Description */ @@ -9451,7 +9678,7 @@ export interface components { } | null; /** * Evaluation Mode - * @description Evaluation mode: 'output_expectation', 'eval_and_retry', or 'sample_and_flag'. + * @description Evaluation mode: 'output_expectation' or 'sample_and_flag'. From Seclai-Version 2026-11-04 'eval_and_retry' returns 400; earlier versions store it as 'sample_and_flag'. */ evaluation_mode?: string | null; /** @@ -9466,7 +9693,7 @@ export interface components { max_credits_per_run?: number | null; /** * Max Retries - * @description Max retries for eval_and_retry mode (1-10). + * @description Deprecated: accepted and has no effect. */ max_retries?: number | null; /** @@ -9496,14 +9723,14 @@ export interface components { prompt_model_auto_rollback_enabled?: boolean | null; /** * Prompt Model Auto Rollback Triggers - * @description Failure signals that trigger rollback: agent_eval_fail, governance_flag, governance_block, agent_run_failed. + * @description Failure signals that trigger rollback: governance_flag, governance_block, agent_run_failed. agent_eval_fail is accepted and never fires; in the list, a run with a failed evaluation is never picked as the rollback target. */ prompt_model_auto_rollback_triggers?: string[] | null; /** @description Auto-upgrade strategy: none, early_adopter, middle_of_road, cautious_adopter. */ prompt_model_auto_upgrade_strategy?: components["schemas"]["PromptModelAutoUpgradeStrategy"] | null; /** * Retry On Failure - * @description Whether to retry on evaluation failure. + * @description Deprecated: accepted and has no effect. */ retry_on_failure?: boolean | null; /** @@ -13235,6 +13462,7 @@ export interface operations { * * SSE events: * - `event: init` — `data` is an `AgentRunResponse` snapshot (includes `run_id`). + * - `event: result` — `data` is an `AgentRunResponse` snapshot once the run reported success (see `result_reported_at`): its `output` is the run's answer. Sent before `done`; for an agent whose `report_success` is `on_result` it can arrive while later steps still run. * - `event: done` — `data` is the final `AgentRunResponse` snapshot (includes `output`, `credits`, etc). * - Other events (e.g. `status`, step events) are forwarded from the run event stream. * - On `timeout` / `error`, the payload includes `run_id` so clients can fetch status via `GET /agents/runs/{run_id}`. @@ -13443,6 +13671,106 @@ export interface operations { }; }; }; + get_metadata_keys_api_api_agents__agent_id__triggers__trigger_id__metadata_keys_get: { + parameters: { + query?: never; + header?: { + /** @description Target a different organization account (OAuth only). When omitted, the user's default account is used. Ignored for API key authentication — the key's account is always used. */ + "X-Account-Id"?: components["parameters"]["X-Account-Id"]; + /** @description Opt into dated, backward-incompatible API changes (format YYYY-MM-DD). When omitted, the account's pinned baseline version is used and responses keep their legacy shapes. Send a date on or after a change's release to adopt it — e.g. `2026-07-27` enables rejection of undeclared query parameters (422) and the canonical `{data, pagination}` envelope (pagination = `{page, limit, total, pages, has_next, has_prev}`) on every list endpoint that previously returned a bare array, a flat `{data, total, page, limit}`, a `{configs, total}`, or another per-resource key. */ + "Seclai-Version"?: components["parameters"]["Seclai-Version"]; + }; + path: { + agent_id: string; + trigger_id: string; + }; + cookie?: never; + }; + requestBody?: never; + responses: { + /** @description Successful Response */ + 200: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["MetadataKeysResponse"]; + }; + }; + /** @description Validation Error */ + 422: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["HTTPValidationError"]; + }; + }; + /** @description A dependency is unavailable. `database_unavailable` is temporary (for example routine maintenance) and carries a `Retry-After` header, after which the request can be retried unchanged. `vector_store_unavailable` means the vector store backing an embeddings operation is not configured; it carries no `Retry-After` because it does not clear without an operator. */ + 503: { + headers: { + /** @description Seconds to wait before retrying. Sent only with `database_unavailable`. */ + "Retry-After"?: number; + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["ServiceUnavailableError"]; + }; + }; + }; + }; + set_metadata_keys_api_api_agents__agent_id__triggers__trigger_id__metadata_keys_put: { + parameters: { + query?: never; + header?: { + /** @description Target a different organization account (OAuth only). When omitted, the user's default account is used. Ignored for API key authentication — the key's account is always used. */ + "X-Account-Id"?: components["parameters"]["X-Account-Id"]; + /** @description Opt into dated, backward-incompatible API changes (format YYYY-MM-DD). When omitted, the account's pinned baseline version is used and responses keep their legacy shapes. Send a date on or after a change's release to adopt it — e.g. `2026-07-27` enables rejection of undeclared query parameters (422) and the canonical `{data, pagination}` envelope (pagination = `{page, limit, total, pages, has_next, has_prev}`) on every list endpoint that previously returned a bare array, a flat `{data, total, page, limit}`, a `{configs, total}`, or another per-resource key. */ + "Seclai-Version"?: components["parameters"]["Seclai-Version"]; + }; + path: { + agent_id: string; + trigger_id: string; + }; + cookie?: never; + }; + requestBody: { + content: { + "application/json": components["schemas"]["SetMetadataKeysRequest"]; + }; + }; + responses: { + /** @description Successful Response */ + 200: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["MetadataKeysResponse"]; + }; + }; + /** @description Validation Error */ + 422: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["HTTPValidationError"]; + }; + }; + /** @description A dependency is unavailable. `database_unavailable` is temporary (for example routine maintenance) and carries a `Retry-After` header, after which the request can be retried unchanged. `vector_store_unavailable` means the vector store backing an embeddings operation is not configured; it carries no `Retry-After` because it does not clear without an operator. */ + 503: { + headers: { + /** @description Seconds to wait before retrying. Sent only with `database_unavailable`. */ + "Retry-After"?: number; + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["ServiceUnavailableError"]; + }; + }; + }; + }; set_slack_trigger_config_api_api_agents__agent_id__triggers__trigger_id__slack_config_put: { parameters: { query?: never; @@ -16398,6 +16726,53 @@ export interface operations { }; }; }; + get_knowledge_base_metadata_key_stats_api_knowledge_bases__knowledge_base_id__metadata_key_stats_get: { + parameters: { + query?: never; + header?: { + /** @description Target a different organization account (OAuth only). When omitted, the user's default account is used. Ignored for API key authentication — the key's account is always used. */ + "X-Account-Id"?: components["parameters"]["X-Account-Id"]; + /** @description Opt into dated, backward-incompatible API changes (format YYYY-MM-DD). When omitted, the account's pinned baseline version is used and responses keep their legacy shapes. Send a date on or after a change's release to adopt it — e.g. `2026-07-27` enables rejection of undeclared query parameters (422) and the canonical `{data, pagination}` envelope (pagination = `{page, limit, total, pages, has_next, has_prev}`) on every list endpoint that previously returned a bare array, a flat `{data, total, page, limit}`, a `{configs, total}`, or another per-resource key. */ + "Seclai-Version"?: components["parameters"]["Seclai-Version"]; + }; + path: { + knowledge_base_id: string; + }; + cookie?: never; + }; + requestBody?: never; + responses: { + /** @description Successful Response */ + 200: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["KnowledgeBaseMetadataKeyStatsResponse"]; + }; + }; + /** @description Validation Error */ + 422: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["HTTPValidationError"]; + }; + }; + /** @description A dependency is unavailable. `database_unavailable` is temporary (for example routine maintenance) and carries a `Retry-After` header, after which the request can be retried unchanged. `vector_store_unavailable` means the vector store backing an embeddings operation is not configured; it carries no `Retry-After` because it does not clear without an operator. */ + 503: { + headers: { + /** @description Seconds to wait before retrying. Sent only with `database_unavailable`. */ + "Retry-After"?: number; + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["ServiceUnavailableError"]; + }; + }; + }; + }; get_me_api_me_get: { parameters: { query?: never; diff --git a/src/types.ts b/src/types.ts index c520e8c..15b8b93 100644 --- a/src/types.ts +++ b/src/types.ts @@ -155,7 +155,15 @@ export type CreateAgentRequest = Omit< * default to `false` on the server; the generated schema marks them required * only because they declare a default. */ -export type UpdateAgentRequest = Partial; +export type UpdateAgentRequest = Omit< + Partial, + "max_retries" | "retry_on_failure" +> & { + /** @deprecated Has no effect: the API accepts it and ignores it. Evaluations never retry a step. */ + max_retries?: number | null; + /** @deprecated Has no effect: the API accepts it and ignores it. Evaluations never retry a step. */ + retry_on_failure?: boolean | null; +}; /** * Request body for updating an agent definition (steps, model, etc.). @@ -241,13 +249,53 @@ export type EmailTriggerConfigResponse = components["schemas"]["EmailTriggerConf /** * Request body for configuring a `SLACK_COMMAND` trigger. * - * Every field is optional, and an omitted field is left unchanged. + * Every field is optional, and an omitted field is left unchanged. `alias` is + * the name people type after `/seclai` to run the agent. */ -export type SetSlackTriggerConfigRequest = components["schemas"]["SetSlackTriggerConfigApiRequest"]; +export type SetSlackTriggerConfigRequest = components["schemas"]["SetSlackTriggerConfigApiRequest"] & { + /** + * @deprecated The API ignores `input`; use `alias` to choose the name people + * type after `/seclai`. A run's input is the text after `/seclai `, or + * the message the shortcut was used on. + */ + input?: string | null; +}; /** A `SLACK_COMMAND` trigger's Slack connection, form, channels and conversation settings. */ export type SlackTriggerConfigResponse = components["schemas"]["SlackTriggerConfigApiResponse"]; +// ─── Agent Trigger Metadata Keys ───────────────────────────────────────────── + +/** Who writes a run metadata key: `system`, `source` (both written by the platform) or `custom`. */ +export type MetadataOrigin = components["schemas"]["MetadataOrigin"]; + +/** + * One run metadata key a trigger declares, which steps reference as + * `{{metadata.}}`. + */ +export type MetadataKeyDeclaration = components["schemas"]["MetadataKeyDeclaration"]; + +/** + * A {@link MetadataKeyDeclaration} as sent to {@link Seclai.setMetadataKeys}: + * `required` defaults to `false` and can be omitted — the generated type marks + * it required only because the schema declares a default. + */ +export type MetadataKeyDeclarationInput = WithServerDefaults; + +/** + * Request body for {@link Seclai.setMetadataKeys}: the whole declaration, which + * replaces the trigger's current one. + */ +export type SetMetadataKeysRequest = Omit & { + metadata_keys: MetadataKeyDeclarationInput[]; +}; + +/** A system or source metadata key a trigger's type can declare. */ +export type DeclarableMetadataKey = components["schemas"]["DeclarableMetadataKey"]; + +/** The run metadata keys a trigger declares, and the ones its type could declare. */ +export type MetadataKeysResponse = components["schemas"]["MetadataKeysResponse"]; + /** Where a Slack trigger runs within its scope: `everywhere` or `channel_opt_in`. */ export type SlackTriggerActivation = components["schemas"]["SlackTriggerActivation"]; @@ -487,7 +535,18 @@ export type EvaluationResultResponse = components["schemas"]["EvaluationResultRe export type EvaluationResultListResponse = VersionedList; /** Request body for creating a manual evaluation result. */ -export type CreateEvaluationResultRequest = WithServerDefaults; +export type CreateEvaluationResultRequest = Omit< + WithServerDefaults, + "retry_count" | "retry_triggered" +> & { + /** @deprecated Has no effect on any count. */ + retry_count?: number; + /** + * @deprecated Retired: evaluations never retry a step. `true` returns HTTP 400 + * from `Seclai-Version` `2026-11-04`. + */ + retry_triggered?: boolean; +}; /** Summary of evaluation results for a criteria. */ export type EvaluationResultSummaryResponse = components["schemas"]["EvaluationResultSummaryResponse"]; @@ -571,6 +630,18 @@ export type CreateKnowledgeBaseBody = components["schemas"]["CreateKnowledgeBase /** Request body for updating a knowledge base. */ export type UpdateKnowledgeBaseBody = components["schemas"]["UpdateKnowledgeBaseBody"]; +/** Item-metadata keys seen in a knowledge base's content, per time window. */ +export type KnowledgeBaseMetadataKeyStatsResponse = components["schemas"]["KnowledgeBaseMetadataKeyStatsResponse"]; + +/** The item-metadata keys seen in one window (`all_time`, `last_30_days` or `last_7_days`). */ +export type MetadataKeyWindowResponse = components["schemas"]["MetadataKeyWindowResponse"]; + +/** One item-metadata key: how many items carry it, how many values it takes, and its top values. */ +export type MetadataKeyCountResponse = components["schemas"]["MetadataKeyCountResponse"]; + +/** One value of an item-metadata key and how many items carry it. */ +export type MetadataValueCountResponse = components["schemas"]["MetadataValueCountResponse"]; + // ─── Memory Banks ──────────────────────────────────────────────────────────── /** Paginated list of memory banks. */ diff --git a/src/versions.ts b/src/versions.ts index bbde5d8..efebbf4 100644 --- a/src/versions.ts +++ b/src/versions.ts @@ -23,10 +23,12 @@ export const SeclaiApiVersion = { V2026_10_19: "2026-10-19", V2026_10_21: "2026-10-21", V2026_10_28: "2026-10-28", + V2026_11_02: "2026-11-02", + V2026_11_04: "2026-11-04", /** Baseline applied to an unpinned, header-less caller. */ Default: "2026-07-01", /** Newest version known to this SDK release. May lag the server. */ - Latest: "2026-10-28", + Latest: "2026-11-04", } as const; /** diff --git a/tests/client.test.ts b/tests/client.test.ts index 28c84ba..94591fd 100644 --- a/tests/client.test.ts +++ b/tests/client.test.ts @@ -10,7 +10,7 @@ import { SeclaiStreamingError, SeclaiApiVersion, } from "../src/index"; -import type { SetAcceptedAttachmentsRequest } from "../src/index"; +import type { SetAcceptedAttachmentsRequest, SetMetadataKeysRequest, SetSlackTriggerConfigRequest } from "../src/index"; import { parseIni, isTokenValid } from "../src/auth"; type RecordedRequest = { @@ -480,6 +480,191 @@ describe("Streaming", () => { await client.runStreamingAgentAndWait("ag_1", body); expect(bodies).toEqual([{ input: "hi" }]); }); + + const reportedRun = { run_id: "run_1", status: "running", output: "answer", result_reported_at: "2026-11-05T00:00:00Z" }; + + const initRun = { run_id: "run_1", status: "running" }; + + test("runStreamingAgentAndWait with throwOnIncomplete throws with the init snapshot when the stream ends after init only", async () => { + const client = makeClient((req) => + makeSseResponse([`event: init\ndata: ${JSON.stringify(initRun)}\n\n`], { signal: req.signal }), + ); + const err = await client.runStreamingAgentAndWait("ag_1", { input: "hi" }, { timeoutMs: 5_000, throwOnIncomplete: true }).catch((e) => e); + expect(err).toBeInstanceOf(SeclaiStreamingError); + expect(err.run).toEqual(initRun); + expect(err.message).toContain("No result had been reported"); + }); + + test("runStreamingAgentAndWait with throwOnIncomplete throws with the result snapshot when the stream ends before done", async () => { + const client = makeClient((req) => + makeSseResponse( + [ + `event: init\ndata: ${JSON.stringify({ run_id: "run_1", status: "pending" })}\n\n`, + `event: result\ndata: ${JSON.stringify(reportedRun)}\n\n`, + ], + { signal: req.signal }, + ), + ); + const err = await client.runStreamingAgentAndWait("ag_1", { input: "hi" }, { timeoutMs: 5_000, throwOnIncomplete: true }).catch((e) => e); + expect(err).toBeInstanceOf(SeclaiStreamingError); + expect(err.run).toEqual(reportedRun); + expect(err.runId).toBe("run_1"); + expect(err.message).toContain("A result had been reported"); + }); + + test("runStreamingAgentAndWait with throwOnIncomplete throws with the result snapshot on timeout, dispatching a buffered event", async () => { + const encoder = new TextEncoder(); + const client = makeClient((req) => { + // Sends init and a result missing its blank line, then stays open until the timeout aborts it. + const stream = new ReadableStream({ + start(controller) { + controller.enqueue(encoder.encode(`event: init\ndata: ${JSON.stringify(initRun)}\n\n`)); + controller.enqueue(encoder.encode(`event: result\ndata: ${JSON.stringify(reportedRun)}\n`)); + req.signal?.addEventListener("abort", () => controller.error(new Error("aborted")), { once: true }); + }, + }); + return new Response(stream as any, { status: 200, headers: { "content-type": "text/event-stream" } }); + }); + const err = await client.runStreamingAgentAndWait("ag_1", { input: "hi" }, { timeoutMs: 20, throwOnIncomplete: true }).catch((e) => e); + expect(err).toBeInstanceOf(SeclaiStreamingError); + expect(err.message).toContain("Timed out after 20ms"); + expect(err.run).toEqual(reportedRun); + }); + + test("runStreamingAgentAndWait with throwOnIncomplete throws with the init snapshot when the stream breaks", async () => { + const encoder = new TextEncoder(); + const client = makeClient(() => { + const stream = new ReadableStream({ + start(controller) { + controller.enqueue(encoder.encode(`event: init\ndata: ${JSON.stringify(initRun)}\n\n`)); + setTimeout(() => controller.error(new TypeError("connection reset")), 5); + }, + }); + return new Response(stream as any, { status: 200, headers: { "content-type": "text/event-stream" } }); + }); + const err = await client.runStreamingAgentAndWait("ag_1", { input: "hi" }, { timeoutMs: 5_000, throwOnIncomplete: true }).catch((e) => e); + expect(err).toBeInstanceOf(SeclaiStreamingError); + expect(err.message).toContain("connection reset"); + expect(err.run).toEqual(initRun); + }); + + test("runStreamingAgentAndWait with throwOnIncomplete throws for a JSON reply whose run has not finished", async () => { + const client = makeClient(() => jsonResponse(reportedRun)); + const err = await client + .runStreamingAgentAndWait("ag_1", { input: "hi" }, { throwOnIncomplete: true }) + .catch((e) => e); + expect(err).toBeInstanceOf(SeclaiStreamingError); + expect(err.run).toEqual(reportedRun); + // returnOnResult accepts it, and without either option it is returned as released. + expect( + await client.runStreamingAgentAndWait("ag_1", { input: "hi" }, { throwOnIncomplete: true, returnOnResult: true }), + ).toEqual(reportedRun); + expect(await client.runStreamingAgentAndWait("ag_1", { input: "hi" })).toEqual(reportedRun); + }); + + test("runStreamingAgentAndWait with throwOnIncomplete returns a JSON reply whose run is completed or failed", async () => { + for (const status of ["completed", "failed"]) { + const run = { ...reportedRun, status }; + const client = makeClient(() => jsonResponse(run)); + expect(await client.runStreamingAgentAndWait("ag_1", { input: "hi" }, { throwOnIncomplete: true })).toEqual(run); + } + }); + + // Without throwOnIncomplete, each ending keeps its 1.8.0 behaviour. + test("runStreamingAgentAndWait by default returns a non-pending snapshot when the stream ends before done", async () => { + const client = makeClient((req) => + makeSseResponse([`event: init\ndata: ${JSON.stringify(initRun)}\n\n`], { signal: req.signal }), + ); + expect(await client.runStreamingAgentAndWait("ag_1", { input: "hi" }, { timeoutMs: 5_000 })).toEqual(initRun); + }); + + test("runStreamingAgentAndWait by default throws the fetch error when the stream breaks", async () => { + const encoder = new TextEncoder(); + const reset = new TypeError("connection reset"); + const client = makeClient(() => { + const stream = new ReadableStream({ + start(controller) { + controller.enqueue(encoder.encode(`event: init\ndata: ${JSON.stringify(initRun)}\n\n`)); + setTimeout(() => controller.error(reset), 5); + }, + }); + return new Response(stream as any, { status: 200, headers: { "content-type": "text/event-stream" } }); + }); + const err = await client.runStreamingAgentAndWait("ag_1", { input: "hi" }, { timeoutMs: 5_000 }).catch((e) => e); + expect(err).toBe(reset); + }); + + test("runStreamingAgentAndWait by default throws SeclaiStreamingError with the snapshot on timeout", async () => { + const encoder = new TextEncoder(); + const client = makeClient((req) => { + const stream = new ReadableStream({ + start(controller) { + controller.enqueue(encoder.encode(`event: init\ndata: ${JSON.stringify(initRun)}\n\n`)); + req.signal?.addEventListener("abort", () => controller.error(new Error("aborted")), { once: true }); + }, + }); + return new Response(stream as any, { status: 200, headers: { "content-type": "text/event-stream" } }); + }); + const err = await client.runStreamingAgentAndWait("ag_1", { input: "hi" }, { timeoutMs: 20 }).catch((e) => e); + expect(err).toBeInstanceOf(SeclaiStreamingError); + expect(err.runId).toBe("run_1"); + expect(err.run).toEqual(initRun); + }); + + test("runStreamingAgentAndWait rethrows a caller abort after a result", async () => { + const encoder = new TextEncoder(); + const caller = new AbortController(); + const client = makeClient((req) => { + const stream = new ReadableStream({ + start(controller) { + controller.enqueue(encoder.encode(`event: result\ndata: ${JSON.stringify(reportedRun)}\n\n`)); + req.signal?.addEventListener("abort", () => controller.error(new DOMException("aborted", "AbortError")), { once: true }); + setTimeout(() => caller.abort(), 5); + }, + }); + return new Response(stream as any, { status: 200, headers: { "content-type": "text/event-stream" } }); + }); + const err = await client + .runStreamingAgentAndWait("ag_1", { input: "hi" }, { timeoutMs: 5_000, signal: caller.signal }) + .catch((e) => e); + expect(err).not.toBeInstanceOf(SeclaiStreamingError); + expect(err.name).toBe("AbortError"); + }); + + test("runStreamingAgentAndWait with returnOnResult returns the result snapshot without waiting for done", async () => { + const encoder = new TextEncoder(); + let cancelled = false; + const client = makeClient(() => { + // Never sends done and never closes. + const stream = new ReadableStream({ + start(controller) { + controller.enqueue(encoder.encode(`event: init\ndata: ${JSON.stringify(initRun)}\n\n`)); + controller.enqueue(encoder.encode(`event: result\ndata: ${JSON.stringify(reportedRun)}\n\n`)); + }, + cancel() { + cancelled = true; + }, + }); + return new Response(stream as any, { status: 200, headers: { "content-type": "text/event-stream" } }); + }); + const result = await client.runStreamingAgentAndWait("ag_1", { input: "hi" }, { timeoutMs: 5_000, returnOnResult: true }); + expect(result).toEqual(reportedRun); + expect(cancelled).toBe(true); + }); + + test("runStreamingAgentAndWait returns done over an earlier result", async () => { + const done = { run_id: "run_1", status: "completed", output: "answer", credits: 1 }; + const client = makeClient((req) => + makeSseResponse( + [ + `event: result\ndata: ${JSON.stringify(reportedRun)}\n\n`, + `event: done\ndata: ${JSON.stringify(done)}\n\n`, + ], + { signal: req.signal }, + ), + ); + expect(await client.runStreamingAgentAndWait("ag_1", { input: "hi" })).toEqual(done); + }); }); // ───────────────────────────────────────────────────────────────────────────── @@ -508,6 +693,122 @@ describe("Polling", () => { }); expect((result as any).status).toBe("completed"); }); + + const reportedRun = { id: "run_1", status: "running", output: "answer", result_reported_at: "2026-11-05T00:00:00Z" }; + + test("runAgentAndPoll throws with the last polled run on timeout, even when it has result_reported_at", async () => { + const client = makeClient((req) => + req.method === "POST" ? jsonResponse({ id: "run_1", status: "pending" }) : jsonResponse(reportedRun), + ); + const err = await client + .runAgentAndPoll("ag_1", { input: "hi" }, { pollIntervalMs: 5, timeoutMs: 30 }) + .catch((e) => e); + expect(err).toBeInstanceOf(SeclaiStreamingError); + expect(err.run).toEqual(reportedRun); + expect(err.message).toContain("A result had been reported"); + }); + + test("runAgentAndPoll returns the initial runAgent() response without polling when it is already finished", async () => { + const completed = { ...reportedRun, status: "completed" }; + for (const [initial, opts] of [ + [reportedRun, { returnOnResult: true }], + [completed, {}], + ] as const) { + let gets = 0; + const client = makeClient((req) => { + if (req.method === "POST") return jsonResponse(initial); + gets++; + return jsonResponse(completed); + }); + const result = await client.runAgentAndPoll("ag_1", { input: "hi" }, { pollIntervalMs: 5, timeoutMs: 5_000, ...opts }); + expect(result).toEqual(initial); + expect(gets).toBe(0); + } + }); + + test("runAgentAndPoll with includeStepOutputs fetches the run before returning a finished initial response", async () => { + const completed = { ...reportedRun, status: "completed" }; + const withSteps = { ...completed, steps: [{ output: "step output" }] }; + const gets: string[] = []; + const client = makeClient((req) => { + if (req.method === "POST") return jsonResponse(completed); + gets.push(new URL(req.url).search); + return jsonResponse(withSteps); + }); + const result = await client.runAgentAndPoll("ag_1", { input: "hi" }, { + pollIntervalMs: 5_000, + timeoutMs: 1_000, + includeStepOutputs: true, + }); + expect(result).toEqual(withSteps); + expect(gets).toEqual(["?include_step_outputs=true"]); + }); + + test("runAgentAndPoll with includeStepOutputs sends no extra GET for a pending initial response", async () => { + let gets = 0; + const client = makeClient((req) => { + if (req.method === "POST") return jsonResponse({ id: "run_1", status: "pending" }); + gets++; + return jsonResponse({ id: "run_1", status: "running" }); + }); + await expect( + client.runAgentAndPoll("ag_1", { input: "hi" }, { pollIntervalMs: 50, timeoutMs: 20, includeStepOutputs: true }), + ).rejects.toBeInstanceOf(SeclaiStreamingError); + expect(gets).toBe(1); + }); + + test("runAgentAndPoll with includeStepOutputs returns a finished start run without fetching after an abort", async () => { + const caller = new AbortController(); + let gets = 0; + const client = makeClient((req) => { + if (req.method === "POST") { + caller.abort(); // while runAgent() is in flight + return jsonResponse({ id: "run_1", status: "completed" }); + } + gets++; + return jsonResponse({ id: "run_1", status: "completed" }); + }); + const result = await client.runAgentAndPoll("ag_1", { input: "hi" }, { includeStepOutputs: true, signal: caller.signal }); + expect(result).toEqual({ id: "run_1", status: "completed" }); + expect(gets).toBe(0); + }); + + test("runAgentAndPoll returns a finished start run when timeoutMs elapses during the start request", async () => { + const client = makeClient(async (req) => { + if (req.method === "POST") { + await new Promise((r) => setTimeout(r, 40)); + return jsonResponse({ id: "run_1", status: "completed" }); + } + throw new Error("unexpected GET"); + }); + const result = await client.runAgentAndPoll("ag_1", { input: "hi" }, { pollIntervalMs: 5, timeoutMs: 10 }); + expect(result).toEqual({ id: "run_1", status: "completed" }); + }); + + test("runAgentAndPoll still throws on timeout without result_reported_at", async () => { + const client = makeClient((req) => + jsonResponse({ id: "run_1", status: req.method === "POST" ? "pending" : "running", result_reported_at: null }), + ); + await expect( + client.runAgentAndPoll("ag_1", { input: "hi" }, { pollIntervalMs: 5, timeoutMs: 30 }), + ).rejects.toBeInstanceOf(SeclaiStreamingError); + }); + + test("runAgentAndPoll with returnOnResult returns once result_reported_at is set", async () => { + let gets = 0; + const client = makeClient((req) => { + if (req.method === "POST") return jsonResponse({ id: "run_1", status: "pending" }); + gets++; + return jsonResponse(gets === 1 ? reportedRun : { ...reportedRun, status: "completed" }); + }); + const result = await client.runAgentAndPoll("ag_1", { input: "hi" }, { + pollIntervalMs: 5, + timeoutMs: 5_000, + returnOnResult: true, + }); + expect(result).toEqual(reportedRun); + expect(gets).toBe(1); + }); }); // ───────────────────────────────────────────────────────────────────────────── @@ -540,6 +841,29 @@ describe("Knowledge Bases", () => { await client.getKnowledgeBase("kb_1"); }); + test("getKnowledgeBaseMetadataKeyStats sends GET /knowledge_bases/:id/metadata-key-stats", async () => { + const client = makeClient((req) => { + const u = new URL(req.url); + expect(req.method).toBe("GET"); + expect(u.pathname).toBe("/knowledge_bases/kb_1/metadata-key-stats"); + expect(u.search).toBe(""); + expect(req.bodyText).toBeUndefined(); + return jsonResponse({ + knowledge_base_id: "kb_1", + windows: [ + { + window: "all_time", + since: null, + total_items: 3, + keys: [{ key: "author", items: 2, distinct_values: 1, top_values: [{ value: "Ada", items: 2 }] }], + }, + ], + }); + }); + const stats = await client.getKnowledgeBaseMetadataKeyStats("kb_1"); + expect(stats.windows[0]!.keys[0]!.top_values[0]!.value).toBe("Ada"); + }); + test("updateKnowledgeBase sends PUT /knowledge_bases/:id", async () => { const client = makeClient((req) => { expect(req.method).toBe("PUT"); @@ -2471,6 +2795,73 @@ describe("Agent Email Triggers", () => { }); }); +describe("Agent Trigger Metadata Keys", () => { + const keysResponse = { + agent_id: "ag_1", + trigger_id: "tr_1", + accepts_any_source_key: false, + declarable_keys: [{ key: "email_from", origin: "system", always_present: true }], + metadata_keys: [{ key: "customer_id", origin: "custom", required: true }], + warnings: [], + }; + + test("getMetadataKeys sends GET /agents/:id/triggers/:tid/metadata-keys", async () => { + const client = makeClient((req) => { + const u = new URL(req.url); + expect(req.method).toBe("GET"); + expect(u.pathname).toBe("/agents/ag_1/triggers/tr_1/metadata-keys"); + expect(u.search).toBe(""); + expect(req.bodyText).toBeUndefined(); + return jsonResponse(keysResponse); + }); + const res = await client.getMetadataKeys("ag_1", "tr_1"); + expect(res.declarable_keys[0]!.origin).toBe("system"); + expect(res.metadata_keys[0]!.key).toBe("customer_id"); + }); + + test("setMetadataKeys sends PUT /agents/:id/triggers/:tid/metadata-keys with the declaration", async () => { + const bodies: unknown[] = []; + const client = makeClient((req) => { + const u = new URL(req.url); + expect(req.method).toBe("PUT"); + expect(u.pathname).toBe("/agents/ag_1/triggers/tr_1/metadata-keys"); + expect(u.search).toBe(""); + bodies.push(JSON.parse(req.bodyText!)); + return jsonResponse({ ...keysResponse, warnings: ["Caller agent no longer fits"] }); + }); + // `required` is server-defaulted, so a key without it must typecheck. + const res = await client.setMetadataKeys("ag_1", "tr_1", { + metadata_keys: [ + { key: "customer_id", origin: "custom", required: true }, + { key: "region", origin: "custom" }, + ], + }); + expect(res.warnings).toEqual(["Caller agent no longer fits"]); + // An empty declaration is sent as-is. + await client.setMetadataKeys("ag_1", "tr_1", { metadata_keys: [] }); + expect(bodies).toEqual([ + { + metadata_keys: [ + { key: "customer_id", origin: "custom", required: true }, + { key: "region", origin: "custom" }, + ], + }, + { metadata_keys: [] }, + ]); + }); + + test("setMetadataKeys takes a SetMetadataKeysRequest body with a non-custom origin", async () => { + const bodies: unknown[] = []; + const client = makeClient((req) => { + bodies.push(JSON.parse(req.bodyText!)); + return jsonResponse(keysResponse); + }); + const body: SetMetadataKeysRequest = { metadata_keys: [{ key: "source_url", origin: "source" }] }; + await client.setMetadataKeys("ag_1", "tr_1", body); + expect(bodies).toEqual([{ metadata_keys: [{ key: "source_url", origin: "source" }] }]); + }); +}); + describe("Agent Slack Triggers", () => { test("setSlackTriggerConfig sends PUT /agents/:id/triggers/:tid/slack-config", async () => { const client = makeClient((req) => { @@ -2523,6 +2914,25 @@ describe("Agent Slack Triggers", () => { }); await client.setSlackTriggerConfig("ag_1", "tr_1", { allow_dms: true }); }); + + test("setSlackTriggerConfig sends alias and returns it", async () => { + const client = makeClient((req) => { + expect(JSON.parse(req.bodyText!)).toEqual({ alias: "support" }); + return jsonResponse({ trigger_id: "tr_1", alias: "support" }); + }); + const config = await client.setSlackTriggerConfig("ag_1", "tr_1", { alias: "support" }); + expect(config.alias).toBe("support"); + }); + + test("SetSlackTriggerConfigRequest still accepts the deprecated input", async () => { + // Released in 1.8.0; the API now ignores it, but callers passing it must keep compiling. + const body: SetSlackTriggerConfigRequest = { input: "{{slack.text}}", alias: "support" }; + const client = makeClient((req) => { + expect(JSON.parse(req.bodyText!)).toEqual({ input: "{{slack.text}}", alias: "support" }); + return jsonResponse({ trigger_id: "tr_1", alias: "support" }); + }); + await client.setSlackTriggerConfig("ag_1", "tr_1", body); + }); }); describe("Agent Examples", () => { @@ -3159,7 +3569,7 @@ describe("API version constants", () => { test("a constant reaches the wire", async () => { const client = makeClient( (req) => { - expect(req.headers["seclai-version"]).toBe("2026-10-28"); + expect(req.headers["seclai-version"]).toBe("2026-11-04"); return jsonResponse({ data: [] }); }, { apiVersion: SeclaiApiVersion.Latest }, @@ -4053,11 +4463,13 @@ describe("Request types", () => { const ref = `components["schemas"]["${name}"]`; if (!types.includes(ref)) continue; // not exported as a request type if (types.includes(`Partial<${ref}>`)) continue; + // Keys an outer Omit, "key" | …> drops are declared + // again by hand, as optional. const wrapped = types.match( - new RegExp(`WithServerDefaults<${ref.replace(/[[\]"]/g, "\\$&")},\\s*([^>]+)>`), + new RegExp(`WithServerDefaults<${ref.replace(/[[\]"]/g, "\\$&")},\\s*([^>]+)>(?:,\\s*([^>]+)>)?`), ); for (const key of defaulted) { - if (!wrapped || !wrapped[1].includes(`"${key}"`)) missing.push(`${name}.${key}`); + if (!wrapped || !`${wrapped[1]} ${wrapped[2] ?? ""}`.includes(`"${key}"`)) missing.push(`${name}.${key}`); } } expect(missing).toEqual([]);