runtime: tool-call events and the built-in meta tools (request_tool, agent_feedback) - #34
Draft
chiplay wants to merge 2 commits into
Draft
runtime: tool-call events and the built-in meta tools (request_tool, agent_feedback)#34chiplay wants to merge 2 commits into
chiplay wants to merge 2 commits into
Conversation
A site that installs a tool layer has no way to see how agents use it: the calls happen entirely in the page, and the results go back to the agent. So every tool execution now dispatches a `sightkick:tool` CustomEvent on `document` — `phase: "start"` as the call begins, `phase: "end"` when it finishes, both carrying the same random `callId`. Both call paths funnel through `runTool`, so the span is opened there and the one fact it can't know — which surface the caller came from — is passed in as `RunOptions.via` (`"modelContext"` from boot's execute wrapper, `"call"` otherwise). Boot hands the emitter the IR name and whether `modelContext` is ours or the browser's when an IR loads. The detail carries argument KEY NAMES and never argument or result values: a tool call routinely holds what a user typed, and telemetry is exactly where that leaks. The error message is truncated to 200 chars for the same reason a stack trace is not telemetry. sightkick ships no analytics adapters and calls no endpoint. It dispatches the event; the page forwards it wherever it already sends things. Bundle cost: +839 bytes minified. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NpEUDK1MADpEDitJ3Px7D4 Signed-off-by: Chip Lay <chip.lay@gmail.com>
A tool layer only ever tells a site what it already decided to expose. The two
things it never hears are "the tool I needed wasn't here" and "that call didn't
do what I expected" — so a gap in the layer looks exactly like nobody using it.
A new optional `meta:` block in .sightkick/ turns on two built-in tools:
meta:
request_tool: true
agent_feedback: true
It compiles to an IR `meta: { requestTool, agentFeedback }` field and nothing
else — the tools reference no components and run no steps, so there is nothing
to compile. The runtime registers them itself, on every view: an agent hits the
gap they cover on whatever page it happens to be on, and "the tool I need isn't
here" is not a per-view fact. Both carry a JSON Schema so an agent sees typed
params rather than a free-text hole.
Each records what it is told as a `sightkick:meta` DOM event, on the same
channel as tool events and with the same bounds — every string trimmed and
capped, since an agent writing free text has no idea what a page will do with
it. sightkick stores nothing and calls no endpoint.
When agent_feedback is on, a FAILED tool result also gains a guidance
breadcrumb pointing at it: the moment an agent has something worth reporting is
the moment a call went wrong, and a tool nobody points at is a tool nobody
calls. request_tool needs no equivalent — it is listed on every view, so an
agent looking for a tool that doesn't exist finds it without a failure to hang
the nudge on.
The todo example enables both, so its golden IR carries the new field; the
search golden is unchanged, which is the check that `meta:` stays optional.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NpEUDK1MADpEDitJ3Px7D4
Signed-off-by: Chip Lay <chip.lay@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two additions to the runtime and the generator, independent of the Atlas work.
Tool-call events
Every tool execution, whether an agent reaches it through
document.modelContext(native or polyfilled) or throughwindow.__sightkick.call, now dispatches asightkick:toolCustomEventondocument: one withphase: "start"and one withphase: "end"sharing acallId. The end event carriesok,skipped,durationMs, and a truncatederrorwhen the call failed. The detail carries argument key names only, never argument or result values. A page forwards the event to whatever analytics it already runs; Sightkick ships no adapters, and the README says so.Bundle cost: +839 bytes minified.
Meta tools
A new optional
meta:key in the tool layer:compiles to
ir.metaand makes the runtime register two built-in tools on every view.request_tool(name, description, example_call?)is always listed, because an agent looking for a tool that does not exist never triggers a failure.agent_feedback(tool?, rating, note?)is also listed, and a failed tool result gains a guidance breadcrumb pointing at it. Both publish asightkick:metaevent with trimmed, length-capped fields, run no DOM steps, and emit nosightkick:tool. A user tool that shadows a built-in name is reported by the generator asmanifest.meta-shadow.examples/todoenables both, so its golden IR gains the field;search.ir.jsonis unchanged. The runtime bundle and its embedded copy are rebuilt and in sync. Two changesets, both minor.Verification
pnpm build,pnpm test(63 tests),pnpm typecheck;go build,go test ./...,go vet -tags e2e ./...; skills regenerated with no drift.🤖 Generated with Claude Code
https://claude.ai/code/session_01NpEUDK1MADpEDitJ3Px7D4
Generated by Claude Code