From ee52596ae75a7c18f8581721a056dbc516c0fc19 Mon Sep 17 00:00:00 2001 From: BR <51544548+Bradenream@users.noreply.github.com> Date: Fri, 4 Sep 2026 13:57:56 -0400 Subject: [PATCH 1/3] docs: name the compile step the quickstart depends on (COR-13656) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A cold agent re-ran the quickstart end to end against the current CLI. It got a working, correctly-configured agent in 5.8 minutes and 50 commands, with one failed invocation — against ~45 minutes on the same task before this round of work. Everything it lost time to was in this file. 1. `vf environment compile` was missing from the workflow the quickstart names. `agent update` writes the definition; `compile` builds the form a conversation actually runs. Skip it and every signal says the change landed — the update returns `Agent updated.`, `vf agent get` reads your text back — while the runtime keeps answering as the template. Nothing errors. The causality was verified rather than assumed, by injecting a marker string: stored but absent from replies before compile, present after. This is the worst failure mode in the product because it returns a confident, plausible, wrong answer, and the paragraph listing the steps to escape the placeholder template omitted the one command that does it. The quickstart is now the three commands in order, with the trap named directly underneath. 2. "Either works immediately" was true and misleading. A new project does have a release, so `published` answers — but that release is a snapshot of the template, so it keeps saying Acme Corp until you publish your own. Test on draft, ship published, and you ship an Acme Corp agent. 3. The response envelope was described but never shown. "Replies arrive as text traces" is accurate and not enough: a typical turn returns five debug traces for one text trace, and a debug payload is a bare string, so the obvious recursive jq errors on it. Measured on a real response, then shown as a worked filter. Verified by following the edited quickstart verbatim on a fresh project: create, update, compile, publish, then converse. Both draft and published answer as the configured agent, with no placeholder text. The project was deleted afterwards. All three edits sit outside the Speakeasy-managed blocks, so regeneration will not undo them. --- README.md | 27 ++++++++++++++++++++++++--- 1 file changed, 24 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 1441aa8..98d2870 100644 --- a/README.md +++ b/README.md @@ -98,13 +98,34 @@ vf conversation send --user-id quickstart-user --project-id "$PROJECT_ID" \ --action '{"type":"text","payload":"What can you help me with?"}' --output-format json ``` -The agent's replies arrive as `text` traces in the response. +The agent's replies arrive as `text` traces in the response. Most traces are not replies — +a typical turn returns five `debug` traces for one `text` — and a `debug` payload is a bare +string, so a recursive filter errors on it. Select the type first: -A new project starts from a **template**, so it answers immediately but introduces itself as a placeholder brand ("Acme Corp support") and its instructions still contain fill-in-the-blank prompts. Getting to *your* agent is the next step, not the last one: edit the instructions (`vf agent update`), add knowledge (`vf document create-url`), run tests (`vf test run create`), and publish (`vf environment publish`). +```bash +... --output-format json | jq -r '.traces[] | select(.type=="text") | .payload.message' +``` + +A new project starts from a **template**, so it answers immediately but introduces itself as a placeholder brand ("Acme Corp support") and its instructions still contain fill-in-the-blank prompts. Getting to *your* agent is the next step, not the last one: + +```bash +vf agent update --project-id "$PROJECT_ID" --environment-alias main \ + --instructions 'You are the support assistant for ...' +vf environment compile --project-id "$PROJECT_ID" --environment-alias main +vf environment publish --project-id "$PROJECT_ID" --environment-alias main --name "v1" +``` + +**`vf environment compile` is required, and it is easy to miss.** `agent update` writes the +definition; `compile` builds the form a conversation actually runs. Without it every signal +says the change landed — the update returns `Agent updated.`, and `vf agent get` reads your +text back — while the runtime keeps answering as the template. Nothing errors. If your agent +still introduces itself as Acme Corp, this is why. + +From there: add knowledge (`vf document create-url`) and run tests (`vf test run create`). Worth knowing before you script against the CLI: -- **`--version-param draft` runs what you are editing; `published` runs the live version.** A new project is created with a first release already in place, so either works immediately — you do not have to publish anything first. +- **`--version-param draft` runs what you are editing; `published` runs the live version.** A new project is created with a first release already in place, so both work immediately — but that release is a snapshot of the **template**, so `published` keeps answering as Acme Corp until you publish your own. Test on `draft`, and publish before you rely on `published`. - **`--action '{"type":"launch"}'` is optional.** Sending a `text` action straight away works; the runtime starts the conversation itself. The launch step is shown because it makes the first turn explicit. - **Pass `--output-format json` explicitly when piping.** Inside AI coding agents (`CLAUDECODE`, `CURSOR_AGENT`, …) the default output is TOON, not JSON. - **Capture values with `--output-format json | jq -r`** — the built-in `--jq` flag emits JSON, so strings keep their quotes. From 245b055af1930d34939cb661dee18f0a3b667dd7 Mon Sep 17 00:00:00 2001 From: BR <51544548+Bradenream@users.noreply.github.com> Date: Fri, 4 Sep 2026 14:42:33 -0400 Subject: [PATCH 2/3] docs: fix three errors an adversarial review found in this PR (COR-13656) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A subagent reviewed this PR by running its claims against the live API rather than reading them. It found three errors, all mine, all verified independently before this fix. 1. The Acme diagnostic was false, and it was the worst kind of false. "If your agent still introduces itself as Acme Corp, this is why" pointed at the missing compile. But "Acme Corp" lives in `agent.prompt`, and this PR's own example only set `--instructions` — which never touches it. Confirmed by field scan on a fresh project: `agent.prompt` is the sole match. So a reader could follow the example, compile correctly, still see Acme Corp, and conclude that compile was broken. A diagnostic that fires after its own fix is worse than no diagnostic. The quickstart never mentioned `--prompt` at all. It now names both fields and what each one holds, and the example sets both. Verified end to end on a fresh project: zero occurrences of "Acme" in the configured agent afterwards. 2. `environment publish` compiles too, so the compile line was redundant exactly where this PR put it. Proven by injecting a marker, confirming six stale draft probes, then publishing with no explicit compile — the draft immediately served the marker. The snippet teaching the trap was therefore the one snippet in which the step does nothing. Compile now sits in the draft-testing loop, which is the loop the quickstart actually uses and the only place the step is yours to remember, and the text says publish compiles as well. 3. The trace hazard was described with the wrong mechanism. "A debug payload is a bare string, so a recursive filter errors on it" is wrong twice: every payload is an object, and the naive filter does not error — it silently returns the runtime's log lines interleaved with the reply ("Main Agent" starting execution, ai result, resources consumption). A quiet wrong answer, not a loud one, which is the more dangerous failure and the better reason to select the type. Measured: 5 debug, 2 block, 1 text on a typical turn; "five" is now "five to seven". Also from the same review: `vf test run create` requires a `--test-id` the quickstart never creates, so it is now `vf test create`; the template string is "Acme Corp", not "Acme Corp support"; and the jq example is a full runnable command rather than a fragment starting with an ellipsis. Verified by following the corrected quickstart verbatim on a fresh project: update both fields, compile, converse. The reply is the configured agent, the documented jq block runs as printed, and no "Acme" remains. Project deleted. --- README.md | 40 +++++++++++++++++++++++++++------------- 1 file changed, 27 insertions(+), 13 deletions(-) diff --git a/README.md b/README.md index 98d2870..cdc526f 100644 --- a/README.md +++ b/README.md @@ -98,30 +98,44 @@ vf conversation send --user-id quickstart-user --project-id "$PROJECT_ID" \ --action '{"type":"text","payload":"What can you help me with?"}' --output-format json ``` -The agent's replies arrive as `text` traces in the response. Most traces are not replies — -a typical turn returns five `debug` traces for one `text` — and a `debug` payload is a bare -string, so a recursive filter errors on it. Select the type first: +The agent's replies arrive as `text` traces in the response, alongside `debug`, `block` and +`choice` traces — a typical turn returns five to seven `debug` traces for one `text`. Select +the type; do not filter on `.message` alone. Debug payloads carry a `.message` too, so an +unselective filter does not fail, it quietly returns the runtime's log lines interleaved with +the reply: ```bash -... --output-format json | jq -r '.traces[] | select(.type=="text") | .payload.message' +vf conversation send --user-id quickstart-user --project-id "$PROJECT_ID" \ + --environment-alias main --version-param draft \ + --action '{"type":"text","payload":"What can you help me with?"}' \ + --output-format json | jq -r '.traces[] | select(.type=="text") | .payload.message' ``` -A new project starts from a **template**, so it answers immediately but introduces itself as a placeholder brand ("Acme Corp support") and its instructions still contain fill-in-the-blank prompts. Getting to *your* agent is the next step, not the last one: +A new project starts from a **template**, so it answers immediately — as someone else. Getting +to *your* agent means editing two separate fields: + +- **`--prompt`** is the global prompt, and it holds the persona. This is where the template's + `Acme Corp` brand lives. Changing `--instructions` alone will not remove it. +- **`--instructions`** is the turn-level behaviour, and the template leaves fill-in-the-blank + text there (`"Greet the user and offer help related to."`). ```bash vf agent update --project-id "$PROJECT_ID" --environment-alias main \ - --instructions 'You are the support assistant for ...' + --prompt 'You are the support assistant for Northwind Coffee.' \ + --instructions 'Help with subscriptions, delivery frequency and refunds.' + vf environment compile --project-id "$PROJECT_ID" --environment-alias main -vf environment publish --project-id "$PROJECT_ID" --environment-alias main --name "v1" ``` -**`vf environment compile` is required, and it is easy to miss.** `agent update` writes the -definition; `compile` builds the form a conversation actually runs. Without it every signal -says the change landed — the update returns `Agent updated.`, and `vf agent get` reads your -text back — while the runtime keeps answering as the template. Nothing errors. If your agent -still introduces itself as Acme Corp, this is why. +**That `compile` is required, and it is easy to miss.** `agent update` writes the definition; +`compile` builds the form a conversation actually runs. Without it every signal says the change +landed — the update returns `Agent updated.`, and `vf agent get` reads your text back — while +the runtime keeps answering as the template. Nothing errors, so there is nothing to search for. + +`vf environment publish` compiles as well, so a publish-then-test loop never hits this. It is +testing on `draft` — the loop the quickstart above uses — where the step is yours to remember. -From there: add knowledge (`vf document create-url`) and run tests (`vf test run create`). +From there: add knowledge (`vf document create-url`) and write tests (`vf test create`). Worth knowing before you script against the CLI: From 5a1bf4b7c926eade2f17a02f9f3c570c223c8ea8 Mon Sep 17 00:00:00 2001 From: BR <51544548+Bradenream@users.noreply.github.com> Date: Fri, 4 Sep 2026 15:12:17 -0400 Subject: [PATCH 3/3] docs: fix two more errors a second review found (COR-13656) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Second adversarial review, again by running the claims rather than reading them. Two real errors, both verified independently before this fix. 1. The piped example I added made a loud failure silent — in exactly the environment this README claims to serve. Run it with something else holding stdin (a CI step, a coding agent), and the CLI correctly refuses to wait forever — but `jq` succeeds on empty input, so the pipeline exits 0. Measured: the stdin error on stderr, no reply, and PIPELINE EXIT: 0. A scripted caller sees success and no answer. Unpiped commands in the same quickstart exit 1 here. So the one example this PR introduced was the one that swallowed its own failure. It now carries `