Skip to content

Stop ask shows as feedback, not a hook error - #845

Open
anirudhkumar-nanonets wants to merge 3 commits into
mainfrom
stop-ask-as-feedback
Open

anirudhkumar-nanonets wants to merge 3 commits into
mainfrom
stop-ask-as-feedback

Conversation

@anirudhkumar-nanonets

@anirudhkumar-nanonets anirudhkumar-nanonets commented Nov 17, 2026 •

Copy link
Copy Markdown
Collaborator

The issue

When trail's Stop hook asks Claude Code to save a learning, Claude Code shows it as an error: a red Ran 1 stop hook row, ⎿ Stop hook error: [trail] …, and the footer notice "Stop hook error occurred · ctrl+o to see". A teammate saw this on 3 Nov and read it as trail crashing. The ask was working. The label was wrong.

Why

Claude Code labels every Stop decision: block, and every exit 2, as a hook error. Since 2.1.163 a Stop hook can instead return hookSpecificOutput.additionalContext. The changelog says it gives Claude feedback and keeps the turn going "without being labeled a hook error". The person sees a gold ⎿ Stop hook feedback: … row and no error notice. The model reads "Stop hook additional context: …" where it read "Stop hook blocking error from command: …".

The continuation works the same way. Claude Code puts a Stop's additionalContext on the same list as a block's reason, and that list drives the next turn with stop_hook_active: true. trail's never-ask-twice-in-a-row guard and note_asked still fire on the stop after. The cap of 8 hook continuations in a row counts both forms.

Older Claude Code builds drop a Stop's additionalContext and end the turn, so sending it everywhere would lose the ask there.

The fix

  • Feedback where Claude Code takes it. stopTakesContext (src/claude/stop.ts) is true only for Claude Code's main agent at 2.1.163 or later, outside a bare run (CLAUDE_CODE_SIMPLE). The Stop payload carries no version, so trail reads the newest version in the last 256 KB of the transcript. When it's true, emitHook sends the ask as hookSpecificOutput.additionalContext.
  • The block everywhere else. Older or unknown builds, subagents, Codex, Copilot CLI and Cursor all keep decision: block, which each of them acts on. The skills offer is also a Stop block, so it gets the same treatment on Claude Code.
  • Reading the answer. askAnswer (src/claude/note-signals.ts) finds the ask in either form: the old isMeta "Stop hook feedback" line, or the hook_additional_context attachment that additionalContext leaves in the transcript.
  • Measuring it. note_asked gains form (block/feedback), so the save rate can be compared by form. TELEMETRY.md and the telemetry contract list the new property.
  • Adds a repo learning on why the ask goes out in two forms.

Checks

Updated 18 Nov, after merging main into the branch (49d36b7c: #727, plus #757, #842, #844 and #847). The merge had no conflicts.

#727 and this PR. Both edit stop.ts, contract.ts and TELEMETRY.md, in different places. #727's "did it help?" vote is marked in turnEnd, from last_assistant_message, before the save ask runs, and the next prompt settles it. It reads neither form of the ask, and both forms continue the turn the same way, so which form the ask takes doesn't change a waiting vote. #727 adds its own event, learning_rated. note_asked keeps kind and saved, plus this PR's form.

  • Typecheck clean.
  • Tests: the Stop, hook, note-ask and telemetry contract suites cover the version gate, each host, a subagent, a bare run, both answer forms and the form property. After the merge, those suites plus the learning vote, skills telemetry, session start, learning-on-touch and scenario-hook suites pass 103/103 with a scratch HOME.
  • Full suite with a scratch HOME: 4,538 pass, 5 fail, 9 skipped. None of the 5 is in a file this PR touches, and the same 5 failed on a clean main in Ask once whether a reused learning helped #727's run on 17 Nov: three graft→trail move tests (graft-to-trail, hosts-trail, upkeep-hooks), and two init-first-minute tests that expect a name where this machine's git user prints "you".
  • Real transcripts, in place of the live check: this machine's Claude Code transcripts hold 151 trail Stop blocks, from Claude Code 2.1.286 to 2.1.326. For the 144 that have their "Stop hook feedback" line, each transcript was cut where the next Stop ran and passed to this branch's askAnswer twice: once as written (block form), and once with that line rewritten as the hook_additional_context attachment (feedback form). Both forms gave the same answers: 62 save asks (11 saved in the repo, 10 personal, 41 not saved), and 82 other trail Stop blocks (skills offers, older ask wording) left out. The attachment's shape matches about 3,800 real ones from other hook events (UserPromptSubmit, PostToolUse, SessionStart, PreToolUse). None comes from a Stop hook yet, since nothing sent one before this change.
  • Not checked live: the claude CLI on the test machine is still signed out (loggedIn: false on 18 Nov), so no real Claude Code session ran this hook. The label and continuation behaviour come from Claude Code's changelog and its 2.1.317 build.

trail check flags seven conflicts. All seven are word matches, not reversals:

  • "Blaming claude -p when the prompt, Stop and PostToolUse hooks went quiet in a sandbox" (TELEMETRY.md, note-signals.ts). That learning is about how trail's hooks load in a sandbox, which this PR doesn't touch.
  • "A property on hook_ms for the Stop hook" and "A property on note_saved alone" (note-signals.ts, stop.ts, contract.ts). The new form property goes on note_asked, the event that learning chose instead of either.
  • New since Ask once whether a reused learning helped #727 merged, both in stop.ts: "Blocking the Stop to make Claude ask" keeps Ask once whether a reused learning helped #727's vote off the Stop. This PR adds no Stop ask; it changes how the existing save ask is sent. "Reading a yes anywhere in the message" is about reading the vote, and this PR reads no votes.

Moves: learnings_committed, through the save rate of note_asked (saved in repo/personal over all asks) split by the new form property. PostHog events in Metabase db 11, posthog_event_full_view. Value on 17 Nov, the day this PR opened: not read, because Metabase was unreachable from the VPN-less machine. On 16 Nov db 11 had no note_asked events, since #739, which adds the event, wasn't released (from #739's description).

🤖 Generated with Claude Code

@trailhq-graft

trailhq-graft Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

🌱 graft blast radius

2 areas changed → 10 areas can be affected. 36 dependent symbols, depth 2.
Tests: 1 area updated its tests.
Tag: @shhdwi — 12 of 12 areas · @qoole — Workspace CLI · @johnybradshaw — Hook I/O

flowchart TB
  A0(("Claude Session State<br/>17 symbols"))
  A1(("Onboarding Flow<br/>4 symbols"))
  A2(("Event Telemetry<br/>3 symbols"))
  A3(("Note Persistence<br/>3 symbols"))
  A4(("Cloud Integration<br/>3 symbols"))
  AX(("5 smaller areas<br/>6 symbols"))
  classDef reached fill:#D9EDF3,stroke:#3AA7C9,stroke-width:1.5px,color:#0E313C;
  class A0,A1,A2,A3,A4 reached;
  classDef tail fill:#EEF2F3,stroke:#9AA4A9,stroke-width:1px,color:#3A4247;
  class AX tail;
Loading
Can be affected Symbols Nearest hop Reached from
Claude Session State 17 src/claude/hooks.ts:L240-L245 handlePreEdit — calls, depth 1 Hook I/O, Telemetry Contract
Onboarding Flow 4 src/init/first-minute.ts:L1-L646 first-minute.ts — imports, depth 1 Telemetry Contract
Event Telemetry 3 src/telemetry/index.ts:L1-L18 index.ts — imports, depth 1 Telemetry Contract
Note Persistence 3 src/notes/memory-notes.ts:L1-L285 memory-notes.ts — imports, depth 1 Telemetry Contract
Cloud Integration 3 src/cloud/nudge.ts:L1-L508 nudge.ts — imports, depth 1 Telemetry Contract
MCP Server 2 src/mcp/server.ts:L1-L257 server.ts — imports, depth 2 Telemetry Contract
4 smaller areas 4 CLI Skills, CLI Entry Point, Learning Sessions, … see below
Who knows this code — 5 people across 12 areas
Area Who knows it
Telemetry Contract · changed @shhdwi — 1 commit, last 26d ago
Hook I/O · changed @shhdwi — 6 commits, last 22d ago · @johnybradshaw — 1 commit, last 9d ago
Claude Session State · affected @shhdwi — 27 commits, last 22d ago
Onboarding Flow · affected @shhdwi — 10 commits, last 22d ago
Event Telemetry · affected @shhdwi — 1 commit, last 26d ago · @afeddersen — 1 commit, last 3mo ago
Note Persistence · affected @shhdwi — 5 commits, last 10d ago · @Faeeth — 1 commit, last 10d ago
Cloud Integration · affected @shhdwi — 22 commits, last 22d ago
MCP Server · affected @shhdwi — 21 commits, last 22d ago
…4 further areas

Ownership is git history over each area's own files, weighted towards recent work (120-day half-life). Merge commits and bots are dropped, and you are dropped from your own PR. A name with no @ has no GitHub handle in its commit email — tag them by hand, or add a .mailmap entry. A suggestion from history, not a CODEOWNERS rule.

All 36 dependent symbols, grouped by area

Claude Session State — 17 symbols in 13 files

  • src/claude/hooks.ts:L240-L245 — handlePreEdit (calls, depth 1)
    244: emitHook('PreToolUse', { context: learned, notice });
  • src/claude/hooks.ts:L537-L627 — main (calls, depth 1)
    554: if (event === 'post-bash-fail') { emitHook('PostToolUseFailure', await postBashPart(input, dir, true)); return; }
  • src/claude/hooks.ts:L1-L628 — hooks.ts (imports, depth 1)
    13: import { emit, emitHook, payloadSessionId } from './hook-io.js';
  • src/claude/prompt-hook.ts:L261-L302 — handlePrompt (calls, depth 1)
    268: if (skills.block) return void emitHook('UserPromptSubmit', { block: skills.block });
  • src/claude/prompt-hook.ts:L271-L272 — say (calls, depth 1)
    272: emitHook('UserPromptSubmit', { context: [skills.context, txt].filter(Boolean).join('\n\n'), notice: [skills.notice, news ?? offer].filter(Boolean).join('\n') || undefined });
  • src/claude/session-metrics.ts:L1-L230 — session-metrics.ts (imports, depth 1)
    37: import type { AgentHost } from '../telemetry/contract.js';
  • src/claude/skills-wiring.ts:L1-L116 — skills-wiring.ts (imports, depth 1)
    16: import { hookMsBucket } from '../telemetry/contract.js';
  • src/claude/state.ts:L1-L170 — state.ts (imports, depth 1)
    10: import type { AgentHost } from '../telemetry/contract.js';
  • src/claude/format.ts:L1-L469 — format.ts (imports, depth 2)
    38: ctx: { ctxPct: number | null; notes?: number; folder?: string },
  • src/claude/learning-vote.ts:L1-L138 — learning-vote.ts (imports, depth 2)
    124: * `ctx` is the telemetry test seam (a scratch home and env).
  • src/claude/prompt-hook.ts:L1-L312 — prompt-hook.ts (imports, depth 2)
    22: import { CHILD_TIMEOUT_MS, emitHook, graftJsonWithin, payloadSessionId, withContextDirArg } from './hook-io.js';
  • src/claude/prompt-skills.ts:L1-L327 — prompt-skills.ts (imports, depth 2)
    80: const ctx = g.ctx;
  • src/claude/skill-use.ts:L1-L80 — skill-use.ts (imports, depth 2)
    13: import { payloadSessionId } from "./hook-io.js";
  • src/claude/stale-count-run.ts:L1-L104 — stale-count-run.ts (imports, depth 2)
  • src/claude/stats.ts:L1-L27 — stats.ts (imports, depth 2)
  • src/claude/statusline.ts:L1-L40 — statusline.ts (imports, depth 2)
  • src/claude/sync-run.ts:L1-L77 — sync-run.ts (imports, depth 2)

Onboarding Flow — 4 symbols in 4 files

  • src/init/first-minute.ts:L1-L646 — first-minute.ts (imports, depth 1)
    253: export function sourceOf(note: Note, ctx: { me?: string; fromTask?: ReadonlySet<string> } = {}): string {
  • src/init/keep.ts:L1-L387 — keep.ts (imports, depth 1)
    37: import { initKindCounts } from '../telemetry/contract.js';
  • src/init/run.ts:L1-L503 — run.ts (imports, depth 1)
    42: import { countBucket, durationBucket, initKindCounts } from '../telemetry/contract.js';
  • src/init/review-screen.ts:L1-L319 — review-screen.ts (imports, depth 2)

Event Telemetry — 3 symbols in 3 files

  • src/telemetry/index.ts:L1-L18 — index.ts (imports, depth 1)
    2: * Anonymous adoption telemetry. `TELEMETRY.md` is the public contract;
  • src/telemetry/sessions.ts:L1-L123 — sessions.ts (imports, depth 1)
    31: import { countBucket, savedTokensBucket, type AgentHost } from './contract.js';
  • src/telemetry/track.ts:L1-L180 — track.ts (imports, depth 1)
    84: function commonProps(ctx: TrackContext): Record<string, string> {

Note Persistence — 3 symbols in 3 files

  • src/notes/memory-notes.ts:L1-L285 — memory-notes.ts (imports, depth 1)
    18: import { countBucket } from "../telemetry/contract.js";
  • src/notes/pr-reviews.ts:L1-L577 — pr-reviews.ts (imports, depth 1)
    24: import { countBucket, durationBucket } from "../telemetry/contract.js";
  • src/notes/claude-export.ts:L1-L128 — claude-export.ts (imports, depth 2)

Cloud Integration — 3 symbols in 3 files

  • src/cloud/nudge.ts:L1-L508 — nudge.ts (imports, depth 1)
    29: import { durationBucket } from "../telemetry/contract.js";
  • src/cloud/lookup-activity.ts:L1-L60 — lookup-activity.ts (imports, depth 2)
  • src/cloud/pair.ts:L1-L368 — pair.ts (imports, depth 2)

MCP Server — 2 symbols in 2 files

  • src/mcp/server.ts:L1-L257 — server.ts (imports, depth 2)
  • src/mcp/tools.ts:L1-L352 — tools.ts (imports, depth 2)

CLI Skills — 1 symbol in 1 file

  • src/skills/cli-skills.ts:L1-L376 — cli-skills.ts (imports, depth 2)

CLI Entry Point — 1 symbol in 1 file

  • src/cli.ts:L1-L3499 — cli.ts (imports, depth 2)
    2266: ctx: {

Learning Sessions — 1 symbol in 1 file

  • src/learn/sessions.ts:L1-L310 — sessions.ts (imports, depth 1)
    27: import { countBucket, durationBucket } from "../telemetry/contract.js";

Workspace CLI — 1 symbol in 1 file

  • src/graph/workspace-cli.ts:L1-L201 — workspace-cli.ts (imports, depth 2)
Test signal per changed area — 1 ✓ · 1 –

Reached = a node under a test path has a resolved edge into the changed symbol. It undercounts anything called indirectly — through a CLI, a spawned process or a dynamic import — so read a low ratio as “look here”, never as a coverage gate.

  • – Telemetry Contract — 1 test file changed here: test/claude-hooks-skills.test.ts
  • ✓ Hook I/O — 5 of 8 reached · 3 test files changed here: test/claude-hooks-skills.test.ts, test/claude-stop.test.ts, test/trail-note-ask.test.ts
    • not reached: stopFeedback, atLeast, askForNote
64 test suites also reference this code

76 symbols, kept out of the diagram and the table so they cannot crowd out the areas a reviewer has to look at.

  • test/claude-copy-dashes.test.ts
  • test/claude-export.test.ts
  • test/claude-format.test.ts
  • test/claude-hook-timeouts.test.ts
  • test/claude-hooks-1b.test.ts
  • test/claude-hooks-perf.test.ts
  • test/claude-hooks.test.ts
  • test/claude-host-standdown.test.ts
  • test/claude-post-bash-walls.test.ts
  • test/claude-prompt-learnings.test.ts
  • test/claude-prompt-private.test.ts
  • test/claude-session-start-learnings.test.ts
  • test/claude-skills-journey-fixtures.ts
  • test/claude-start-skills-1b.test.ts
  • test/claude-state.test.ts
  • test/claude-statusline.test.ts
  • test/claude-stop-trace.test.ts
  • test/claude-tally.test.ts
  • test/hook-savings.test.ts
  • test/hosts-codex-hooks.test.ts
  • …44 more

⚠️ 3 changed files not in the graph (.trail/learnings/2026-11-16-trail-s-stop-ask-goes-out-anirudhkumar.md, .trail/learnings/2026-11-18-without-a-live-claude-code-run-anirudhkumar.md, TELEMETRY.md) — no parser claims the extension, or the index predates the file.

graft blast · origin/main...HEAD · depth 2 · 10 changed files

Open the interactive graph → — click an area to see its dependent symbols at file:line.

github-actions Bot added a commit that referenced this pull request Nov 17, 2026
github-actions Bot added a commit that referenced this pull request Nov 18, 2026
github-actions Bot added a commit that referenced this pull request Nov 18, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant