From 7abf5ee33dc72fee5efbdf94310f086baf0cb45f Mon Sep 17 00:00:00 2001 From: AFK Agent Date: Fri, 23 Oct 2026 17:22:13 -0400 Subject: [PATCH 1/2] fix(forge-friction): classify themes as operational vs shape-gap before routing to forge The friction analyzer surfaces tool errors from session telemetry, but almost all of them are operational noise (timeouts, rate limits, usage caps, hook blocks) rather than genuine workflow shape gaps that would make useful skills. This caused /forge-friction to propose skills for infrastructure issues that should be hooks or config changes. Add a 3-step pipeline: Filter, Classify, Route. - Step 1 (Filter): unchanged confidence-class filtering - Step 2 (Classify): each theme is tagged operational or shape-gap using explicit indicator lists (timeouts, rate limits, hook blocks, connection errors, model rejections, iteration caps = operational; recurring ad-hoc multi-step workarounds without skill coverage = shape-gap) - Step 3 (Route): operational themes get a one-line recommendation (hook/config/budget fix); only shape-gap themes proceed to /forge Dry-run mode now outputs two separate tables. Auto mode skips operational themes for forge. Interactive mode lists operational recommendations first (informational, no y/n), then shape-gap candidates (y/n to forge). --- skills/forge-friction/SKILL.md | 67 ++++++++++++++++++++++++++-------- 1 file changed, 52 insertions(+), 15 deletions(-) diff --git a/skills/forge-friction/SKILL.md b/skills/forge-friction/SKILL.md index a43ff0a..9f59f79 100644 --- a/skills/forge-friction/SKILL.md +++ b/skills/forge-friction/SKILL.md @@ -1,6 +1,6 @@ --- name: forge-friction -description: "Surface recurring friction from session telemetry and generate targeted skills. Use when the user runs /forge-friction." +description: "Surface recurring friction from session telemetry, classify as operational vs shape-gap, and generate targeted skills for shape-gap themes only. Use when the user runs /forge-friction." argument-hint: "[--dry-run] [--auto]" --- @@ -18,13 +18,21 @@ Tool-error categories carry a failure-class suffix. `:timeout`, `:truncated`, an ## Dry-run mode -If `$ARGUMENT` contains `--dry-run`: after the analyzer runs, identify themes using the filter criteria in "Your job" below, but instead of presenting them interactively, output a single markdown table: +If `$ARGUMENT` contains `--dry-run`: after the analyzer runs, identify, filter, and classify themes using the criteria in "Your job" below, but instead of presenting them interactively, output two sections: + +**Operational** (recommendations only — not skill candidates): + +| Category | Count | Recommendation | +|---|---|---| +| ... | N | PreToolUse hook / config change / budget adjustment / ... | + +**Shape-gap** (forge candidates): | Category | Count | Example friction_detail | |---|---|---| | ... | N | "truncated quote 1"; "truncated quote 2" | -One row per theme. Truncate each `friction_detail` to ~80 chars. Up to 3 examples per row (semicolon-separated in the cell). If no themes pass the filter, print "No themes in lookback window." +One row per theme. Truncate each `friction_detail` to ~80 chars. Up to 3 examples per row (semicolon-separated in the cell). If no themes pass the filter in either table, print "No themes in lookback window." After the table (or "no themes" line), print: @@ -38,30 +46,59 @@ If `$ARGUMENTS` contains `--auto`: skip all y/n prompts and auto-approve all qua ## Your job -Read through the friction categories and their examples. For each category with 3+ sessions, analyze the `friction_detail` strings to identify **recurring themes** — specific, repeated failure modes that a skill could address. +Read through the friction categories and their examples. For each category with 3+ sessions, analyze the `friction_detail` strings to identify **recurring themes** — specific, repeated failure modes. -Not all friction is fixable by a skill. Filter for themes where: +### Step 1 — Filter + +Not all friction is actionable. Filter for themes where: - The same failure mode repeats across multiple sessions (not one-off issues) -- A skill could change the agent's default behavior to avoid the friction -- The fix is a workflow shape change, not a reminder or checklist +- The failure mode is specific enough to act on (not a grab-bag of unrelated errors) - **Prefer high-confidence failure classes.** Pursue `:timeout` / `:truncated` / `:slow` categories first; they are real friction. Treat `:plain` and `:unclassified` categories as low-confidence — only pursue one if its `friction_detail` / `goal` examples make the recurring failure mode unmistakable. +### Step 2 — Classify + +For each theme that passes the filter, classify it as **operational** or **shape-gap**: + +**Operational** — infrastructure/tooling misbehavior, not a missing workflow. Indicators: +- Tool timeouts, slow tools, truncated output (`:timeout`, `:slow`, `:truncated` on any tool) +- Rate limits, usage limits, connection errors (`subagent_fail:RateLimitError`, `UsageLimitError`, `APIConnectionError`, `APIConnectionTimeoutError`) +- Hook blocks working as designed (`hook_block:*` categories) +- Session closures from external causes (`closure:abort`, `closure:timeout` from usage caps, network, or user signal) +- Permission errors, idle watchdog kills (`subagent_fail:PermissionDeniedError`, `IdleWatchdogError`) +- Model rejection errors (`subagent_fail:Error` from HTTP 400/404 on model ids) +- Iteration cap exhaustion from budget undersizing (`closure:iteration_cap`) + +**Shape-gap** — a recurring workflow pattern agents perform without skill coverage. Indicators: +- Multiple sessions performing the same multi-step sequence ad-hoc (visible in `goal` field) +- A structural problem agents solve differently each time with no consistent approach +- Goals that consistently `partially_achieved` or `aborted` because no skill owns the workflow shape — not because tools broke +- The `friction_detail` shows the agent improvising a multi-step workaround, not a single tool failing + +### Step 3 — Route + +**Operational themes:** For each, emit a one-line **recommendation** — the right fix category (PreToolUse hook, config change, budget adjustment, existing skill routing fix, or infrastructure improvement). Do NOT propose `/forge` for operational themes — they are not skill candidates. + +**Shape-gap themes:** These are the forge candidates. Route them through the interactive or auto flow below. + **If `--auto` is present** in `$ARGUMENTS`: -- Auto-forge only high-confidence themes. **Skip `:plain` and `:unclassified` tool-error categories** — without human review, auto-forging the low-confidence residue produces noise skills. Pursue `:timeout` / `:truncated` / `:slow` and non-tool-error categories that pass the filter. -- For each actionable theme, immediately run `/forge` from this plugin, seeding it with: "Create a skill that addresses this recurring friction: [theme summary]. Examples: [2-3 friction_detail quotes]." +- List operational themes with their recommendations (no forge). +- Auto-forge only shape-gap themes. **Skip `:plain` and `:unclassified` tool-error categories entirely in auto mode** — without human review, the low-confidence residue produces noise. +- For each shape-gap theme, immediately run `/forge` from this plugin, seeding it with: "Create a skill that addresses this recurring workflow gap: [theme summary]. Examples: [2-3 friction_detail quotes showing the ad-hoc workaround pattern]." - Chain the invocations sequentially (wait for forge + qualify to complete before starting the next one). - After each forge + qualify completes, log to telemetry (see telemetry format below). -- Do not ask for user confirmation; auto-approve all qualifying themes. +- Do not ask for user confirmation; auto-approve all qualifying shape-gap themes. **If `--auto` is not present** (interactive mode): -1. **For each actionable theme**: summarize it in one line with the count, then quote 2-3 representative `friction_detail` examples. Ask: "Generate a skill for this? (y/n)" +1. **Operational themes first**: list each with its one-line recommendation. No y/n prompt — these are informational. + +2. **Shape-gap themes next**: for each, summarize it in one line with the count, then quote 2-3 representative `friction_detail` examples. Ask: "Generate a skill for this? (y/n)" -2. **If the user says yes**: - - Run `/forge` from this plugin, seeding it with: "Create a skill that addresses this recurring friction: [theme summary]. Examples: [2-3 friction_detail quotes]." +3. **If the user says yes**: + - Run `/forge` from this plugin, seeding it with: "Create a skill that addresses this recurring workflow gap: [theme summary]. Examples: [2-3 friction_detail quotes showing the ad-hoc workaround pattern]." - After forge + qualify complete, log to telemetry (see telemetry format below). -3. **If the user says no**, skip it and move on. +4. **If the user says no**, skip it and move on. **Telemetry** (logged after each skill generation, same in both auto and interactive modes). @@ -90,4 +127,4 @@ Or adjust the lookback window: python3 "${PLUGIN_ROOT}/scripts/friction/analyzer.py" --days 60 ``` -After processing, summarize: how many themes found, how many the user approved, how many skills generated. +After processing, summarize: how many themes found, how many classified operational (with recommendations listed), how many classified shape-gap, how many approved for forge, how many skills generated. From d55a2533cdf6b3ddc6349ab95a9759f30f6a4e92 Mon Sep 17 00:00:00 2001 From: Griffin Long Date: Sat, 24 Oct 2026 14:40:38 -0400 Subject: [PATCH 2/2] fix(forge-friction): correct $ARGUMENT typo and remove skill-addressable language from Auto mode MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - F2: $ARGUMENT → $ARGUMENTS on line 21 (singular form silently disabled --dry-run gate) - F3: Auto mode intro scoped to Step 2 shape-gap classification pipeline, removing stale skill-addressable filter criteria --- skills/forge-friction/SKILL.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/skills/forge-friction/SKILL.md b/skills/forge-friction/SKILL.md index 9f59f79..17f231f 100644 --- a/skills/forge-friction/SKILL.md +++ b/skills/forge-friction/SKILL.md @@ -18,7 +18,7 @@ Tool-error categories carry a failure-class suffix. `:timeout`, `:truncated`, an ## Dry-run mode -If `$ARGUMENT` contains `--dry-run`: after the analyzer runs, identify, filter, and classify themes using the criteria in "Your job" below, but instead of presenting them interactively, output two sections: +If `$ARGUMENTS` contains `--dry-run`: after the analyzer runs, identify, filter, and classify themes using the criteria in "Your job" below, but instead of presenting them interactively, output two sections: **Operational** (recommendations only — not skill candidates): @@ -42,7 +42,7 @@ Stop. Do NOT proceed to the y/n prompts, `/forge` dispatch, or telemetry write d ## Auto mode -If `$ARGUMENTS` contains `--auto`: skip all y/n prompts and auto-approve all qualifying themes. For each theme that passes the filter criteria (3+ sessions, recurring failure mode, skill-addressable), immediately run `/forge` with the theme as a seed. Chain the invocations sequentially, then log telemetry for each generated skill. `--auto` respects `--dry-run` (if both flags are present, dry-run takes precedence and no /forge invocation or telemetry occurs). +If `$ARGUMENTS` contains `--auto`: skip all y/n prompts and auto-approve all qualifying themes. For each theme classified as shape-gap in Step 2, immediately run `/forge` with the theme as a seed. Chain the invocations sequentially, then log telemetry for each generated skill. `--auto` respects `--dry-run` (if both flags are present, dry-run takes precedence and no /forge invocation or telemetry occurs). ## Your job