diff --git a/skills/forge-friction/SKILL.md b/skills/forge-friction/SKILL.md index a43ff0a..17f231f 100644 --- a/skills/forge-friction/SKILL.md +++ b/skills/forge-friction/SKILL.md @@ -1,6 +1,6 @@ --- name: forge-friction -description: "Surface recurring friction from session telemetry and generate targeted skills. Use when the user runs /forge-friction." +description: "Surface recurring friction from session telemetry, classify as operational vs shape-gap, and generate targeted skills for shape-gap themes only. Use when the user runs /forge-friction." argument-hint: "[--dry-run] [--auto]" --- @@ -18,13 +18,21 @@ Tool-error categories carry a failure-class suffix. `:timeout`, `:truncated`, an ## Dry-run mode -If `$ARGUMENT` contains `--dry-run`: after the analyzer runs, identify themes using the filter criteria in "Your job" below, but instead of presenting them interactively, output a single markdown table: +If `$ARGUMENTS` contains `--dry-run`: after the analyzer runs, identify, filter, and classify themes using the criteria in "Your job" below, but instead of presenting them interactively, output two sections: + +**Operational** (recommendations only — not skill candidates): + +| Category | Count | Recommendation | +|---|---|---| +| ... | N | PreToolUse hook / config change / budget adjustment / ... | + +**Shape-gap** (forge candidates): | Category | Count | Example friction_detail | |---|---|---| | ... | N | "truncated quote 1"; "truncated quote 2" | -One row per theme. Truncate each `friction_detail` to ~80 chars. Up to 3 examples per row (semicolon-separated in the cell). If no themes pass the filter, print "No themes in lookback window." +One row per theme. Truncate each `friction_detail` to ~80 chars. Up to 3 examples per row (semicolon-separated in the cell). If no themes pass the filter in either table, print "No themes in lookback window." After the table (or "no themes" line), print: @@ -34,34 +42,63 @@ Stop. Do NOT proceed to the y/n prompts, `/forge` dispatch, or telemetry write d ## Auto mode -If `$ARGUMENTS` contains `--auto`: skip all y/n prompts and auto-approve all qualifying themes. For each theme that passes the filter criteria (3+ sessions, recurring failure mode, skill-addressable), immediately run `/forge` with the theme as a seed. Chain the invocations sequentially, then log telemetry for each generated skill. `--auto` respects `--dry-run` (if both flags are present, dry-run takes precedence and no /forge invocation or telemetry occurs). +If `$ARGUMENTS` contains `--auto`: skip all y/n prompts and auto-approve all qualifying themes. For each theme classified as shape-gap in Step 2, immediately run `/forge` with the theme as a seed. Chain the invocations sequentially, then log telemetry for each generated skill. `--auto` respects `--dry-run` (if both flags are present, dry-run takes precedence and no /forge invocation or telemetry occurs). ## Your job -Read through the friction categories and their examples. For each category with 3+ sessions, analyze the `friction_detail` strings to identify **recurring themes** — specific, repeated failure modes that a skill could address. +Read through the friction categories and their examples. For each category with 3+ sessions, analyze the `friction_detail` strings to identify **recurring themes** — specific, repeated failure modes. -Not all friction is fixable by a skill. Filter for themes where: +### Step 1 — Filter + +Not all friction is actionable. Filter for themes where: - The same failure mode repeats across multiple sessions (not one-off issues) -- A skill could change the agent's default behavior to avoid the friction -- The fix is a workflow shape change, not a reminder or checklist +- The failure mode is specific enough to act on (not a grab-bag of unrelated errors) - **Prefer high-confidence failure classes.** Pursue `:timeout` / `:truncated` / `:slow` categories first; they are real friction. Treat `:plain` and `:unclassified` categories as low-confidence — only pursue one if its `friction_detail` / `goal` examples make the recurring failure mode unmistakable. +### Step 2 — Classify + +For each theme that passes the filter, classify it as **operational** or **shape-gap**: + +**Operational** — infrastructure/tooling misbehavior, not a missing workflow. Indicators: +- Tool timeouts, slow tools, truncated output (`:timeout`, `:slow`, `:truncated` on any tool) +- Rate limits, usage limits, connection errors (`subagent_fail:RateLimitError`, `UsageLimitError`, `APIConnectionError`, `APIConnectionTimeoutError`) +- Hook blocks working as designed (`hook_block:*` categories) +- Session closures from external causes (`closure:abort`, `closure:timeout` from usage caps, network, or user signal) +- Permission errors, idle watchdog kills (`subagent_fail:PermissionDeniedError`, `IdleWatchdogError`) +- Model rejection errors (`subagent_fail:Error` from HTTP 400/404 on model ids) +- Iteration cap exhaustion from budget undersizing (`closure:iteration_cap`) + +**Shape-gap** — a recurring workflow pattern agents perform without skill coverage. Indicators: +- Multiple sessions performing the same multi-step sequence ad-hoc (visible in `goal` field) +- A structural problem agents solve differently each time with no consistent approach +- Goals that consistently `partially_achieved` or `aborted` because no skill owns the workflow shape — not because tools broke +- The `friction_detail` shows the agent improvising a multi-step workaround, not a single tool failing + +### Step 3 — Route + +**Operational themes:** For each, emit a one-line **recommendation** — the right fix category (PreToolUse hook, config change, budget adjustment, existing skill routing fix, or infrastructure improvement). Do NOT propose `/forge` for operational themes — they are not skill candidates. + +**Shape-gap themes:** These are the forge candidates. Route them through the interactive or auto flow below. + **If `--auto` is present** in `$ARGUMENTS`: -- Auto-forge only high-confidence themes. **Skip `:plain` and `:unclassified` tool-error categories** — without human review, auto-forging the low-confidence residue produces noise skills. Pursue `:timeout` / `:truncated` / `:slow` and non-tool-error categories that pass the filter. -- For each actionable theme, immediately run `/forge` from this plugin, seeding it with: "Create a skill that addresses this recurring friction: [theme summary]. Examples: [2-3 friction_detail quotes]." +- List operational themes with their recommendations (no forge). +- Auto-forge only shape-gap themes. **Skip `:plain` and `:unclassified` tool-error categories entirely in auto mode** — without human review, the low-confidence residue produces noise. +- For each shape-gap theme, immediately run `/forge` from this plugin, seeding it with: "Create a skill that addresses this recurring workflow gap: [theme summary]. Examples: [2-3 friction_detail quotes showing the ad-hoc workaround pattern]." - Chain the invocations sequentially (wait for forge + qualify to complete before starting the next one). - After each forge + qualify completes, log to telemetry (see telemetry format below). -- Do not ask for user confirmation; auto-approve all qualifying themes. +- Do not ask for user confirmation; auto-approve all qualifying shape-gap themes. **If `--auto` is not present** (interactive mode): -1. **For each actionable theme**: summarize it in one line with the count, then quote 2-3 representative `friction_detail` examples. Ask: "Generate a skill for this? (y/n)" +1. **Operational themes first**: list each with its one-line recommendation. No y/n prompt — these are informational. + +2. **Shape-gap themes next**: for each, summarize it in one line with the count, then quote 2-3 representative `friction_detail` examples. Ask: "Generate a skill for this? (y/n)" -2. **If the user says yes**: - - Run `/forge` from this plugin, seeding it with: "Create a skill that addresses this recurring friction: [theme summary]. Examples: [2-3 friction_detail quotes]." +3. **If the user says yes**: + - Run `/forge` from this plugin, seeding it with: "Create a skill that addresses this recurring workflow gap: [theme summary]. Examples: [2-3 friction_detail quotes showing the ad-hoc workaround pattern]." - After forge + qualify complete, log to telemetry (see telemetry format below). -3. **If the user says no**, skip it and move on. +4. **If the user says no**, skip it and move on. **Telemetry** (logged after each skill generation, same in both auto and interactive modes). @@ -90,4 +127,4 @@ Or adjust the lookback window: python3 "${PLUGIN_ROOT}/scripts/friction/analyzer.py" --days 60 ``` -After processing, summarize: how many themes found, how many the user approved, how many skills generated. +After processing, summarize: how many themes found, how many classified operational (with recommendations listed), how many classified shape-gap, how many approved for forge, how many skills generated.