Skip to content

fix(forge-friction): classify themes as operational vs shape-gap before routing to forge #13

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
griffinwork40 merged 2 commits into from
Oct 24, 2026
Merged

fix(forge-friction): classify themes as operational vs shape-gap before routing to forge #13

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
7abf5ee
fix(forge-friction): classify themes as operational vs shape-gap befo…
Oct 23, 2026
d55a253
fix(forge-friction): correct $ARGUMENT typo and remove skill-addressa…
griffinwork40 Oct 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 53 additions & 16 deletions skills/forge-friction/SKILL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: forge-friction
description: "Surface recurring friction from session telemetry and generate targeted skills. Use when the user runs /forge-friction."
description: "Surface recurring friction from session telemetry, classify as operational vs shape-gap, and generate targeted skills for shape-gap themes only. Use when the user runs /forge-friction."
argument-hint: "[--dry-run] [--auto]"
---

Expand All @@ -18,13 +18,21 @@ Tool-error categories carry a failure-class suffix. `:timeout`, `:truncated`, an

## Dry-run mode

If `$ARGUMENT` contains `--dry-run`: after the analyzer runs, identify themes using the filter criteria in "Your job" below, but instead of presenting them interactively, output a single markdown table:
If `$ARGUMENTS` contains `--dry-run`: after the analyzer runs, identify, filter, and classify themes using the criteria in "Your job" below, but instead of presenting them interactively, output two sections:

**Operational** (recommendations only — not skill candidates):

| Category | Count | Recommendation |
|---|---|---|
| ... | N | PreToolUse hook / config change / budget adjustment / ... |

**Shape-gap** (forge candidates):

| Category | Count | Example friction_detail |
|---|---|---|
| ... | N | "truncated quote 1"; "truncated quote 2" |

One row per theme. Truncate each `friction_detail` to ~80 chars. Up to 3 examples per row (semicolon-separated in the cell). If no themes pass the filter, print "No themes in lookback window."
One row per theme. Truncate each `friction_detail` to ~80 chars. Up to 3 examples per row (semicolon-separated in the cell). If no themes pass the filter in either table, print "No themes in lookback window."

After the table (or "no themes" line), print:

Expand All @@ -34,34 +42,63 @@ Stop. Do NOT proceed to the y/n prompts, `/forge` dispatch, or telemetry write d

## Auto mode

If `$ARGUMENTS` contains `--auto`: skip all y/n prompts and auto-approve all qualifying themes. For each theme that passes the filter criteria (3+ sessions, recurring failure mode, skill-addressable), immediately run `/forge` with the theme as a seed. Chain the invocations sequentially, then log telemetry for each generated skill. `--auto` respects `--dry-run` (if both flags are present, dry-run takes precedence and no /forge invocation or telemetry occurs).
If `$ARGUMENTS` contains `--auto`: skip all y/n prompts and auto-approve all qualifying themes. For each theme classified as shape-gap in Step 2, immediately run `/forge` with the theme as a seed. Chain the invocations sequentially, then log telemetry for each generated skill. `--auto` respects `--dry-run` (if both flags are present, dry-run takes precedence and no /forge invocation or telemetry occurs).

## Your job

Read through the friction categories and their examples. For each category with 3+ sessions, analyze the `friction_detail` strings to identify **recurring themes** — specific, repeated failure modes that a skill could address.
Read through the friction categories and their examples. For each category with 3+ sessions, analyze the `friction_detail` strings to identify **recurring themes** — specific, repeated failure modes.

Not all friction is fixable by a skill. Filter for themes where:
### Step 1 — Filter

Not all friction is actionable. Filter for themes where:
- The same failure mode repeats across multiple sessions (not one-off issues)
- A skill could change the agent's default behavior to avoid the friction
- The fix is a workflow shape change, not a reminder or checklist
- The failure mode is specific enough to act on (not a grab-bag of unrelated errors)
- **Prefer high-confidence failure classes.** Pursue `:timeout` / `:truncated` / `:slow` categories first; they are real friction. Treat `:plain` and `:unclassified` categories as low-confidence — only pursue one if its `friction_detail` / `goal` examples make the recurring failure mode unmistakable.

### Step 2 — Classify

For each theme that passes the filter, classify it as **operational** or **shape-gap**:

**Operational** — infrastructure/tooling misbehavior, not a missing workflow. Indicators:
- Tool timeouts, slow tools, truncated output (`:timeout`, `:slow`, `:truncated` on any tool)
- Rate limits, usage limits, connection errors (`subagent_fail:RateLimitError`, `UsageLimitError`, `APIConnectionError`, `APIConnectionTimeoutError`)
- Hook blocks working as designed (`hook_block:*` categories)
- Session closures from external causes (`closure:abort`, `closure:timeout` from usage caps, network, or user signal)
- Permission errors, idle watchdog kills (`subagent_fail:PermissionDeniedError`, `IdleWatchdogError`)
- Model rejection errors (`subagent_fail:Error` from HTTP 400/404 on model ids)
- Iteration cap exhaustion from budget undersizing (`closure:iteration_cap`)

**Shape-gap** — a recurring workflow pattern agents perform without skill coverage. Indicators:
- Multiple sessions performing the same multi-step sequence ad-hoc (visible in `goal` field)
- A structural problem agents solve differently each time with no consistent approach
- Goals that consistently `partially_achieved` or `aborted` because no skill owns the workflow shape — not because tools broke
- The `friction_detail` shows the agent improvising a multi-step workaround, not a single tool failing

### Step 3 — Route

**Operational themes:** For each, emit a one-line **recommendation** — the right fix category (PreToolUse hook, config change, budget adjustment, existing skill routing fix, or infrastructure improvement). Do NOT propose `/forge` for operational themes — they are not skill candidates.

**Shape-gap themes:** These are the forge candidates. Route them through the interactive or auto flow below.

**If `--auto` is present** in `$ARGUMENTS`:
- Auto-forge only high-confidence themes. **Skip `:plain` and `:unclassified` tool-error categories** — without human review, auto-forging the low-confidence residue produces noise skills. Pursue `:timeout` / `:truncated` / `:slow` and non-tool-error categories that pass the filter.
- For each actionable theme, immediately run `/forge` from this plugin, seeding it with: "Create a skill that addresses this recurring friction: [theme summary]. Examples: [2-3 friction_detail quotes]."
- List operational themes with their recommendations (no forge).
- Auto-forge only shape-gap themes. **Skip `:plain` and `:unclassified` tool-error categories entirely in auto mode** — without human review, the low-confidence residue produces noise.
- For each shape-gap theme, immediately run `/forge` from this plugin, seeding it with: "Create a skill that addresses this recurring workflow gap: [theme summary]. Examples: [2-3 friction_detail quotes showing the ad-hoc workaround pattern]."
- Chain the invocations sequentially (wait for forge + qualify to complete before starting the next one).
- After each forge + qualify completes, log to telemetry (see telemetry format below).
- Do not ask for user confirmation; auto-approve all qualifying themes.
- Do not ask for user confirmation; auto-approve all qualifying shape-gap themes.

**If `--auto` is not present** (interactive mode):

1. **For each actionable theme**: summarize it in one line with the count, then quote 2-3 representative `friction_detail` examples. Ask: "Generate a skill for this? (y/n)"
1. **Operational themes first**: list each with its one-line recommendation. No y/n prompt — these are informational.

2. **Shape-gap themes next**: for each, summarize it in one line with the count, then quote 2-3 representative `friction_detail` examples. Ask: "Generate a skill for this? (y/n)"

2. **If the user says yes**:
- Run `/forge` from this plugin, seeding it with: "Create a skill that addresses this recurring friction: [theme summary]. Examples: [2-3 friction_detail quotes]."
3. **If the user says yes**:
- Run `/forge` from this plugin, seeding it with: "Create a skill that addresses this recurring workflow gap: [theme summary]. Examples: [2-3 friction_detail quotes showing the ad-hoc workaround pattern]."
- After forge + qualify complete, log to telemetry (see telemetry format below).

3. **If the user says no**, skip it and move on.
4. **If the user says no**, skip it and move on.

**Telemetry** (logged after each skill generation, same in both auto and interactive modes).

Expand Down Expand Up @@ -90,4 +127,4 @@ Or adjust the lookback window:
python3 "${PLUGIN_ROOT}/scripts/friction/analyzer.py" --days 60
```

After processing, summarize: how many themes found, how many the user approved, how many skills generated.
After processing, summarize: how many themes found, how many classified operational (with recommendations listed), how many classified shape-gap, how many approved for forge, how many skills generated.
Loading