Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .claude-plugin/marketplace.json
Original file line number Diff line number Diff line change
Expand Up @@ -82,10 +82,10 @@
{
"name": "aidd-qa",
"source": "./plugins/aidd-qa",
"description": "Acceptance QA: validates observable behavior against acceptance criteria and records reviewer evidence. Browser is the only supported interface today.",
"description": "Acceptance QA: verify browser or CLI criteria and demonstrate journeys.",
"strict": true,
"metadata": {
"recommended": false
"recommended": true
}
}
]
Expand Down
12 changes: 7 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Unify **engineering teams** around **standardized workflows** and **shared best
🧱 **IDE agnostic** · 🏗️ **Legacy systems** · 🌱 **Token-optimized** · 🇫🇷 **Made in France**

<p>
<!--counts:start--><kbd>9 plugins</kbd> · <kbd>52 skills</kbd> · <kbd>2 agents</kbd><!--counts:end-->
<!--counts:start--><kbd>9 plugins</kbd> · <kbd>53 skills</kbd> · <kbd>2 agents</kbd><!--counts:end-->
</p>

[![Open Source](https://img.shields.io/badge/Open_Source-Yes-yellow?logo=open-source-initiative&logoColor=white)](https://opensource.org/)
Expand Down Expand Up @@ -58,7 +58,7 @@ Why not just write your own commands? → [FAQ](docs/FAQ.md#-why-aidd-instead-of

### Claude Code

Installs the 6 stable plugins (`aidd-ui` is 🚧 alpha, `aidd-telemetry` 🧪 beta, and `aidd-qa` 🆕 new, install those separately — see [Plugins](#-plugins)).
Installs the 7 recommended plugins (`aidd-ui` is 🚧 alpha and `aidd-telemetry` 🧪 beta, install those separately — see [Plugins](#-plugins)).

**In the session** (slash commands)

Expand All @@ -70,6 +70,7 @@ Installs the 6 stable plugins (`aidd-ui` is 🚧 alpha, `aidd-telemetry` 🧪 be
/plugin install aidd-vcs@aidd-framework
/plugin install aidd-pm@aidd-framework
/plugin install aidd-orchestrator@aidd-framework
/plugin install aidd-qa@aidd-framework
```


Expand All @@ -84,6 +85,7 @@ claude plugin install aidd-dev@aidd-framework
claude plugin install aidd-vcs@aidd-framework
claude plugin install aidd-pm@aidd-framework
claude plugin install aidd-orchestrator@aidd-framework
claude plugin install aidd-qa@aidd-framework
```
</details

Expand Down Expand Up @@ -240,7 +242,7 @@ learning only when it is durable enough to improve the next feature.

## 🧩 Plugins

Nine plugins covering the whole SDLC — **install the six stable ones**; they work together. (`aidd-ui` is 🚧 **alpha**, `aidd-telemetry` 🧪 **beta**, and `aidd-qa` 🆕 **new** — all three off the curated path.)
Nine plugins covering the whole SDLC — **install the seven recommended ones**; they work together. (`aidd-ui` is 🚧 **alpha** and `aidd-telemetry` 🧪 **beta** — both off the curated path.)

<table>
<tr>
Expand Down Expand Up @@ -324,9 +326,9 @@ Answers what a piece of work cost — tokens, models, and which skill spent them

### 🎬 [aidd-qa](plugins/aidd-qa/README.md) 🆕

`1 skill` · **new**
`2 skills` · **new**

Acceptance QA — locks browser scenarios from acceptance criteria and records reviewer evidence.
Acceptance QA: verify browser or CLI criteria and demonstrate observed journeys.

</td>
</tr>
Expand Down
2 changes: 1 addition & 1 deletion aidd_docs/memory/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,6 @@ The concern-to-plugin taxonomy is canonical in [`docs/ARCHITECTURE.md`](../../do

## Gotchas

- 9 plugins ship, 3 off the curated install path: `aidd-ui` is alpha, `aidd-telemetry` beta and opt-in, `aidd-qa` new and unproven outside this repository.
- 9 plugins ship, 2 off the curated install path: `aidd-ui` is alpha, `aidd-telemetry` beta and opt-in.
- A skill never links outside itself: the tree ships both flat and as a marketplace, so no relative path survives both.
- Bundled hooks run Node. No `node` on `PATH`, no memory refresh and no run journal.
10 changes: 2 additions & 8 deletions aidd_docs/memory/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,12 @@ How the project is tested: the layers, the tools, and the conventions. Where tes

| Surface | Validated by |
| --- | --- |
| Skills, agents, rules (markdown) | each action's own `## Test`, run end to end against a real project |
| Skills, agents, rules (markdown) | static guards check structure; verify execution through host traces and artifacts from a frozen version |
| `scripts/` and bundled hooks | `node --test` under the wrapper below |
| `cli/` | vitest, four projects — see the CLI bank |
| `kanban/` | its own vitest suite. It shares no code with `cli/` |
| Per-tool distributions | golden snapshots in `cli/tests/golden/`, mirrored by the `build-per-tool` CI matrix; Claude Code's own `plugin validate` over a fresh claude build, in `cli-ci.yml` |
| Browser journeys | `aidd-qa:01-acceptance-qa`, see below |
| Browser journeys | none in this repository |

## Tools

Expand Down Expand Up @@ -41,9 +41,3 @@ How the project is tested: the layers, the tools, and the conventions. Where tes
| `pnpm exec lefthook run pre-push` | the local gate before pushing |

CI runs more: `validate.yml` re-runs the whole pre-commit over the whole tree on every push and pull request, and `cli-ci.yml` adds jobs no local hook has. Both are in `deployment.md`.

## Browser QA

- Runner: `npx --yes @playwright/cli@0.1.17`, the framework pin. Never `latest` during QA.
- Also required: `ffmpeg` and `ffprobe`. Output is WebM evidence per scenario.
- Owned by `aidd-qa:01-acceptance-qa`; this repository ships the capability, it has no browser journey of its own.
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
---
status: done
---

# Instruction: split

## Architecture projection

```txt
plugins/aidd-qa/
├── .claude-plugin/plugin.json ✏️ lists 02-demo
└── skills/
├── 01-acceptance-qa/ ✏️ verify only: no recording, encoding, ffmpeg
└── 02-demo/ ✅ the recording rules, moved unchanged
```

## Acceptance criteria

- Verify writes `qa.md` with no video, and needs no `ffmpeg`.
- On the golden fixtures, verdicts by criterion, run verdict, out-of-interface and rejected sets match `next`.
- The demo records a `qa.md` scenario with the moved rules: VP9, 1280 wide, at most 12 s.
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
---
status: done
---

# Instruction: verify-rules

## Architecture projection

```txt
plugins/aidd-qa/skills/01-acceptance-qa/ ✏️ one clause per rule, where the decision is made
```

## Acceptance criteria

- One happy path per journey: a spec with two journeys runs both.
- A CLI criterion is verified by its command and expected output.
- Invoked by another skill or agent, it never asks: a missing decision yields `blocked` with the question.
- Test-only storage is proven before any start, by memory, an isolated fixture, or the user; an environment name is not proof.
- A crashed or UI-less entry gets one documented fallback, else `blocked`.
- Viewport and permissions come from the criterion, then `Browser QA` memory, then the default.
- A vague criterion becomes one observable condition, else it is rejected.
- A timed boundary is probed on both sides; measured values reach `Actual`.
- A short transition is measured inside `run-code`.
- Destructive means data outside the scenario's fixture or an irreversible external effect.
- `qa.md` holds `Limitations`; a limitation that weakens proof yields `blocked`.
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
---
status: done
---

# Instruction: demo

## Architecture projection

```txt
plugins/aidd-qa/skills/02-demo/ ✏️ browser video and CLI transcript
```

## Acceptance criteria

- A browser journey yields one video; a CLI journey yields the command and its output.
- A failing journey is shown as observed and labelled not expected.
- Invoked by another skill or agent, it asks nothing and returns its paths.
- A video lands in the folder the caller names, else beside the journey's source, never in a temporary directory.
- A video is named after its journey, never `happy-path` for a bug.
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
---
status: done
---

# Instruction: sdlc-wiring

## Architecture projection

```txt
plugins/aidd-orchestrator/skills/01-sdlc/references/03-check.md ✏️ QA after review, demo for the pull request
plugins/aidd-dev/agents/checker.md ✏️ may invoke QA and demo
.claude-plugin/marketplace.json, README.md ✏️ aidd-qa on the recommended path
```

## Acceptance criteria

- #919's acceptance criteria.
30 changes: 30 additions & 0 deletions aidd_docs/tasks/2026_10/2026_10_12_qa-verify-and-demo/plan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
---
objective: "Acceptance QA verifies browser and CLI criteria without video, a demo skill shows any journey, and the SDLC runs both before the draft pull request."
status: implemented
---

# Plan: QA verifies, demo shows

## Overview

| Field | Value |
| ---------- | ----- |
| **Goal** | Split `aidd-qa` into verify and show, harden verify for real journeys, wire both into the SDLC |
| **Source** | #918 (agreed contract in its body), #919 |

## Phases

| # | Phase | File |
| --- | ----------------- | ---------------------------- |
| 1 | split | [`phase-1.md`](./phase-1.md) |
| 2 | verify-rules | [`phase-2.md`](./phase-2.md) |
| 3 | demo | [`phase-3.md`](./phase-3.md) |
| 4 | sdlc-wiring | [`phase-4.md`](./phase-4.md) |

## Decisions

| Decision | Why |
| --- | --- |
| Verify carries no media; the demo owns it | a verdict is the step results; a missing `ffmpeg` must not block it |
| The demo never decides where its artifact goes | the caller (pull request, defect, user) does |
| Golden is verdicts by criterion, not bytes | an LLM run varies in wording, never in what it proves |
7 changes: 4 additions & 3 deletions docs/CATALOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ The exhaustive list of AIDD plugins, skills, and actions. Skills are invoked thr
- [aidd-orchestrator](#-aidd-orchestrator) - async orchestration (optional)
- [aidd-ui](#-aidd-ui) - UI / UX (🚧 alpha, not ready)
- [aidd-telemetry](#-aidd-telemetry) - measurement, hooks and skills (🧪 beta, off the curated path)
- [aidd-qa](#-aidd-qa) - acceptance QA (🆕 new, off the curated path)
- [aidd-qa](#-aidd-qa) - acceptance QA (🆕 new)

---

Expand Down Expand Up @@ -125,8 +125,9 @@ CLI, and each skill says so before doing anything else if it is missing.

## 🎬 aidd-qa

Locks a scenario scope from acceptance criteria, runs it against a reviewed candidate, and hands back a per-scenario verdict with recorded evidence. Browser only, for now.
Verify browser or CLI criteria; demonstrate observed journeys.

| Skill | Role | Actions |
| ------------------- | --------------------------------------------------------------------- | --------------------------------------------------------------------- |
| `01-acceptance-qa` | Lock scenarios from acceptance criteria, run them, and report a verdict per scenario | `01-load-scope`, `02-prerequisites`, `03-prepare-run`, `04-run-scenarios` |
| `01-acceptance-qa` | Verify criteria into an acceptance report | `01-load-scope`, `02-prerequisites`, `03-prepare-run`, `04-run-scenarios` |
| `02-demo` | Show a journey as video or transcript | `01-record` |
4 changes: 3 additions & 1 deletion plugins/aidd-dev/agents/checker.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ You are the checker. Your job is to judge finished work against its validator an
- Run the checklist on every code or diff, leaving no item unchecked.
- Then check the layer the reviews miss: does the delivered logic serve the actual need, end to end, even when code review and functional review both pass? Name any gap between intent and result.
- Demand command output or file evidence, never bare claims. Lean strict: a false alarm costs less than a missed defect.
- When a review skill fits the work, run it and let it write its report; that report is your deliverable and your judgment is what fills it. Never hand-write a parallel prose review beside it.
- Run the requested skill, otherwise a fitting review skill. Let it write its report; never hand-write a parallel review.
- Return your verdict, findings, and score on top. Hold yourself accountable for whatever you pass.

# Checklist
Expand Down Expand Up @@ -43,3 +43,5 @@ This is the behavioral baseline. Apply it to every code or diff, and extend it w
- `/aidd-dev:05-review`
- `/aidd-dev:04-audit`
- `/aidd-refine:02-challenge`
- `/aidd-qa:01-acceptance-qa`
- `/aidd-qa:02-demo`
2 changes: 1 addition & 1 deletion plugins/aidd-orchestrator/skills/01-sdlc/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ argument-hint: request | interactive request

## Behavior

Decide the mode as `mode.md` says, then read only it and the current zone's reference. Verify that every named provider is installed before calling it. A zone that cannot proceed stops the run and says what it would take to resume.
Decide the mode as `mode.md` says, then read only it and the current zone's reference. Verify every required provider is available through the host before calling it. A zone that cannot proceed stops the run and says what it would take to resume.

Spawn specialized agents for isolated work. Parallelize independent work when it is faster. Give each agent one focused task that a smaller model can execute. Repeat the responsible zone when delegated work returns an actionable gap.

Expand Down
24 changes: 17 additions & 7 deletions plugins/aidd-orchestrator/skills/01-sdlc/references/03-check.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,7 @@

## Behavior

- Use one fresh checker, independent from implementation, to review the candidate against the contract, plan, and validation evidence.
- After the review clears, the same checker challenges whether the real outcome is trustworthy and serves the user.
Use one fresh, independent `@aidd-dev:checker` throughout the diagram, one skill per call.

## Findings

Expand All @@ -17,7 +16,7 @@ A checker's finding is routed by what it is about. A refusal the user gives at a

## Handoff

- Commit whatever this zone wrote, then open the draft pull request when the checker returns no actionable finding.
- Commit whatever this zone wrote, then open the draft pull request with the report and artifact paths when the checker returns no actionable finding.

```mermaid
---
Expand All @@ -32,6 +31,9 @@ flowchart TD
ValidationReports["$validation_reports"]
Checker(["@aidd-dev:checker"])
Review["/aidd-dev:05-review"]
QA["/aidd-qa:01-acceptance-qa"]
QAReport["$qa.md"]
Demo["/aidd-qa:02-demo"]
end

subgraph ChallengeStage["Challenge the outcome"]
Expand All @@ -45,6 +47,7 @@ flowchart TD
Frame["01 Frame"]
Todo["/aidd-dev:10-todo"]
Deliver["02 Deliver"]
Stop["Blocker + resume action"]
end

subgraph ShipStage["Ship the reviewed candidate"]
Expand All @@ -59,7 +62,14 @@ flowchart TD
ValidationReports --> Checker
Checker --> Review
Review -- "Return actionable review findings." --> Findings
Review -- "When the review clears, challenge the outcome." --> Challenge
Review -- approve --> QA
QA -- pass --> QAReport
QAReport -- "Read passing scenarios." --> Demo
Demo --> Challenge
QA -- skipped --> Challenge
QA -- fail --> Deliver
QA -- "blocked: criteria" --> Frame
QA -- "blocked: execution" --> Stop
Challenge -- "Return actionable challenge findings." --> Findings
Challenge -- "When the outcome is trustworthy, open the draft pull request." --> PullRequest
Findings -- "Use product findings as the next Frame source." --> Frame
Expand All @@ -73,8 +83,8 @@ flowchart TD
classDef artifact fill:#DCFCE7,stroke:#16A34A,color:#14532D,stroke-width:2px
classDef zone fill:#F1F5F9,stroke:#64748B,color:#0F172A,stroke-width:2px

class Review,Challenge,Todo,PullRequest skill
class Review,QA,Demo,Challenge,Todo,PullRequest skill
class Checker agent
class Contract,Plan,CommittedCandidate,ValidationReports,Findings,PullRequestUrl artifact
class Frame,Deliver zone
class Contract,Plan,CommittedCandidate,ValidationReports,QAReport,Findings,PullRequestUrl artifact
class Frame,Deliver,Stop zone
```
5 changes: 3 additions & 2 deletions plugins/aidd-qa/.claude-plugin/plugin.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,14 @@
"$schema": "https://json.schemastore.org/claude-code-plugin-manifest.json",
"name": "aidd-qa",
"version": "1.0.0",
"description": "Acceptance QA: validates observable behavior against acceptance criteria and records reviewer evidence. Use when a reviewed candidate needs evidence that it meets its acceptance criteria. Do NOT use for diff review, unit or integration tests, or fixing the application.",
"description": "Verify browser or CLI acceptance criteria and demonstrate journeys with observed evidence.",
"author": {
"name": "AI-Driven Dev",
"url": "https://github.com/ai-driven-dev"
},
"skills": [
"./skills/01-acceptance-qa"
"./skills/01-acceptance-qa",
"./skills/02-demo"
],
"keywords": [
"qa",
Expand Down
11 changes: 10 additions & 1 deletion plugins/aidd-qa/CATALOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ Auto-generated index of skills, agents, references and assets shipped by the `ai
- [`.claude-plugin`](#claude-plugin)
- [`skills`](#skills)
- [`skills/01-acceptance-qa`](#skills01-acceptance-qa)
- [`skills/02-demo`](#skills02-demo)

---

Expand All @@ -30,5 +31,13 @@ Auto-generated index of skills, agents, references and assets shipped by the `ai
| `actions` | [04-run-scenarios.md](skills/01-acceptance-qa/actions/04-run-scenarios.md) | - |
| `assets` | [qa-report-template.md](skills/01-acceptance-qa/assets/qa-report-template.md) | - |
| `references` | [interface-browser-playwright-cli.md](skills/01-acceptance-qa/references/interface-browser-playwright-cli.md) | - |
| `-` | [SKILL.md](skills/01-acceptance-qa/SKILL.md) | `Validate a reviewed candidate's observable behavior against its acceptance criteria and record short named videos as reviewer evidence. Use when acceptance criteria exist and a browser-observable journey needs proof it holds. Do NOT use for diff review, unit or integration tests, or application fixes.` |
| `-` | [SKILL.md](skills/01-acceptance-qa/SKILL.md) | `Prove reviewed browser or command journeys against acceptance criteria. Use for acceptance QA, not diff review, unit or integration tests, or application fixes.` |

#### `skills/02-demo`

| Group | File | Description |
|-------|------|---|
| `actions` | [01-record.md](skills/02-demo/actions/01-record.md) | - |
| `references` | [interface-browser-playwright-cli.md](skills/02-demo/references/interface-browser-playwright-cli.md) | - |
| `-` | [SKILL.md](skills/02-demo/SKILL.md) | `Show a journey or bug through a browser video or CLI transcript. Use for reviewer evidence, not acceptance verification or fixes.` |

11 changes: 4 additions & 7 deletions plugins/aidd-qa/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,13 @@

# aidd-qa

Acceptance QA concern for the AI-Driven Development framework.
Verify browser or CLI acceptance criteria into `qa.md`; demonstrate journeys with a video or transcript.

> Status: new, off the curated install path until proven.

First time? Install with `/plugin install aidd-qa@aidd-framework`, then run `aidd-qa:01-acceptance-qa`.

Validates a reviewed candidate's observable behavior against its acceptance criteria and records reviewer evidence. Scenarios come only from acceptance criteria, never from the diff or the source code. Browser is the only supported interface today; a scenario without a browser-observable outcome is listed out of interface rather than tested.
Install: `/plugin install aidd-qa@aidd-framework`.

## Skills

| Skill | Description |
|---|---|
| [acceptance-qa](skills/01-acceptance-qa/SKILL.md) | Validate a reviewed candidate against its acceptance criteria and record one short named video per locked scenario. |
| [acceptance-qa](skills/01-acceptance-qa/SKILL.md) | Verify a reviewed candidate against its criteria. |
| [demo](skills/02-demo/SKILL.md) | Show a journey's observed outcome. |
Loading