diff --git a/CHANGELOG.md b/CHANGELOG.md index 51f3205..69cd0b4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,7 @@ ### Added - **`pipelex-integrate` — wire an MTHDS method into a Python or TypeScript codebase.** Given a local bundle, a published `method_ref` at a tag, or a catalog `method_id`, and a project, the skill picks the codegen target by audience (`ts-zod`; `python-pydantic` for a hosted-API consumer, `python-structures` for a Pipelex host), has the workshop's `mthds_codegen` write the generated tree into one dedicated directory per method through its `output_dir` arm — no artifact byte ever passes through the model, and a refused write is never worked around by writing bytes from the conversation — excludes that tree from the project's formatters and linters *before* it exists while keeping it under the type checker, records a `sources.json` sidecar (selector, target, pipe signature, source hashes) so a second run is a refresh and a bundle edit is detectable, wires an offline drift gate (`scripts/codegen-check.mjs` over `@pipelex/sdk`'s `runCodegenCheck`) into a TypeScript project's existing check, and writes one typed call-site module per method over `startAndWaitForResult` / `start_and_wait` and the generated binder or model. The call site is typed from the main pipe's signature on the validate verdict. A Python consumer gets no offline gate yet — the Python SDK has no check and the skill will not add the `pipelex` runtime to get one — and the report says so. A project made from a Pipelex starter keeps its own codegen harness: the skill runs the project's `codegen` script or `make add-method` and never writes a second layout beside the first. Two things it refuses outright, because both are silent: generating a second method into a directory that already holds another method's tree (every method of a target emits the same file names, so that overwrites the first and reports no orphan), and reaching a project the workshop was not launched in (a path inside the workshop is legal wherever it points, so containment is read before the first write and a tree that landed beside the wrong project is never moved across). A write whose only fault is orphans is not one of them: stamped files the new lock does not list make the drift check report the tree non-current, but with nothing else drifted the generation itself is sound, so the skill reads `orphans[]` and `drifts[]` rather than the verdict, finishes the integration, and reports the orphan paths by name — never deleted, with a dedicated directory per generation as the fix and the gate counting each orphan as a drift until the directory holds one generation. Any other drift ends the run. One upstream defect it names rather than patches: the `ts-zod` emitter writes `binder.ts`'s sibling import without a file extension, which a plain Node ESM project rejects at type-check and at runtime while a bundler resolution accepts — the generated tree is stamped and hashed, so the fix is the emitter's and the skill says so. Ships `references/typescript.md`, `references/python.md` and `references/codegen-check.mjs`. +- **`pipelex-scaffold` — the front door to a project that does not exist yet.** Two branches and no templates of its own: one of the Pipelex starters (`pipelex-starter-js` for a web app with contract-rendered forms, `pipelex-starter-python` for a CLI or service), acquired as a fresh-history local clone by default or through `gh repo create --template` after confirmation, committed once as it came, then renamed by the clone's **own** `bootstrap` skill, read from its `SKILL.md` and never reimplemented; or the ecosystem's initializer (`uv init --package`, `npm create next-app@latest`, …) when the user wants their framework. Both branches end with the env-file convention, the key filled only from the shell environment and never asked for in the conversation, and a hand-off to `/pipelex-integrate`. It is the plugin's third MCP-free skill. Ships `references/starters.md` and `references/initializers.md`. - **`pipelex-vibe/mcp/vibe-mcp.toml` — the Mistral Vibe target bakes the workshop launcher**: Vibe has no plugin manifest, so the target now ships the `npx -y @pipelex/mcp@latest` launcher as a `[[mcp_servers]]` stdio entry to append to the end of `~/.vibe/config.toml`, rendered from the same `[vars.mcp_server]` block as the Claude and Codex manifests and enforced by `make check`. Write your API key into the entry's `env` table, because Vibe spawns stdio servers with a minimal environment and never sees an exported `PIPELEX_API_KEY`; the MCP-backed skills' Vibe stop message now points at the fragment and says so, and their Vibe auth line no longer claims the server reads the session environment. Before appending, delete the `mcp_servers = []` line a new Vibe config carries and any `pipelex` server registered by hand: either leftover stops Vibe from starting. Vibe records its configuration, that key included, in every session log under `~/.vibe/logs/session/`, so redact the key before sharing one. - **`pipelex-synthetic-inputs` — a skill that renders the files a method needs, from code.** PDFs through `reportlab` (canvas letters, multi-page Platypus reports, tables, and a composed line-item document whose totals come from its items) and PNGs through `Pillow` and `matplotlib` in four categories: `chart` (bar, line, pie, scatter), `diagram` (a node/edge list laid out on a grid with clipped arrows), `document_scan` (an A4-at-150-dpi page put through a seeded skew/tint/grain/vignette post-process, for OCR and document-understanding methods) and `screenshot` (window chrome, sidebar, stat tiles, and a status-badged table or card grid). Word and Excel come along from `pipelex-inputs`. No AI is involved anywhere, and only packages whose licences are compatible with MIT are used. **Photographs and handwriting are deliberately out of scope** — code cannot render either to a standard a vision model would accept, so the skill says so and asks for a real file instead of handing a method an imitation. It is MCP-free, the second such skill after `pipelex-explain`, and it installs what it needs itself: `uv` with ephemeral packages, or a venv it creates under the user's cache directory when `uv` is absent. Nothing is installed into the project, and installing a *tool* always asks first. - **The recipes refuse to render a file that would be silently wrong.** A recipe is copied and adapted, so the content block is where things go wrong: each one now bounds its content against the page it is drawing on and stops with the cure rather than exiting 0 on a file that lies. A `document_scan` whose items overrun the page, a `screenshot` whose rows run off the canvas, a `diagram` with two nodes in one grid cell, an unsubstituted `` still in the path — every one of these used to print success and hand a method an input missing exactly the field it was meant to read. Line-item descriptions wrap instead of overprinting the quantity beside them, the PDF table recipe sizes its columns instead of drawing them off the paper, and every recipe renders beside its target and renames on success, so a crash can no longer truncate a file the user already had. @@ -12,7 +13,7 @@ ### Changed -- **The family hands the method to the code.** `pipelex-design`'s delivery and `pipelex-inputs`' closing report point at `/pipelex-integrate` when the workspace holds a codebase; `pipelex-edit` and `pipelex-design`'s re-entry announce stale generated types after a bundle edit by reading the `sources.json` sidecars `pipelex-integrate` writes — the only drift guard a Python consumer has until its SDK gains an offline check. The plugin's light-template pin (`explicit: false` on every `mthds_inputs_template` call) gains one exception: `pipelex-integrate` passes `explicit: true` on its fallback path, when the verdict carries no main-pipe signature, because it needs each input's concept ref to type a parameter. +- **The family hands the method to the code.** `pipelex-design`'s delivery and `pipelex-inputs`' closing report point at `/pipelex-integrate` when the workspace holds a codebase and, for the design skill, at `/pipelex-scaffold` when it holds none; `pipelex-edit` and `pipelex-design`'s re-entry announce stale generated types after a bundle edit by reading the `sources.json` sidecars `pipelex-integrate` writes — the only drift guard a Python consumer has until its SDK gains an offline check. The plugin's light-template pin (`explicit: false` on every `mthds_inputs_template` call) gains one exception: `pipelex-integrate` passes `explicit: true` on its fallback path, when the verdict carries no main-pipe signature, because it needs each input's concept ref to type a parameter. - **`pipelex-design` is model-invocable.** The skill shipped `disable-model-invocation: true` on Claude and Vibe, so it could only be reached by typing `/pipelex-design`. That made `pipelex-edit`'s routing a dead end — it classified a structural change, then had to hand the user a slash command to type and throw away the baseline verdict it had just produced. The flag is gone on every target, the skill's description now carries natural-language triggers ("design a method", "create a pipeline", "add a step", "rewire this pipeline") so the model can actually reach it, and `pipelex-edit` names the affected pipes and invokes `/pipelex-design` directly instead of stopping. The consent gate stays where it belongs: the design run still announces its captured contract in one line before writing anything. - **`pipelex-inputs` delegates file generation instead of doing it inline.** (Breaking) Its Document Generation section and Fallback Strategy block are gone, and the `native.Image` / `native.Document` strategy rows now delegate to `pipelex-synthetic-inputs`. "Generate File Inputs" is the delegation contract — the request fields to pass, the bare path that comes back and goes into `inputs.json`, and what happens when the factory cannot deliver: that one input is left unfilled with the reason in the report, and the rest of the flow continues. Anyone relying on the inline recipes should read them in the new skill's `references/`, where they are deeper and now executed by tests. - **Tooling:** Pinned `ruff` to an exact `0.16.4`, replacing the `>=0.6.8` floor. The exact pin matches what the Ruff VS Code extension now bundles, which matters because Ruff 0.16 lints `pyproject.toml` itself: the extension syncs the config file to the language server, and a pre-0.16 binary parses it as Python source and paints phantom `invalid-syntax` diagnostics on lines like `requires-python`. A floor let the editor and the CLI resolve to different binaries; an exact pin cannot. This is a dev dependency, so nothing shipped changes, and the upgrade produced no lint findings and no reformatting. @@ -20,11 +21,16 @@ ### Fixed -- **The code `pipelex-integrate` emits now compiles and runs the cases it claimed.** A review pass over the emitted artifacts — as opposed to the skill prose, which the dogfood matrix had exercised — found that none of them had ever been compiled or executed. The TypeScript call site declared its inputs as an `interface`, which TypeScript gives no implicit index signature and so cannot assign to the SDK's `inputs: Record`: it failed `TS2322` on every resolution, bundler included. Both call sites read only `main.mthds`, so any multi-file bundle produced types that were correct and a run that could not load what they were projected from; they now submit every `.mthds` file of the bundle, because a bundle is one closure. The verdict's `pipe_ref` is namespaced and the run route takes the bare code, and nothing said to strip the domain — a confusion that type-checks, passes the offline gate, and surfaces only on a real run. A list-valued output is narrowed according to its `multiplicity` instead of being parsed as a single concept. The call site's own relative imports are named as the agent's to extend with `.js` where the project's resolution demands it, rather than being misrouted to the emitter defect that never applies to them. The offline drift gate reported `current` for a sidecar whose `sources` was present but not an object — checking nothing, printing nothing, exiting 0 — and for an artifact given a byte-order mark, which the default decoder strips before hashing; both now fail closed, no branch is both silent and green, and the verdict's explanation survives a pipe. -- **Running the emitted artifacts a second time found the defects the first pass's own fixes left behind.** Where the previous round read `references/codegen-check.mjs` and rewrote the Python call site's bundle reader, this one executed both. The gate still reported `current`, exit 0, for a sidecar whose whole content was valid JSON but not an object (`null`, `[]`, a string, a number, a boolean): the previous round guarded `sources` against exactly this and reached it through `sidecar?.sources`, where optional chaining does to the sidecar precisely what `??` would have done to `sources`. And the Python reader that same round had switched to `rglob` returned an empty list for a missing or empty bundle directory without raising, so a wrong `BUNDLE_DIR` sent the run out with `mthds_contents=[]` and failed server-side against the pipe instead of locally against the path — the TypeScript twin gets that for free from `readdir`'s `ENOENT`, and the Python one now asks for it. Both are pinned by tests, the gate's by one that runs the script rather than reading it. +- **The code the two new skills emit now compiles and runs the cases it claimed.** A review pass over the emitted artifacts — as opposed to the skill prose, which the dogfood matrix had exercised — found that none of them had ever been compiled or executed. The TypeScript call site declared its inputs as an `interface`, which TypeScript gives no implicit index signature and so cannot assign to the SDK's `inputs: Record`: it failed `TS2322` on every resolution, bundler included. Both call sites read only `main.mthds`, so any multi-file bundle produced types that were correct and a run that could not load what they were projected from; they now submit every `.mthds` file of the bundle, because a bundle is one closure. The verdict's `pipe_ref` is namespaced and the run route takes the bare code, and nothing said to strip the domain — a confusion that type-checks, passes the offline gate, and surfaces only on a real run. A list-valued output is narrowed according to its `multiplicity` instead of being parsed as a single concept. The call site's own relative imports are named as the agent's to extend with `.js` where the project's resolution demands it, rather than being misrouted to the emitter defect that never applies to them. The scaffold's `uv add` recipes ran in whatever directory the agent stood in, which from a parent is the user's own project and lockfile rather than the one just created; the minimal TypeScript recipe would have committed `node_modules/` into the commit meant to be a readable baseline; `npm create next-app` was written without the `--` separator, so npm ate the flags and the advertised non-interactive command prompted; the `.env` copy had no `-n` and would overwrite a key the user had already filled on the fresh-clone path; and the `rm -rf /.git` was not chained to the clone that creates ``. The offline drift gate reported `current` for a sidecar whose `sources` was present but not an object — checking nothing, printing nothing, exiting 0 — and for an artifact given a byte-order mark, which the default decoder strips before hashing; both now fail closed, no branch is both silent and green, and the verdict's explanation survives a pipe. +- **Running the emitted artifacts a second time found the defects the first pass's own fixes left behind.** Where the previous round compiled the TypeScript call site, read `references/codegen-check.mjs` and rewrote the Python call site's bundle reader, this one executed both and ran the scaffold's initializer recipes, then integrated into the project they actually produce, which is where the two halves meet. `tsc --init` writes `"types": []` as an active key — switching off the `@types/node` the recipe installed on the line before — so the call site failed `TS2591` on `node:fs/promises`, on `node:path` and on `process`, with cascading implicit-any on the bundle reader; nothing in the errors points at the tsconfig, so the recipe now sets `"types": ["node"]` itself and says which TypeScript major it got, because the `tsc --init` template moved with the major. Every `uv init` now carries `--no-workspace`: run inside a directory that already holds a `pyproject.toml`, a bare one does not create a standalone project at all but appends a `[tool.uv.workspace]` table to **the user's own file** and leaves the lock at the parent root, so the new project had no lockfile and did not resolve alone — the same hazard the `uv add` parentheses were added for, one command earlier, and the parentheses could not see it. The follow-on `npm install` lines in the Vite and Express recipes are parenthesised for that reason too, and npm is the worse of the two: where `uv add` fails outright when it finds no project nearby, npm finds the parent, writes the dependency into the user's `package.json`, puts `node_modules/` in the user's tree, and exits 0. Branch B's pristine commit could reach an enclosing repository: `git -C ` sets git's working directory and scopes nothing, so with no `.git` of its own `` is governed by the user's repo and a pathspec-less `add -A` staged that whole worktree — the user's unrelated files committed under this skill's message. The step now tests `git -C rev-parse --show-toplevel` instead of trusting a list of initializers that `git init` (`uv init` is on that list only when it creates a standalone project), both commits carry a `-- .` pathspec, and the read-back names paths rather than a count that cannot tell a correct scaffold from a swept-up worktree. The clone guard the last round added to the skill was missing from `references/starters.md`, the file the skill names as carrying every command — so the one irrecoverable line shipped unguarded in the copy an agent reads it from. And the offline gate still reported `current`, exit 0, for a sidecar whose whole content was valid JSON but not an object (`null`, `[]`, a string, a number, a boolean): the previous round guarded `sources` against exactly this and reached it through `sidecar?.sources`, where optional chaining does to the sidecar precisely what `??` would have done to `sources`. And the Python reader that same round had switched to `rglob` returned an empty list for a missing or empty bundle directory without raising, so a wrong `BUNDLE_DIR` sent the run out with `mthds_contents=[]` and failed server-side against the pipe instead of locally against the path — the TypeScript twin gets that for free from `readdir`'s `ENOENT`, and the Python one now asks for it. All of these are now pinned by tests, the gate's by one that runs the script rather than reading it. - **The helper budget said two and listed one.** `pipelex-integrate` announced "at most two shared helpers", named the client factory, and closed with "and nothing else" — residue of the wire-output helper that was struck for being lossy and is forbidden by name a few lines below. A stale count would be cosmetic; this one was written permission to create the single thing the campaign removed. It is one helper, in the skill, in both places it was stated, and in `docs/decisions.md`. - **The reference-copy test asserted that `shutil` copies bytes.** It built a fresh tree into a temp directory and compared it against the source it had just been built from, so it could not observe the only failure that matters — a stale committed copy under `pipelex/`, `pipelex-codex/` or `pipelex-vibe/`, which is what a user installs. It now compares the committed copies themselves, and its three-target parametrization exercises three different trees instead of three identical calls. -- **The guards the dogfood pass added no longer misfire on the cases they created.** The occupied-directory rule refused a `codegen.lock` with no sidecar as another generation's, but the lock is written a step before the sidecar, so every stop between them leaves exactly that state for the method being integrated — and a harness-owned layout keeps no sidecar by design, so the rule forbade the write its own harness section prescribes. Both are now named exceptions, with regeneration in place as the answer rather than a second tree for one method. The containment pre-check is read on resolved paths, because a lexical reading passes a symlink pointing outside the workshop and fails a project under `/tmp` that the tool would have accepted; it is also taken as soon as the project is identified, since by the time the old placement ran, a bundle copy and the tooling exclusions were already on disk. The three stated causes of an absent `main_pipe` are all three in the failure table, where the remedy differs by cause, and the same signature is read from the verdict's text summary when the structured field did not arrive. +- **The guards the dogfood pass added no longer misfire on the cases they created.** The occupied-directory rule refused a `codegen.lock` with no sidecar as another generation's, but the lock is written a step before the sidecar, so every stop between them leaves exactly that state for the method being integrated — and a harness-owned layout keeps no sidecar by design, so the rule forbade the write its own harness section prescribes. Both are now named exceptions, with regeneration in place as the answer rather than a second tree for one method. The containment pre-check is read on resolved paths, because a lexical reading passes a symlink pointing outside the workshop and fails a project under `/tmp` that the tool would have accepted; it is also taken as soon as the project is identified, since by the time the old placement ran, a bundle copy and the tooling exclusions were already on disk. Reaching a runtime behind `nvm`, `fnm`, `volta`, `asdf` or `mise` means resolving it to a path and carrying that into every later command, not sourcing a shell that the next command will not inherit — otherwise the prerequisite read as met, the pristine commit was spent, and the delegated bootstrap then failed; a shim that answers nothing is not a runtime, the starter's version floor still applies, and a manager that would install a version it lacks is the toolchain install this step forbids. The workshop is spawned on the harness's own `PATH`, so on that machine the hand-off needs a relaunch rather than the unconditional promise the report used to make. The three stated causes of an absent `main_pipe` are all three in the failure table, where the remedy differs by cause, and the same signature is read from the verdict's text summary when the structured field did not arrive. +- **A directory holding nothing but `.git` is somewhere `pipelex-scaffold` will build.** The skill refused any target directory that was not empty, which made it refuse the state it produces itself one step later: `mkdir my-app && cd my-app && git init` is an ordinary way for a user to arrive, and the skill's own initializer branch runs `git init -b main` in the directory it is working in. A lone `.git` now reads as empty, and **both branches serve it**: the initializer runs in the directory as it stands, while the starter branch — which `git clone` cannot aim at a directory already holding a `.git` — acquires into a temporary path beside it, discards the template's history while the clone is still its own, and copies the template in, dotfiles included, so the repository the user made goes on standing with their branch, their history and their remote. Everything else goes on refusing — `.DS_Store`, `.idea/`, `.vscode/` and `Thumbs.db` included — because the exception is one directory entry named in full and not a class of files the agent may decide to overlook, which is the judgement the refusal exists to prevent. Nothing about clearing changed: a directory that is not empty is still never emptied, and making room is still never offered. +- **The scaffold's recipes now match the guarantees its own references state.** A review round over the skill as an executable document, rather than over its prose, found the places where the two disagreed. The named-framework recipe still prescribed a bare `uv init --package ` while `references/initializers.md` said in bold that `--no-workspace` is on every `uv init` above — so the one line an agent executes appended a `[tool.uv.workspace]` table to the user's own `pyproject.toml`, the exact hazard the reference documents, and the discipline test pinned the reference alone and never the skill body. The key write was an unconditional append, which on the fresh-clone shortcut put a second `PIPELEX_API_KEY` after one the user had already filled; every dotenv reader resolves a repeated name to the later line, so a stale exported value silently replaced a working key while the report said theirs was kept — it is gated on a file-side presence test now. `gh repo create --clone` takes no destination and clones into `./`, which every later step addressed as ``. The pristine commit carried its `-- .` pathspec on the staging but not on the commit, and a bare `git commit` commits the whole index, so anything the user had staged in an enclosing repository rode along under this skill's message — both commits carry it now, and the guard's description says what it actually guarantees. The minimal TypeScript recipe opened with `mkdir `, which aborts its own `&&` chain when `` is the "here" directory the skill explicitly permits, and its default resolution is the one the same file calls broken, now flagged where the command is chosen rather than only below it. The fresh-clone shortcut and the template-checkout stop shared a detection signal and prescribed opposite actions; the shortcut now names the `origin` that separates a copy from the template itself. +- **The acquisition that serves a lone `.git` now serves it when the user says "here".** A second review round, over the recipe as something executed rather than read, found the temporary path derived from the destination's *spelling*: `dirname .` is `.`, so a destination given as `.` — the ordinary one, since the user is standing in the directory they just `git init`-ed — put the temporary clone **inside** the destination, where the emptiness re-check found it beside `.git` and refused. The acquisition worked on every shape except the one it was written for, and it was invisible because every recipe test bound the destination to an absolute path; the destination is resolved first now, which also lets every later mention be quoted, so a directory whose name holds a space reaches `cp` whole. The same round found the references' acquisition blocks carrying both starters' clone URLs on consecutive lines of a block whose own prose calls it one chain — run as written, the second clone fails on the destination the first just filled and its handler deletes the successful clone before anything is copied — while the skill body had always used a single placeholder; the blocks name one starter now. The confirmation rule was reconciled with the ruling it predates: it exempted the pristine commit as landing "on a directory this skill just created", which the preserving acquisition made false on all three of its grounds, so that commit now says what it will stage and asks, in every mode. And the initializer branch locks its Python project before handing it over — `uv init` writes a `pyproject.toml` and neither a lock file nor an environment, and `pipelex-integrate` reads an absent lock file as `pip install` into an active environment that `uv init` never made. +- **Two rules that were stated in one copy and executed from another.** The round before this one put the `-- .` pathspec on the pristine commit as well as on its staging and wrote that it is on both commands for a reason; `references/starters.md` kept the bare commit, so the rule held only where an agent does not read the command from — the same half-application that round was convened to fix, in the mirror direction, and its changelog line said both commits carried it when one did not. The initializers reference told the reader to run `git init -b main` "only if it did not initialize a repository itself", which is the inference the skill body forbids by name in favour of testing `git -C rev-parse --show-toplevel`, and the difference is the case that motivates the test: `uv init` initializes nothing when the parent already holds a project, and staging in a directory governed by an enclosing repository sweeps the user's whole worktree into this skill's commit. Separately, the lone-`.git` rule justified itself with "branch B runs `git init -b main` in the directory it is working in one step later" — which in that very case it must not, the user having just run it themselves. +- **A key cannot reach the transcript through the write half of the step that handles it.** The rule covered looking a key up and not putting one down: it sanctioned "an in-place edit", which for a file-editing tool means passing the literal value as a parameter, and a tool call's parameters are the transcript. The value now moves only through a shell that expands the variable itself, reading the env file back afterwards is refused by name, and confirming the write uses a file-side presence test that reveals nothing. The report says the value was taken from the environment **and not validated**, because a placeholder passes a presence test and fails the first run. The failure-table row that carried this guidance had an unescaped `|` inside a code span and rendered as four broken cells. - **Hosted-console connector instructions in the README**: the passage told readers to put an API key in the connector URL (`?api_key=plx_sk_...`) or an `Authorization: Bearer` header, a channel removed from the console in `@pipelex/mcp` 0.12.0 that no longer connects at all. It now says to add the connector by its plain URL and sign in with your Pipelex account, and to remove and re-add any connector registered the old way. - **The file factory no longer touches the user's own project.** Every `uv run` line in the skill and its references now passes `--no-project`. Without it `uv run` walks up from the working directory, finds the nearest project, and *syncs* it — so rendering a test PDF inside a checkout created a `.venv/` and wrote a `uv.lock` the user never asked for, in a repository the skill has no business modifying. `--no-project` resolves the ephemeral `--with` packages against nothing at all, which is what the recipes always meant; a test asserts every shipped runner line carries the flag. - **The PDF reference no longer offers a public URL that has stopped serving a PDF.** The documented last resort answered with an HTML error page rather than a document, so an agent following the recipe would have handed a method a file that is not a PDF at the exact moment it had already failed to render one. There is now no last resort for any format: the skill asks the user for their own file, and says plainly that it will neither fabricate one nor substitute a document the brief did not ask for. diff --git a/CLAUDE.md b/CLAUDE.md index fedc125..7d8fcca 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -36,6 +36,7 @@ templates/ # SOURCE OF TRUTH — all .j2 templates live here │ ├── pipelex-inputs/SKILL.md.j2 # inputs.json preparation (MCP-backed) │ ├── pipelex-synthetic-inputs/SKILL.md.j2 # File factory: render PDFs/PNGs/Office files from code, no AI (no MCP dependency) │ ├── pipelex-integrate/SKILL.md.j2 # Wire a method into a TS/Python codebase: codegen write arm, exclusions, sidecar, gate, typed call site (MCP-backed) +│ ├── pipelex-scaffold/SKILL.md.j2 # Front door to a new project: a starter through its own bootstrap, or the ecosystem's initializer (no MCP dependency) │ └── shared/ │ ├── frontmatter.md.j2 # Common YAML frontmatter (included by templates) │ ├── mthds-reference.md.j2 # MTHDS language reference (rendered per target) @@ -53,7 +54,8 @@ templates/ # SOURCE OF TRUTH — all .j2 templates live here skills/ # SOURCE OF TRUTH for static (non-templated) skill assets, copied verbatim into every target ├── pipelex-design/references/writing-mthds.md # MTHDS authoring reference ├── pipelex-synthetic-inputs/references/ # pdf.md, png.md, office.md — runnable recipes, executed by tests/recipes -└── pipelex-integrate/references/ # typescript.md, python.md, codegen-check.mjs — detection tables, call-site templates, the offline gate copied into TS projects +├── pipelex-integrate/references/ # typescript.md, python.md, codegen-check.mjs — detection tables, call-site templates, the offline gate copied into TS projects +└── pipelex-scaffold/references/ # starters.md, initializers.md — the two starters side by side, the ecosystem initializers pipelex/ # Claude prod plugin (generated, checked in) pipelex-codex/ # Codex plugin (generated, checked in) pipelex-vibe/ # Mistral Vibe target (generated, checked in; loaded via skill_paths; mcp/vibe-mcp.toml copied into ~/.vibe/config.toml) @@ -146,4 +148,4 @@ So there is nothing to enable — the bundled hook loads on its own (hooks are S The plugin imports nothing and requires no install. Validation rides on the vendored `check.mjs` bundle (wasm engine + `@pipelex/sdk` → hosted API) and, for the MCP-backed skills (`pipelex-design`, `pipelex-organize`, `pipelex-edit`, `pipelex-inputs`, `pipelex-integrate`), on the plugin-declared `pipelex-mcp` server (tools `mthds_validate` / `mthds_inputs_template`; `mthds_codegen`, whose write arm — `output_dir`, relative to the workshop's working directory — writes `pipelex-integrate`'s generated trees to disk so no artifact byte passes through the model; `mthds_prepare_inputs`, which uploads `pipelex-inputs`' file-bearing values to Pipelex storage and rewrites them to `pipelex-storage://` references so a run can reach them; plus the `mthds_run` family powering `pipelex-inputs`' closing offer to run; declared in the Claude and Codex manifests, and shipped on Vibe as the `mcp/vibe-mcp.toml` config fragment the user copies into `~/.vibe/config.toml`). The baked declaration is the **local workshop launcher** — `npx -y @pipelex/mcp@latest` over stdio, from the `[vars.mcp_server]` block in `targets/defaults.toml` — never a hosted URL: the hosted console is a connector users add in their host's own UI (see the README's "One install, one server" section and `docs/decisions.md`). Credential delivery: on Claude the manifest's `userConfig` prompts for the API key / base URL at enable time (keychain-stored) and the MCP entry spawns the `launch-pipelex-mcp.sh` wrapper, which receives them as `PIPELEX_PLUGIN_*` via `${user_config.*}` substitution and promotes each to `PIPELEX_API_KEY`/`PIPELEX_BASE_URL` **only when non-empty** — the canonical credential channel (required for Claude Desktop, which carries no shell env), with the non-empty guard keeping an unfilled option from shadowing a shell-exported key; injecting `PIPELEX_*` directly instead makes an empty option surface as a config-class `Unauthorized` that hard-stops every MCP-backed skill; on Codex the manifest forwards `PIPELEX_API_KEY`/`PIPELEX_BASE_URL` by name via `env_vars` because Codex whitelist-filters MCP spawn env; on Vibe the user writes the key into the fragment's `env` table, because Vibe also spawns stdio servers with a minimal environment and expands no variables in its config. Dev override: point `command`/`args` at a local checkout in `targets/defaults.toml` + `make build` on Claude; a same-named `[mcp_servers.pipelex]` entry in `~/.codex/config.toml` on Codex. -**`pipelex-synthetic-inputs` depends on none of that.** It is the second MCP-free skill after `pipelex-explain` — no tool, no key, no Pipelex service — and its only dependency is a Python it can reach: `uv` with ephemeral `--with` packages on the normal rung, and a venv it creates itself under `${XDG_CACHE_HOME:-$HOME/.cache}/pipelex-plugins/synth-venv` when `uv` is absent. Swapping the runner line is the *only* difference between the two rungs, and `tests/recipes` proves it by running real recipes through both. When neither rung is reachable the skill stops with the exact missing piece and, called from `pipelex-inputs`, returns no path so that one input is left unfilled rather than aborting the flow. Keep it out of the `MCP_SKILLS` tuple in `tests/unit/test_gen_skill_docs.py`. +**`pipelex-synthetic-inputs` depends on none of that.** It is the second MCP-free skill after `pipelex-explain` — no tool, no key, no Pipelex service — (`pipelex-scaffold` is the third: git, the starters' own bootstrap scripts and the ecosystem's initializers are all it uses, and it hands every project it creates to `pipelex-integrate`) and its only dependency is a Python it can reach: `uv` with ephemeral `--with` packages on the normal rung, and a venv it creates itself under `${XDG_CACHE_HOME:-$HOME/.cache}/pipelex-plugins/synth-venv` when `uv` is absent. Swapping the runner line is the *only* difference between the two rungs, and `tests/recipes` proves it by running real recipes through both. When neither rung is reachable the skill stops with the exact missing piece and, called from `pipelex-inputs`, returns no path so that one input is left unfilled rather than aborting the flow. Keep it and `pipelex-scaffold` out of the `MCP_SKILLS` tuple in `tests/unit/test_gen_skill_docs.py`. diff --git a/README.md b/README.md index 85e67a6..464f6b0 100644 --- a/README.md +++ b/README.md @@ -8,8 +8,8 @@ This is the plugin generation that pairs with the hosted Pipelex API and the (cl ## What's inside -- **Skills** — Pipelex skills for working with MTHDS bundles: `pipelex-explain` (read and explain a bundle), `pipelex-design` (design a method contract-first with a complexity-adaptive workflow: build a fully understood shallow graph directly as a coherent runnable bundle, or use validated signature-driven refinement for deep, uncertain, staged, or resumable work), `pipelex-organize` (regroup a construction-shaped or otherwise awkward layout into coherent module files — or a single file when the method is simple — proven equivalent through the MCP validation verdict; automatically follows `pipelex-design` only when the converged layout needs it), `pipelex-edit` (contract-preserving edits to an existing bundle — prompts, model references, mechanical renames — proven with a before/after MCP validation verdict; structural or contract changes route to `/pipelex-design`, which re-enters the affected region directly or through signatures according to its complexity), `pipelex-inputs` (prepare an `inputs.json` for a method — placeholder template, synthetic data, user files, or a mix — from the input template the MCP server projects; targets a local bundle or a registered catalog method by its `mt_…` id, uploads any local files to Pipelex storage so the run can reach them, and closes by offering to start the run through the MCP run tools), `pipelex-synthetic-inputs` (render the files a method needs when the user has none — PDFs through reportlab, PNGs through Pillow and matplotlib as charts, diagrams, scanned-looking pages or app screenshots, plus Word and Excel; `pipelex-inputs` delegates to it for every file-typed input), `pipelex-integrate` (wire a method into an existing TypeScript or Python codebase: the workshop's `mthds_codegen` writes drift-proof generated types into one directory per method, the skill excludes that tree from formatters, records a `sources.json` sidecar, wires the offline drift check into the project's gate on TypeScript, and writes one typed call site that runs the method through `@pipelex/sdk` or `pipelex-sdk`; a project made from a Pipelex starter keeps its own codegen harness). - `pipelex-synthetic-inputs` is the plugin's second MCP-free skill after `pipelex-explain`: it needs no API key and no Pipelex service. It renders files from code — no image-generation model, no hosted method — using only packages whose licences are compatible with MIT, and it installs them itself, through `uv`'s ephemeral environments or an isolated venv under your cache directory when `uv` is absent. Nothing is installed into your project, and installing a *tool* always asks first. Photographs and handwriting are deliberately out of scope: code cannot render either convincingly, and the skill asks for your own file rather than handing a method an imitation. +- **Skills** — Pipelex skills for working with MTHDS bundles: `pipelex-explain` (read and explain a bundle), `pipelex-design` (design a method contract-first with a complexity-adaptive workflow: build a fully understood shallow graph directly as a coherent runnable bundle, or use validated signature-driven refinement for deep, uncertain, staged, or resumable work), `pipelex-organize` (regroup a construction-shaped or otherwise awkward layout into coherent module files — or a single file when the method is simple — proven equivalent through the MCP validation verdict; automatically follows `pipelex-design` only when the converged layout needs it), `pipelex-edit` (contract-preserving edits to an existing bundle — prompts, model references, mechanical renames — proven with a before/after MCP validation verdict; structural or contract changes route to `/pipelex-design`, which re-enters the affected region directly or through signatures according to its complexity), `pipelex-inputs` (prepare an `inputs.json` for a method — placeholder template, synthetic data, user files, or a mix — from the input template the MCP server projects; targets a local bundle or a registered catalog method by its `mt_…` id, uploads any local files to Pipelex storage so the run can reach them, and closes by offering to start the run through the MCP run tools), `pipelex-synthetic-inputs` (render the files a method needs when the user has none — PDFs through reportlab, PNGs through Pillow and matplotlib as charts, diagrams, scanned-looking pages or app screenshots, plus Word and Excel; `pipelex-inputs` delegates to it for every file-typed input), `pipelex-integrate` (wire a method into an existing TypeScript or Python codebase: the workshop's `mthds_codegen` writes drift-proof generated types into one directory per method, the skill excludes that tree from formatters, records a `sources.json` sidecar, wires the offline drift check into the project's gate on TypeScript, and writes one typed call site that runs the method through `@pipelex/sdk` or `pipelex-sdk`; a project made from a Pipelex starter keeps its own codegen harness), and `pipelex-scaffold` (start a project where none exists — one of the Pipelex starter templates run through its own `bootstrap` skill, or the ecosystem's initializer — and hand it to `pipelex-integrate`). + `pipelex-synthetic-inputs` is the plugin's second MCP-free skill after `pipelex-explain` (`pipelex-scaffold` is the third — it needs only git, the starters' own scripts and the ecosystem's initializers): it needs no API key and no Pipelex service. It renders files from code — no image-generation model, no hosted method — using only packages whose licences are compatible with MIT, and it installs them itself, through `uv`'s ephemeral environments or an isolated venv under your cache directory when `uv` is absent. Nothing is installed into your project, and installing a *tool* always asks first. Photographs and handwriting are deliberately out of scope: code cannot render either convincingly, and the skill asks for your own file rather than handing a method an imitation. - **Hooks** — a CLI-free validation hook that checks `.mthds` files on edit (Claude/Codex `PostToolUse`, Mistral Vibe `post_tool`). On every target, lint and format run locally through a bundled WASM engine (offline, no credentials — the file is also auto-formatted in place), and full semantic validation calls the hosted Pipelex API when `PIPELEX_API_KEY` is set. Everything fails open: no Node → the hook no-ops; no key / API unreachable → only the validate stage is skipped. See [docs/hooks.md](docs/hooks.md). - **MCP server declaration** — on Claude Code and Codex the plugin declares the `pipelex-mcp` server as the **local workshop launcher** (`npx -y @pipelex/mcp@latest`, stdio; tools `mthds_validate` for bundle validation (its verdict carries the main pipe's signature, which `pipelex-integrate` types call sites from), `mthds_inputs_template` for input templates, `mthds_codegen` to project a method's concepts into typed code (`ts-zod`, `python-pydantic`, `python-structures`) and, with `output_dir`, write the tree to disk, `mthds_prepare_inputs` to upload a filled template's file values to Pipelex storage so a run can reach them, and the `mthds_run` family for durable runs — each takes submitted file contents or a registered method's catalog id (`mt_…`) as `method_id`, operating on the method's current stored content; by-id calls require an API key since the catalog is org-scoped), which the MCP-backed skills require. The harness spawns it automatically at session start, and it authenticates to the Pipelex API with the key from the plugin configuration (prompted at enable time on Claude Code) or, as a fallback, `PIPELEX_API_KEY` from your session environment — the same credential the hook uses. On Mistral Vibe, which has no plugin manifest, the target ships the same launcher as a config fragment instead: nothing is spawned until you add it to `~/.vibe/config.toml`, and the server takes its key only from that entry's `env` table, never from your shell (see [Mistral Vibe](#mistral-vibe)). Unlike the fail-open hook, the MCP-backed skills stop with a setup instruction when the tools are absent. The hosted console is never baked into the plugin — see [One install, one server](#one-install-one-server--workshop-vs-console) and [docs/decisions.md](docs/decisions.md). diff --git a/docs/build-targets.md b/docs/build-targets.md index 3119141..5c03ba0 100644 --- a/docs/build-targets.md +++ b/docs/build-targets.md @@ -39,7 +39,7 @@ scripts/gen_skill_docs.py renders .j2 templates with merged variables **`templates/`** holds all `.j2` source files. Never edit files in `pipelex/`, `pipelex-codex/`, or `pipelex-vibe/` directly — they are generated output. -**`skills/`** at the repo root (if present) holds only static per-skill assets (`references/` subdirectories) that are copied into every target. Several skills use it — `pipelex-design`, `pipelex-synthetic-inputs`, `pipelex-integrate` — and a reference need not be Markdown: `pipelex-integrate` ships `codegen-check.mjs`, a script the skill copies verbatim into TypeScript projects, so a reference edit is followed by `make build` and, for a script, by running it (`node --check` at the least). **`pipelex/`**, **`pipelex-codex/`**, and **`pipelex-vibe/`** are generated output directories (build artifacts checked into git). +**`skills/`** at the repo root (if present) holds only static per-skill assets (`references/` subdirectories) that are copied into every target. Several skills use it — `pipelex-design`, `pipelex-synthetic-inputs`, `pipelex-integrate`, `pipelex-scaffold` — and a reference need not be Markdown: `pipelex-integrate` ships `codegen-check.mjs`, a script the skill copies verbatim into TypeScript projects, so a reference edit is followed by `make build` and, for a script, by running it (`node --check` at the least). **`pipelex/`**, **`pipelex-codex/`**, and **`pipelex-vibe/`** are generated output directories (build artifacts checked into git). ## Target configuration diff --git a/docs/decisions.md b/docs/decisions.md index d04f70c..aed6725 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -159,9 +159,9 @@ So file synthesis became **`pipelex-synthetic-inputs`**, a skill of its own. `pi - **Prose recipes, guarded by execution.** The reference files carry complete runnable blocks the agent copies and adapts, rather than shipped generator scripts — the content of a synthetic file differs every time, and a parameterized script would become a JSON DSL for documents. What was missing from the predecessor is the guard, and it is the reason the old image recipe rotted unnoticed: `tests/recipes` extracts every block, runs it, and checks the file's magic bytes, a PDF's page objects and a PNG's declared pixel size. Opt-in (`make test-recipes`), since a cold `uv` cache downloads packages. A router `SKILL.md` plus one reference per format keeps every invocation from paying for every format. - **The skill owns its environment, and asks before installing a tool.** A ladder run once per invocation: `uv` with ephemeral `--with` packages first; failing that, a venv the skill creates under `${XDG_CACHE_HOME:-$HOME/.cache}/pipelex-plugins/synth-venv` and fills itself. Swapping the runner line is the only difference between the rungs, which `tests/recipes` proves by running real recipes through both. Nothing is installed into the user's project; installing a **tool** (`uv` itself, a system package) always asks first, in every mode, because a package in a cache and a binary on the machine are different commitments. When no rung is reachable the skill stops with the exact missing piece and the command that supplies it — and, called from `pipelex-inputs`, returns no path so that one input is left unfilled with its reason in the report, rather than aborting the whole inputs flow. -## The method reaches the code — `pipelex-integrate` (2026-09-06) +## The method reaches the code — `pipelex-integrate` and `pipelex-scaffold` (2026-09-06) -Every skill before this one acts on `.mthds` files; none touched the codebase that calls the method, so the plugin stopped helping at the moment a user was happy with a method. `pipelex-integrate` closes that gap, decided with Louis on 2026-08-30 and extended on 2026-09-06, the session that ruled harness deference and added it to the design as a further box. The design is `wip/pipelex-integrate/design.md`, whose decision boxes carry their own dates; the tracker is `plan.md` beside it. +Every skill before these acts on `.mthds` files; none touched the codebase that calls the method, so the plugin stopped helping at the moment a user was happy with a method. Two skills close that gap, decided with Louis on 2026-08-30 (the integrate design's ten boxes) and 2026-09-06, the session that widened the campaign to greenfield projects with four rulings and ruled harness deference into the integrate design as a further box. The designs are `wip/pipelex-integrate/design.md` and `wip/pipelex-integrate/scaffold-design.md`; the tracker is `plan.md` beside them. - **`pipelex-integrate` writes a complete typed call site, and stops there.** One module per method — an async function typed from the pipe's signature, running through the SDK's self-healing lifecycle call and narrowing `main_stuff` through the generated binder or model — plus exactly one shared helper, the client factory; no tests, routes or UI. The name `pipelex-codegen` was rejected: tools are the contract and skills are the manual, named after user tasks, and codegen is one step of integrating. - **The write arm is the only arm.** Every `mthds_codegen` call passes `output_dir`; a refused or failed write is a refusal, never a fallback to writing the returned bytes from the conversation — a re-emitted artifact is one trailing newline from a broken stamp. Generated files are never edited, formatted or linted; the tooling exclusions go in before the tree exists; one directory per method; orphans are reported and never deleted. @@ -170,16 +170,21 @@ Every skill before this one acts on `.mthds` files; none touched the codebase th - **The drift gate is asymmetric, and says so.** TypeScript gets `references/codegen-check.mjs` over `@pipelex/sdk`'s `runCodegenCheck`, wired into the project's existing gate. A `python-pydantic` consumer gets no gate, because the Python SDK has no offline check yet and adding the `pipelex` runtime for one would reverse what the target means; the report says so, and the editing skills' notice is the guard meanwhile. - **No wire-null helper.** The design carried a schema-guided `wireOutput` helper against the ts-zod `.optional()` defect; the emitter fix (`.nullish()`) shipped in pipelex v0.56.0 before the skill did, and the helper — lossy once the emitter is fixed — is never written. - **A project that owns a codegen harness keeps it.** A starter-derived project regenerates, checks and scaffolds through its own scripts; the skill defers to them and never writes a second layout beside the first. This is the personas split: the mechanism may move into the SDK, the policy stays with the consumer. -- **References carry the language detail, and one of them is a script.** `pipelex-integrate` ships `typescript.md`, `python.md` and `codegen-check.mjs` — the first non-Markdown reference, copied byte for byte into every target and then into users' TypeScript projects, so a change to it is verified by running it, not by rendering it. +- **`pipelex-scaffold` is the front door, with two branches and no templates of its own.** A project comes from one of the two GitHub-template starters, run through the clone's own `bootstrap` skill (read from its `SKILL.md`, never reimplemented here), or from the ecosystem's initializer when the user wants their framework. Cookiecutter and copier were rejected: the starters are living apps with CI and end-to-end tests that a Jinja-ified template could not keep, and the rename is already a deterministic script with a dry run. The skill acquires with a fresh-history local clone by default (what the template button produces) or `gh repo create --template` after confirmation, makes exactly one commit — the pristine template, so the Python bootstrap's `git mv` works and the rename is a reviewable diff — fills the key only from the shell environment, adds no SDK dependency, and is the plugin's third MCP-free skill. Whether a CLI front door for non-agent users is still wanted is a separate open decision in the workspace ledger. +- **Two skills, not one.** The integrate design is large and the trigger families differ ("use this method in my app" against "start a new project"); the greenfield tail is integrate anyway, so the hand-off chain is the plugin's existing pattern. +- **References carry the language detail, and one of them is a script.** `pipelex-integrate` ships `typescript.md`, `python.md` and `codegen-check.mjs` — the first non-Markdown reference, copied byte for byte into every target and then into users' TypeScript projects, so a change to it is verified by running it, not by rendering it; `pipelex-scaffold` ships `starters.md` and `initializers.md`. -## What the integrate dogfood changed (2026-09-12) +## What the integrate and scaffold dogfood changed (2026-09-12) -The skill was run as cold headless sessions against a local `pipelex-mcp` build of `dev`, in scratch projects and in scratch copies of both starters. The rules below are what those runs changed; the scenarios and their verdicts are in `wip/pipelex-integrate/plan.md`, and the design carries the same amendments. +Both skills were run as cold headless sessions against a local `pipelex-mcp` build of `dev`, in scratch projects and in scratch copies of both starters. The rules below are what those runs changed; the scenarios and their verdicts are in `wip/pipelex-integrate/plan.md`, and the designs carry the same amendments. - **The validate verdict's `main_pipe` is absent for three reasons, and the skill asserts none of them.** The entry pipe is the API's `default_pipe_ref` — for a published package, its `METHODS.toml` entry rather than the bundle's own declaration, which is why the call site is typed and run against `main_pipe.pipe_ref`. The signature is omitted whole when nothing settles an entry pipe, when the pipe's contract does not narrow, or when the workshop predates the signature. The skill named only the first and the third, and told a by-ref or by-id source to blame an old workshop. - **An occupied generated directory is refused even when the user names it.** Every method of a target emits the same file names, so generating a second method into another method's directory overwrites it and reports an empty `orphans[]` — a clean-looking generation. The guard is the sidecar: a directory is this method's only when a `sources.json` there names it, a lock with no sidecar included. - **Containment is read before the first write, not after an error.** A path inside the workshop's working directory is legal wherever it points, so a harness launched beside the project accepts a write into the wrong tree; a run that hit this moved the tree across afterwards and left the project with a refresh that fails the same way every time. The skill now reads the path from the workshop's working directory first, and never moves a tree into place. -- **A file the skill does not own is never cleared, and never offered for clearing.** Integrate met a hand-written file at an artifact path and offered to delete it. The answer is another directory. +- **A file the skill does not own is never cleared, and never offered for clearing.** Integrate met a hand-written file at an artifact path and offered to delete it; scaffold met a non-empty target directory and offered to move its contents aside and merge them back. The answer in both cases is another directory. +- **A directory holding nothing but `.git` reads as empty; everything else still refuses.** (Ruled 2026-09-13, `L-260912-724b71`.) The non-empty refusal treated any existing entry as occupancy, which made the skill refuse a state it produces itself: `mkdir my-app && cd my-app && git init` is an ordinary way to arrive, branch B runs `git init -b main` in the directory it is working in one step later, and `references/initializers.md` already documents `uv init --package --no-workspace .` for the empty-"here" case. A lone `.git` is now read as empty. **The cruft list was declined in the same ruling** — `.DS_Store`, `.idea/`, `.vscode/` and `Thumbs.db` go on refusing until a real report names one — because a list that grows by guesswork is how this rule drifts back into the agent judging which of a user's files matter, which is the thing the refusal exists to forbid; so the exception is written as one directory entry by name and never as a predicate over ignorable files. The separate rule that the skill never *offers* to clear anything is untouched: a directory read as empty is never cleared either way. +- **Branch A acquires beside a directory that already holds a repository, so the lone-`.git` exception means one thing on both branches.** (Ruled 2026-09-13, `L-260913-f28d9d`.) The ruling above is implementable on the initializer branch, where `uv init` accepts such a directory, and unimplementable on the starter branch, whose first command on the chosen directory is `git clone` — which refuses any destination already holding a `.git`. The skill therefore admitted a directory it could not populate, and the founder's own motivating case dead-ended for anyone who wanted the opinionated starter rather than the initializer. **Scoping the allowance to the initializer branch was rejected**: the exception exists because `mkdir my-app && cd my-app && git init` is an ordinary way to arrive, which is not specific to initializers, so that user has exactly the same claim on the starter and would have met the refusal the exception was written to remove. Branch A now clones into a `mktemp` path beside the directory, discards the template's history there, re-reads the directory and copies the template in. The properties that make it safe are executed rather than asserted, in `tests/unit/test_pipelex_scaffold_skill.py`: **no `rm -rf` ever addresses a path under the chosen directory**, because the discard is spent on the temporary path before anything moves; **`cp -R "$tmp"/. "$dir"/` carries the entries beginning with a dot**, which `mv "$tmp"/*` drops while exiting `0`; and **the `ls -A` line admits exactly one entry**, `.git`, which the clone no longer has, so a collision is impossible rather than unlikely and anything else stops the run with nothing copied. The end state is the one the default recipe reaches — the template in place, no template history, no Pipelex remote — reached by a different route, and nothing is initialised because the repository is already the user's. **The destination is resolved to an absolute path before the temporary path is derived from it** (added in review round 2): the chosen directory is usually spelled `.`, since the user is standing in the directory they just `git init`-ed, and `dirname .` is `.` — so deriving the sibling from the spelling made the temporary directory a *child*, which the `ls -A` line then saw beside `.git` and refused. The recipe served every shape except the one the ruling was written for, and every recipe test bound `` to an absolute path, which is why it took an execution test spelled `.` to show it. +- **A runtime behind a version manager is not a missing toolchain.** Scaffold, on a `PATH` without `node`, found the machine's `nvm` and carried on — which installs nothing and is the useful answer. It now checks `nvm`, `fnm`, `volta`, `asdf` and `mise`, says which it used, and stops only when no runtime can be reached. - **One upstream defect is named rather than worked around.** The `ts-zod` emitter writes `binder.ts`'s sibling import with no file extension, which a plain Node ESM project rejects at type-check and at runtime while a bundler resolution accepts — which is why the JS starter never met it. Filed as `L-260912-857a5a` against `pipelex`. The tree is stamped and hashed, so the skill reports it, never patches the file, never drops the tree from the type checker, and leaves a change of `moduleResolution` to the user. ## Orphans alone do not end the run — `orphans[]` non-empty with `drifts[]` empty (2026-09-13) diff --git a/pipelex-codex/skills/pipelex-design/SKILL.md b/pipelex-codex/skills/pipelex-design/SKILL.md index 6bff6b6..c83a35c 100644 --- a/pipelex-codex/skills/pipelex-design/SKILL.md +++ b/pipelex-codex/skills/pipelex-design/SKILL.md @@ -182,7 +182,7 @@ After the gate: 1. **Organize only when the layout needs it.** A direct result that is already coherent skips `/pipelex-organize`. A converged stepwise result normally invokes it automatically because one-definition-per-file construction history and satisfied headers need regrouping. A naturally coherent result in either mode does not take an organization round trip solely for process compliance. 2. **Project the input schema.** Call `mthds_inputs_template` with the final whole-bundle `files` submission plus `explicit: false`. Show the returned compact template, but **do not save it as `inputs.json`** — input preparation belongs exclusively to `/pipelex-inputs`. 3. **Present the flow.** Point to the interactive method graph where the host rendered the valid verdict's view; in terminal hosts, present a concise text flow of the final structure. -4. **Hand off inputs — and the code.** Suggest preparing real inputs with `/pipelex-inputs`. Then, when the workspace holds a codebase (a `package.json` or a `pyproject.toml`), say that `/pipelex-integrate` wires the method into it with generated types and a typed call site. +4. **Hand off inputs — and the code.** Suggest preparing real inputs with `/pipelex-inputs`. Then, when the workspace holds a codebase (a `package.json` or a `pyproject.toml`), say that `/pipelex-integrate` wires the method into it with generated types and a typed call site; when it holds none and the user wants an application around the method, `/pipelex-scaffold` creates one and hands it to `/pipelex-integrate`. > **NEVER write `inputs.json` manually.** If the user provides files, paths, or wants to run with real data, invoke `/pipelex-inputs` — it handles the template, path resolution, placeholder formatting, and file copying. diff --git a/pipelex-codex/skills/pipelex-integrate/SKILL.md b/pipelex-codex/skills/pipelex-integrate/SKILL.md index d19fab6..4e4fc74 100644 --- a/pipelex-codex/skills/pipelex-integrate/SKILL.md +++ b/pipelex-codex/skills/pipelex-integrate/SKILL.md @@ -1,6 +1,6 @@ --- name: pipelex-integrate -description: Wire an MTHDS method into a Python or TypeScript codebase with generated, drift-proof types and one typed call site that runs it through @pipelex/sdk or pipelex-sdk. Use when the user says "use this method in my app", "call this from my code", "generate types for this method", "wire the method into my project", "add this pipeline to my service", "typed client for this method", "integrate the method", "refresh the generated types", "regenerate the types", "the types are stale", or wants application code that runs a .mthds method — from a local bundle, a catalog id (mt_…) or a published method_ref address. Also the refresh path after a bundle edit. Not for authoring or editing the method itself (/pipelex-design, /pipelex-edit), and not for a workspace with no project in it yet, where the project has to exist first. +description: Wire an MTHDS method into a Python or TypeScript codebase with generated, drift-proof types and one typed call site that runs it through @pipelex/sdk or pipelex-sdk. Use when the user says "use this method in my app", "call this from my code", "generate types for this method", "wire the method into my project", "add this pipeline to my service", "typed client for this method", "integrate the method", "refresh the generated types", "regenerate the types", "the types are stale", or wants application code that runs a .mthds method — from a local bundle, a catalog id (mt_…) or a published method_ref address. Also the refresh path after a bundle edit. Not for authoring or editing the method itself (/pipelex-design, /pipelex-edit), and not for a project that does not exist yet (/pipelex-scaffold). --- @@ -52,7 +52,7 @@ Automatic by default: state the target, the destination and the generator in one - **A published address** — `method_ref: "github.com//[/]@"`. The tag is the pin: a `method_ref` **without a tag** floats and is refused for a committed integration; ask for the tag. - **A catalog id** — `method_id: "mt_…"` (resolve a name through `mthds_list_methods` when present). The catalog is unversioned: an edit to the stored method silently invalidates committed types with nothing offline to detect it. Say so in one line, recommend committing the source or publishing an address, and proceed only on the user's say-so. -**The project** is the nearest directory holding a `package.json` or a `pyproject.toml` (or `setup.py` / `requirements.txt`) above the working area. A workspace holding several (a monorepo, a full-stack repo) is a question — which app? — never a guess. **No project at all** → this is not an integration yet: say so and stop, because there is nothing to wire the method into. Creating the project is the user's own step — a starter, their framework's initializer, their own layout — and integration begins once it exists. +**The project** is the nearest directory holding a `package.json` or a `pyproject.toml` (or `setup.py` / `requirements.txt`) above the working area. A workspace holding several (a monorepo, a full-stack repo) is a question — which app? — never a guess. **No project at all** → this is not an integration yet: offer the `pipelex-scaffold` skill (open `../pipelex-scaffold/SKILL.md`), which creates one and hands it back here. Then look for a **codegen harness**: a `codegen` script in `package.json` or a `codegen` Makefile target, a `sources.json` carrying a `derived` map, `docs/codegen.md` or `docs/add-method.md`, a `methods/` directory beside `src/generated/` or `/generated/`. Either of the first two decides; the rest only corroborate. If the project has one, follow [A project that owns a codegen harness](#a-project-that-owns-a-codegen-harness) from here. diff --git a/pipelex-codex/skills/pipelex-scaffold/SKILL.md b/pipelex-codex/skills/pipelex-scaffold/SKILL.md new file mode 100644 index 0000000..6906aa2 --- /dev/null +++ b/pipelex-codex/skills/pipelex-scaffold/SKILL.md @@ -0,0 +1,205 @@ +--- +name: pipelex-scaffold +description: Start a new project that will call MTHDS methods through Pipelex, in TypeScript or Python — from one of the Pipelex starter templates or from the ecosystem's own initializer — and hand it to /pipelex-integrate. Use when the user says "start a new project with Pipelex", "I have a method and need an app around it", "create a Next.js app that runs my method", "set up a Pipelex project from scratch", "new Python CLI for this method", "which starter should I use", "bootstrap a Pipelex project", or wants a codebase where none exists yet. Also use when the user is standing in a freshly cloned pipelex-starter-js or pipelex-starter-python that has not been renamed yet — this skill runs the template's own bootstrap for them. Not for adding Pipelex to code that already exists: that is /pipelex-integrate. + +--- + +# Scaffold a project for Pipelex methods + +Give a user who has no project yet a project that is ready for `/pipelex-integrate`. This skill has exactly two branches and carries no templates of its own: + +- **One of the Pipelex starters** when the user wants the opinionated shape: `pipelex-starter-js` for a web app whose forms are rendered from the methods' own contracts, `pipelex-starter-python` for a CLI or service that runs methods in the three execution modes. You acquire the template, commit it once as it came, then run the clone's **own** `bootstrap` skill — the rename logic lives in the starters and is never reimplemented here. +- **The ecosystem's own initializer** when the user wants their framework or a minimal project: `uv init --package`, `npm create next-app@latest`, `django-admin startproject`, whatever the framework documents. You run it; you never assemble a project by hand. + +Both branches end the same way: an env file that follows the starters' convention, one pristine commit that makes everything after it reviewable, and the hand-off — to `/pipelex-integrate` when a method exists, to `/pipelex-design` first when none does. + +**What this skill is not.** Not a template engine (no cookiecutter, no copier, no framework matrix of its own), not a bootstrap (the starters own theirs), not a runner or a dev-server launcher, not a deployer. It needs no MCP tool and no API key: git, the starters' scripts and the ecosystem's initializers are all it uses. + +## Choosing the branch + +A cheap, reliable signal decides; an inconclusive one asks one question; nothing is guessed twice. + +| Question | Signals, in order | When inconclusive | +|---|---|---| +| **Language** | the user's word; the language the method's consumer is written in; a framework the user named | ask | +| **Which branch** | a **named framework** the starters do not carry (FastAPI, Django, Express, Hono, a plain library, a Lambda) → the initializer; **"minimal"**, **"no demo code"**, **"just a project"** → the initializer; a **web app people use in a browser**, forms, an upload flow → the JS starter; a **CLI, script, batch job, worker or service** in Python → the Python starter | one question offering the matching starter first, saying what it brings (durable runs, forms or CLI modes, codegen wiring, CI, its own `release` skill) and what it costs (demos to keep as references or to strip) | +| **Where** | the directory the user named; **"here"** when the working directory is empty — **and a directory whose only entry is `.git` is empty for this rule**, because `mkdir my-app && cd my-app && git init` is an ordinary way to arrive here and a repository the user made is not work of theirs to write over: branch B runs its initializer in the directory as it stands and leaves that repository alone — it does **not** re-run `git init` there, since Step 3's test finds `` is already its own repository — while branch A acquires beside it and moves in, so the repository already there goes on standing either way (Step 2); else a kebab-case directory named after the project | ask; never write into a directory that exists and is not empty, and never offer to move, delete or merge what it holds to make room — the answer is another directory. **A lone `.git` is the only entry that does not make a directory non-empty, and that is a ruling about `.git` and nothing else** — not a class of files you may decide to overlook. Every other entry still refuses, `.DS_Store`, `.idea/`, `.vscode/` and `Thumbs.db` included: judging which of a user's files matter is the thing this rule exists to forbid, and a list that grows by guesswork is how it would come back. The rule is about **a directory you are creating a project in**, which is why the fresh-clone shortcut below is not an exception to it: there the project is already there and you are finishing it, not writing over someone's work | +| **GitHub or local** | the user asked for a GitHub repository → `gh repo create --template`, after confirmation; otherwise a local clone with fresh history | local | + +**The fresh-clone shortcut.** A starter clone already in the working directory that has not been bootstrapped — `package.json` still says `pipelex-starter-js`, or `pyproject.toml` still says `name = "piper"` — **and whose `origin` does not point at `Pipelex/pipelex-starter-…`** is branch A entered at step 4: acquisition already happened, so go straight to running the clone's bootstrap. Do not clone again. + +[references/starters.md](references/starters.md) compares the two starters and carries every command below; [references/initializers.md](references/initializers.md) carries the initializers. + +## Mode + +Automatic by default, with the plugin's usual rules: an explicit user signal wins ("just do it" → automatic; "walk me through" → interactive); a genuinely ambiguous branch is one question, asked once; a request that gave every input up front proceeds without re-asking. Two things always confirm, in every mode: **`gh repo create`**, because it creates a repository on GitHub, and whatever the clone's bootstrap skill confirms on its own account. The pristine commit does not need confirmation **on a directory this skill created** — it holds the template as it came, and no user content is at stake. **The acquisition into a directory that already held a repository is the exception**, and it is a third thing that always confirms: there the commit lands on the user's branch, on top of their history, and `add -A -- .` records whatever their worktree was already showing along with the template (Step 3). None of the three grounds above holds, so state what will be staged and what it will land on, and ask — in every mode. + +## Branch A — one of the starters + +### Step 1: Prerequisites + +Check before touching anything, and **stop** on a missing piece with the exact thing missing and the starter README's own line about it. Never install a toolchain — but a runtime the machine already has and only the `PATH` is missing is not a missing piece: when `node` or `uv` is absent while a version manager on the machine carries one (`nvm`, `fnm`, `volta`, `asdf`, `mise`), activate it for this work and say in the report which one you used and that the user's own shell may not have it. Stop only when no usable runtime can be reached that way. + +**Activating it means resolving it to an absolute path, not sourcing a shell.** Your shell state does not survive from one command to the next — each one starts again from the user's profile, which is the profile that did not have the runtime — so `. nvm.sh` or `eval "$(fnm env)"` in one call buys nothing in the next. Resolve the binary once (`ls "$NVM_DIR"/versions/node/*/bin/node`, `volta which node`, `mise which node`, `asdf which node`, `fnm exec --using= -- which node`), keep that directory, and prefix **every** later command with it — `PATH=":$PATH" …` — the clone's bootstrap and its `make all` / `make agent-check` included, because those are separate commands too. Verify the runtime answers under that prefix **before** step 2, so a machine you cannot actually reach stops while nothing has been created; discovering it at step 4 has already spent the pristine commit. + +Three things this clause does not license. **A shim is not a runtime**: `asdf` and `mise` put a `node` on the `PATH` that exists and then fails with "no version set", so the test is that `node --version` *answers*, not that the binary resolves — and that case is a stop, not a manager to activate. **The floor still applies**: a manager holding Node 18 does not satisfy the starter's `engines` floor, and "a runtime the machine already has" never means a version below it. And **`volta` and `mise` install on first use** — `volta run`, `mise x` and `mise use` will fetch a version they do not have — which is the toolchain install this step forbids: use only a version the manager already holds, and stop rather than let it download one. Note too that `nvm`, `fnm` and `volta` manage Node alone and can never supply `uv`. + +- **JavaScript**: Node at or above the floor the starter's `package.json` `engines` field names (`node --version`; 22.12 at writing — the SDK is ESM-only and the starter's e2e specs `require()` it), and `npm`. +- **Python**: `uv` on the PATH (the starter installs and locks with it) and a Python inside the starter's `requires-python` range that `uv python find` can see (3.11 to 3.14 at writing). +- **Both**: `git`. The GitHub form also needs `gh` authenticated — `gh auth status`. + +### Step 2: Acquire the template + +**Local, the default.** Clone shallow, read the template's identity, then detach from it: + +```bash +git clone --depth 1 https://github.com/Pipelex/.git || exit +git -C rev-parse HEAD # the template SHA, for the commit message +# the template version: package.json "version" (JS) or pyproject.toml version (Python) +rm -rf /.git && git -C init -b main +``` + +The `|| exit` on the clone is not decoration: the line below it deletes a `.git` directory, and if the clone never ran — a network failure, or `` already existing — that `rm -rf` finds whatever `.git` is actually at that path. On a directory the skill just created it destroys nothing; on a repository of the user's it destroys their history irrecoverably. Run the destructive line only behind a clone that succeeded, and never type it on a path you have not just created. + +The clone's `.git` is removed on purpose: it is the template's history and remote, and leaving it would make `git status` and a future `git push` belong to Pipelex's template rather than to the user's project. This is exactly what GitHub's "Use this template" button produces — a copy with no history and no remote — and it is why the starters' READMEs tell humans not to clone directly. Fresh history is how you honour that. + +**Local, into a directory that already holds a repository.** The "Where" rule reads a directory whose only entry is `.git` as empty, and `git clone` cannot serve it: git refuses any destination already holding a `.git` and stops with `destination path '' already exists and is not an empty directory`. So acquire **beside** the directory and move in. The end state is the one the default recipe reaches — the template in ``, no template history, no Pipelex remote — and the repository the user made goes on standing instead of being replaced: + +```bash +dir=$(cd && pwd) || exit 1 +tmp=$(mktemp -d "$(dirname "$dir")/.pipelex-starter-XXXXXX") || exit 1 +git clone --depth 1 https://github.com/Pipelex/.git "$tmp" || { rm -rf "$tmp"; exit 1; } +git -C "$tmp" rev-parse HEAD # the template SHA, for the commit message +# the template version: package.json "version" (JS) or pyproject.toml version (Python) +rm -rf "$tmp/.git" || { rm -rf "$tmp"; exit 1; } +[ "$(ls -A "$dir")" = ".git" ] || { rm -rf "$tmp"; exit 1; } +cp -R "$tmp"/. "$dir"/ || { rm -rf "$tmp"; exit 1; } +rm -rf "$tmp" +``` + +**The first line resolves the destination, and that is what keeps the temporary path a sibling rather than a child.** `` is very often `.` here: `mkdir my-app && cd my-app && git init` is the "Where" rule's own account of how a user arrives at a directory holding nothing but `.git`, and they then ask for the project *here*. `dirname .` is `.`, so deriving the parent from the spelling would put the temporary directory **inside** the destination, where the `ls -A` line below finds it sitting beside `.git` and refuses — every time, on exactly the case this section exists to serve. Resolving to an absolute path first also pins the destination for the rest of the chain, so no later line can be re-read against a working directory that has moved, and it is what lets every mention below be quoted: a name with a space reaches `cp` whole instead of arriving as two arguments. + +**No `rm -rf` here ever addresses a path under ``, and that is the ordering rather than a coincidence.** The template's history is discarded while the clone is still at a path `mktemp` made for this one command, so the destructive line is spent before anything moves: the only two paths any delete is pointed at are `"$tmp/.git"` and `"$tmp"`. Compare the default recipe, where the same line runs on `` itself and `|| exit` is the whole thing standing between it and a user's history. Here there is nothing for a guard to hold, which is what makes this the form you may aim at a directory holding somebody's repository. + +**`cp -R "$tmp"/. "$dir"/`, and never `mv "$tmp"/* "$dir"/`.** The glob matches no entry beginning with a dot, so the naive move leaves `.gitignore`, `.env.example`, `.github/` and `.claude/` behind, exits `0`, and the line after it deletes the temporary directory they are still sitting in — a starter arriving without its `.gitignore`, whose pristine commit then swallows `node_modules/`, reported as a success. The trailing `/.` copies the directory's *contents*, dotfiles included, with no shell globbing involved at all. Confirm it with `ls -A "$dir"` after the copy rather than trusting the form. + +**The `ls -A` line is the "Where" rule read again, against the copy.** It is not the decision — the "Where" question settled that — it is the last look before anything lands, put next to the copy so nothing can change between the two. It admits exactly one entry, `.git`, which the clone has not had since the line above: a collision is therefore impossible rather than merely unlikely, and the template can only add to the directory. Anything else — `.git` beside a file of the user's, a `.DS_Store`, a `README.md` they wrote — stops here with nothing copied, the temporary path removed and the directory as it was. A discarded shallow clone is the cheap half of that trade. Nothing of the user's is overwritten, moved or deleted to make room, here or anywhere. + +**The chain goes out as one command.** The guards hold only inside one shell — the same reason the `|| exit` above is load-bearing, stated in full in [references/starters.md](references/starters.md) — and split across separate calls this one loses its cleanup too, leaving the temporary directory beside the user's project with no line left to remove it. + +**Nothing is initialized here.** The default recipe ends `git init -b main` because it has just deleted the only repository at that path. This one ends on the user's repository, their branch and their remote, which is the whole point of taking the long way round. + +**GitHub, on request.** When the user asked for a repository on GitHub: + +```bash +gh repo create / --template Pipelex/ --private --clone +``` + +`--clone` takes no destination argument: it clones into `./` under the current working directory, so for this form `` **is** `` — either choose the repository name to match the directory the "Where" question settled, or rebind `` to `./` before step 4, because every step after this one addresses `` literally. Visibility is the user's call: ask, default `--private`. Creating a repository on GitHub is outward-facing, so **state the exact command and confirm before running it**, in every mode. GitHub writes the initial commit itself — skip step 3 and continue at step 4. If `gh` is absent or not authenticated, fall back to the local clone and say the repository can be created later with `gh repo create --source .`. + +Both forms take the template's default-branch head. Do not offer a release tag unless the user asks for one. + +### Step 3: Commit the pristine template — exactly once + +```bash +git -C add -A -- . && git -C commit -m "Start from Pipelex/ ()" -- . +``` + +This is the **one commit this skill makes**, and it is load-bearing twice over: the Python starter's bootstrap renames the package directory with `git mv`, which refuses a path git does not track, and a committed baseline is what turns the bootstrap's edits into a diff the user can read before committing them. Nothing of the user's is in it — it is the template as it came. One qualification on the acquisition into a directory that already held a repository: the commit lands on the user's branch, on top of their history rather than opening a new one, and `add -A -- .` also records whatever deletion their worktree was already showing — their own pending change and not one this skill made, so name it in the report instead of undoing it. That is the commit the Mode section sends back for confirmation, and this is what to put in front of the user: `git -C status --short` before staging says what will ride along, and it is the difference between a baseline commit and a line in their history that says "Start from Pipelex/…" over a change they made. + +### Step 4: Run the clone's own bootstrap + +Read `/.claude/skills/bootstrap/SKILL.md` and follow it as written. The project's skills are not loaded in this session — it began elsewhere — so read the file; do not look for a `/bootstrap` command. Run every command it gives from inside the project directory (`cd && …`, or `-C `), because that skill assumes it is standing in the repo root. + +Feed it what the conversation already holds — the project name, title, description, author, repository URL, license — so that it asks once, consolidated, for whatever is left, exactly as its own Step 2 says. It dry-runs, previews, runs, re-syncs the lock file, runs the project's own checks (`make all` on JS; `make agent-check` and `make agent-test` on Python), and removes itself. Its rules stand unchanged: it never commits, its edits stay uncommitted for the user's review (the Python renames are staged by `git mv`, which its skill explains), and a red check is fixed, never skipped. **Add nothing to that procedure and reimplement none of it.** If the clone carries no bootstrap skill — a future template dropped it — follow the README's "manual equivalent" list and say that the template changed. + +### Step 5: The env file + +```bash +cp -n /.env.example /.env.local # JS: Next.js reads .env.local +cp -n /.env.example /.env # Python: python-dotenv reads .env +``` + +`-n` because this is the one step that can destroy something of the user's. On the fresh-clone shortcut the directory is one they were already working in, and a plain `cp` would overwrite an `.env.local` they had filled with their own key — the skill's whole posture is that nothing of the user's is ever cleared, and an env file is the most expensive thing in the tree to lose. An existing env file is left exactly as it is; read whether it already carries a key with the file-side test below, and say in the report that you kept theirs. + +Fill `PIPELEX_API_KEY` **from the shell environment when it is set there**, and leave it empty otherwise, telling the user where a key comes from (`app.pipelex.com`) and that this file is where it goes. **Never print a key, and never ask for one in the conversation.** Test for it without printing it — `[ -n "${PIPELEX_API_KEY:-}" ] && echo set || echo unset`. + +**The value moves only through a shell that expands the variable itself, and never through you.** A redirection is safe precisely because the shell does the expanding and only the variable's *name* is transcribed. **Append only when the file does not already carry a key**, which is the other half of the `-n` above: `grep -q '^PIPELEX_API_KEY=.\+' /.env.local || printf 'PIPELEX_API_KEY=%s\n' "$PIPELEX_API_KEY" >> /.env.local`. On the fresh-clone shortcut the file is one the user may have filled themselves, and an unconditional append puts a second assignment *after* theirs — every dotenv reader resolves a repeated name to the later line, so their working key is silently replaced by whatever the shell happened to export, while the report tells them you kept theirs. A file-editing tool is the one form that cannot be made safe, whatever it is called — it takes a **literal** string, so you would have to know the value to pass it, and a tool call's parameters are the transcript. So: no file-editing tool on a line carrying the key, no `env | grep PIPELEX`, no `echo $PIPELEX_API_KEY`, no command substitution in a message, and **no reading the env file back** once written — `cat .env.local`, a `grep` over it, or opening it to check your work is the reflex after writing and the first move when a later step fails, and it puts the key in the transcript just as surely. To confirm the write landed, test the file the same way you tested the environment: `grep -q '^PIPELEX_API_KEY=.\+' /.env.local && echo filled || echo empty`. A key in the transcript is a key to rotate, and it is not yours to spend. + +A value that passed the test is copied verbatim and never inspected, so say in the report that it was taken from the environment **and not validated** — a placeholder someone exported once passes a presence test and fails the first run, and this skill never calls the API, so it cannot tell the difference. `PIPELEX_BASE_URL` stays as the example ships it. Confirm the file is gitignored before writing a key into it — both starters ignore it, but check. + +### Step 6: Verify and hand off + +The bootstrap's own checks are the verification; do not start `make dev`. Write the report (below), then hand the user's method to `/pipelex-integrate`, which recognizes the starter's codegen harness (`npm run codegen`, `make codegen`, `make add-method`) and defers to it rather than writing a second one. + +## Branch B — the ecosystem's initializer + +### Step 1: Prerequisites + +As in branch A, for the language chosen. + +### Step 2: Run the initializer — never assemble by hand + +- **A named framework** uses its documented initializer with its non-interactive flags: `npm create next-app@latest -- --ts --app --src-dir --eslint --use-npm --yes`, where the `--` is what passes the flags to the initializer instead of to npm and without it `create-next-app` prompts; `uv init --package --no-workspace ` then `uv add "fastapi[standard]"` **from inside ``**, because `uv add` writes to whatever project its working directory resolves to and from the parent that is the user's, not the new one; and so on — [references/initializers.md](references/initializers.md) carries the common ones. An initializer that only runs interactively is handed to the user to run, and you resume when it is done. +- **No framework named** takes the language's own minimal initializer: Python → `uv init --package --no-workspace `, which gives the import package `/pipelex-integrate` wants and a console-script entry; TypeScript → `npm init -y`, then `npm install --save-dev typescript @types/node` and `npx tsc --init` with strict mode, ES modules and a `src/` root. **Read [references/initializers.md](references/initializers.md) before running either**, and not only for the flags: it is where the two costs of the TypeScript default are written down — it is the resolution that meets the emitter's extensionless-import defect, and `tsc --init` switches off the `@types/node` the line before it installed — and both are the kind of thing the integration, not the scaffold, gets blamed for. + +Nothing beyond what the initializer writes is authored by this skill: no example code, no folder layout of its own, no opinion the framework did not ship. + +**On Python, finish with `uv sync` from inside ``.** `uv init` writes a `pyproject.toml` and stops: no lock file, no environment. `/pipelex-integrate` picks the package manager off the lock file and reads no lock file as `pip install` into the active environment, so a project handed over without one is a uv project installed into with pip — and `uv init` left no environment for pip to find either. The recipes that end in a `uv add` are locked by that command; the minimal and script forms, which are exactly what "no framework named" selects, are locked only by this line. [references/initializers.md](references/initializers.md) carries it with the rest of the post-initializer sequence. + +### Step 3: Version control and the pristine commit + +**Test whether `` is its own repository; never infer it from which initializer ran.** `git -C rev-parse --show-toplevel` must print `` itself, and when it does not, run `git init -b main` in the directory before staging anything. The list of initializers that `git init` on their own is not a substitute for that test, because membership in it is conditional: `uv init` initializes a repository when it creates a standalone project and **does not** when the parent directory already holds one, where it makes `` a workspace member of the enclosing project instead. A `` with no `.git` of its own is governed by whatever repository encloses it — the user's — and `git -C ` sets git's working directory without scoping anything, so the staging below would sweep that whole worktree: the user's unrelated untracked files, wherever they sit, committed into their repository under this skill's message. That is the one outcome this step exists to prevent, and the read-back catches it only if you read the paths and not just the count. + +**Then confirm there is a `.gitignore` covering the dependency tree and the build output**: `npm init -y` and `tsc --init` write none, so the minimal TypeScript recipe — and the Express and library recipes built on it — reach this step with a populated `node_modules/` and nothing excluding it, and the staging below would commit the whole dependency tree into the one commit that is supposed to be a readable baseline. Write `node_modules/`, `dist/` and `.env` into a `.gitignore` first where the initializer left none, and read back what is staged — `git -C diff --cached --name-only` over the paths themselves, not a `--stat | tail -1` whose count cannot tell a correct scaffold from a swept-up worktree — before committing. Then the one commit, for the same reason as branch A: + +```bash +git -C add -A -- . && git -C commit -m "Scaffold project" -- . +``` + +The `-- .` pathspec is the second half of the guard, and it is on **both** commands for a reason: `add -A -- .` bounds what this command stages, but a bare `git commit` then commits the *whole index*, so anything the user had staged elsewhere in an enclosing repository before the session goes into the commit under this skill's message. With the pathspec on the commit too, the staging and the commit are both held to `` and below even when the repository turns out to be an enclosing one, and the user's own staged work is left staged and uncommitted where they put it. + +**An initializer that commits as well as `git init`s has already made this commit.** `create-next-app` is one: it runs `git init`, stages everything and commits, so the tree is clean and the command above stops with `nothing to commit` — which is the initializer having done the job, not a failure of it. Take that commit as the pristine one, exactly as branch A takes GitHub's, and name it and its message in the report. Never force a second empty commit on top of it. + +### Step 4: The env file + +Write `.env.example` with the two lines the starters share, make sure `.env` is gitignored, and copy the example to `.env` under the same key rule as branch A: + +``` +PIPELEX_BASE_URL=https://api.pipelex.com +PIPELEX_API_KEY= +``` + +### Step 5: Hand off + +Add **no** SDK dependency and create **no** empty `methods/` directory: `/pipelex-integrate` adds `@pipelex/sdk` or `pipelex-sdk` when it writes the first call site, and creates `methods//` when it places the first bundle. A project with nothing to integrate yet has nothing Pipelex-shaped in it beyond the env convention, and that is correct. + +## The report + +Say, in this order: what was created and where; which template or initializer it came from, at which version and SHA; that this skill made exactly one commit and what it holds; what the bootstrap changed and that those changes are uncommitted for review, in the bootstrap's own words (branch A); which env file was written and whether the key was filled from the environment or left for the user; the demos the starter still carries and where the README's removal checklist is (branch A); and the hand-off. + +Two lines are easy to forget and matter: + +- **The project's own instructions and skills load in a session started inside it.** Its `CLAUDE.md` / `AGENTS.md` and its `release` and `bump-*` skills are not in the current session; `cd `, then starting Codex there, is how they arrive. +- **`/pipelex-integrate` still works from here meanwhile**, because the Pipelex workshop writes anywhere under the directory the harness was launched in, and the new project sits there. One exception, and it is the version-manager machine of step 1: the workshop is spawned with `npx` on the **harness's** own `PATH`, which no activation of yours reaches, so a `node` only reachable through `nvm` or `fnm` means no workshop at all. Say so there instead — the hand-off needs the harness restarted from a shell where the runtime is active. Hand the method to it by opening `../pipelex-integrate/SKILL.md` and following it; a bundle that lives elsewhere on disk is copied into the project by that skill. No method yet → `/pipelex-design` first. + +## When something goes wrong + +| Condition | Do this | +|---|---| +| A toolchain piece is missing (Node below the floor, no `uv`, no git) | STOP, name the exact missing piece and the starter README's line about it; never install a toolchain — first check a version manager the machine already has (`nvm`, `fnm`, `volta`, `asdf`, `mise`) and use its runtime, saying so | +| The target directory exists and is not empty — anything at all beyond a lone `.git` | STOP, ask for another; never delete, move or write into it, and never offer to. **A directory holding nothing but `.git` is empty here and is written into**; that exception is the one directory entry by name and not a class, so `.DS_Store`, `.idea/`, `.vscode/`, `Thumbs.db` and anything else still refuse. On branch A that directory is served by the acquisition beside it (Step 2) and never by a `git clone` into it, which git refuses outright | +| `git clone` fails (network, permissions) | report git's error verbatim; nothing to clean up beyond an empty directory, and on the acquisition beside an existing repository the chain removes its own temporary path and copies nothing | +| `gh` is absent or not authenticated | fall back to the local clone; say the GitHub repository can be created later with `gh repo create --source .` | +| The clone carries no `bootstrap` skill | follow the README's manual list; say the template changed | +| The bootstrap's checks are red | its own rule: fix the cause and re-run; never hand off on red | +| An initializer is interactive with no non-interactive form | hand the command to the user to run in the session; resume after | +| `PIPELEX_API_KEY` is not in the shell environment | leave the value empty in the env file; say where a key comes from and where it goes; never ask for it in the conversation | +| you need to know whether a key is set | test it without printing it (`[ -n "${PIPELEX_API_KEY:-}" ] && echo set`); never `env \| grep PIPELEX`, never echo the value, never move it with a file-editing tool, never read the env file back — a key in the transcript is a key to rotate | +| The working directory is the template's own checkout (its `origin` remote points at `Pipelex/pipelex-starter-…`) | STOP: this is the template, not a copy of it — acquire a copy in another directory. This is what separates it from the fresh-clone shortcut, which is a copy of the template and carries somebody else's `origin` or none | + +## Reference + +- [references/starters.md](references/starters.md) — the two starters side by side: what each brings, its prerequisite floors, the acquisition commands, its env file, its bootstrap, its demos and their removal checklist, and the codegen harness `/pipelex-integrate` will find. +- [references/initializers.md](references/initializers.md) — per language, the minimal default and the common frameworks' non-interactive initializers, whether each runs `git init`, and where the import package or `src/` root lands. +- `/pipelex-integrate` — the skill this one hands every project to. diff --git a/pipelex-codex/skills/pipelex-scaffold/references/initializers.md b/pipelex-codex/skills/pipelex-scaffold/references/initializers.md new file mode 100644 index 0000000..bd2e5c0 --- /dev/null +++ b/pipelex-codex/skills/pipelex-scaffold/references/initializers.md @@ -0,0 +1,50 @@ +# Ecosystem initializers + +Branch B of `/pipelex-scaffold` runs an initializer that already exists and authors nothing of its own beyond what that initializer writes. This file lists the common ones with their non-interactive forms. Flags change between versions: when a command below prompts anyway or rejects a flag, read its `--help` and prefer its own current non-interactive form over improvising a layout by hand. An initializer with no non-interactive form is handed to the user to run in the session. + +After the initializer: `git init -b main` only if `git -C rev-parse --show-toplevel` does not print `` itself — **the test, never the inference from which initializer ran**, because the `git init?` columns below are conditional and `/pipelex-scaffold`'s Step 3 gives the case that makes them so (`uv init` initializes nothing when the parent directory already holds a project, and staging in a `` governed by an enclosing repository sweeps the user's whole worktree into this skill's commit). Then one pristine commit (`Scaffold project`), the two-line `.env.example` (`PIPELEX_BASE_URL=https://api.pipelex.com`, `PIPELEX_API_KEY=`), `.env` gitignored and copied from it. No SDK dependency — `/pipelex-integrate` adds it. + +**On Python, run `uv sync` from inside `` before the pristine commit, and commit the `uv.lock` it writes.** `uv init` writes a `pyproject.toml` and nothing else: no lock file and no environment. That matters at the hand-off rather than here, because `/pipelex-integrate` reads the lock file to decide how to install — `uv.lock` selects uv, and **nothing** selects `pip install` into whatever environment happens to be active. A project handed over without its lock is therefore a uv project that the next skill installs into with pip, or cannot install into at all, because `uv init` left no environment for pip to find either. The recipes below that end in a `uv add` get a lock from that command; the minimal and script forms, which are the defaults when no framework is named, get one only from this step. + +## Python + +| Want | Command | `git init`? | Where the import package lands | +|---|---|---|---| +| **Minimal (the default when no framework is named)** | `uv init --package --no-workspace ` | yes (`--vcs none` to skip) | `src//` with `__init__.py` and a console-script entry in `pyproject.toml` | +| A script-style app rather than a package | `uv init --app --no-workspace ` | yes | `main.py` at the root — no import package; `/pipelex-integrate` will put the generated tree under `generated/` at the root | +| FastAPI service | `uv init --package --no-workspace && (cd && uv add "fastapi[standard]")` | yes | as minimal | +| Django project | `uv init --package --no-workspace && (cd && uv add django && uv run django-admin startproject config .)` | yes (from `uv init`) | the Django project package `config/` plus `src//`; ask which one owns the Pipelex call sites | +| Typer CLI (the Python starter's shape, without the starter) | `uv init --package --no-workspace && (cd && uv add typer)` | yes | as minimal | +| An existing `pyproject.toml` layout the user prefers (poetry, pdm, hatch) | the tool the user names: `poetry new `, `pdm init --non-interactive`, `hatch new ` | poetry: no; pdm: no; hatch: no | per tool — `poetry new` and `hatch new` make `/` or `src//` | + +`uv init` refuses a directory that already holds a project; on an empty "here" directory use `uv init --package --no-workspace .` — it names the package after the directory. + +**`--no-workspace` is on every `uv init` above, and it is the same hazard the `uv add` parentheses below address, one command earlier.** Run inside a directory that already holds a `pyproject.toml` — the scaffold being made inside an existing project, which is the common case — a bare `uv init --package ` does not create a standalone project at all. It prints `Adding as member of workspace …`, appends a `[tool.uv.workspace]` table naming `` to **the user's own `pyproject.toml`**, and then the first `uv add` writes the lockfile at the *parent* root, so the new project has no `uv.lock` of its own and does not resolve standalone. Editing a file of the user's is exactly what this skill does not do, and a project that needs its parent to resolve is not the project the report says was handed over. `--no-workspace` leaves the parent untouched and gives `` its own lock. With no parent project it changes nothing, so it is safe to pass always, which is why it is not conditional. + +**Every `uv add` above runs inside ``, and the parentheses are why.** `uv add` resolves the project from its *working* directory upwards, so run from the parent it writes the dependency into whatever project it finds there — the user's own `pyproject.toml` and lockfile, when the scaffold is being made inside an existing workspace — or fails outright when it finds none. Neither is the new project. `uv add --directory ` is equivalent if you prefer a flag to a subshell. + +## TypeScript / JavaScript + +| Want | Command | `git init`? | Where `src/` lands | +|---|---|---|---| +| **Minimal (the default when no framework is named)** | `mkdir -p && cd && npm init -y && npm install --save-dev typescript @types/node && npx tsc --init --strict --module nodenext --target es2022 --rootDir src --outDir dist` then set `"type": "module"` in `package.json` — **but read the resolution note below first: prefer `--module esnext --moduleResolution bundler` unless the user actually wants Node's own resolution**, because `nodenext` is the shape that meets the emitter defect | no | `src/` (create it); `/pipelex-integrate` puts the generated tree under `src/generated/` | +| Next.js app (the JS starter's shape, without the starter) | `npm create next-app@latest -- --ts --app --src-dir --eslint --use-npm --yes` | yes (`--disable-git` to skip) | `src/app/`; generated tree under `src/generated/` | +| Vite + React | `npm create vite@latest -- --template react-ts` then `(cd && npm install)` | no | `src/` | +| Hono server | `npm create hono@latest -- --template nodejs --pm npm --install` | no | `src/` | +| Express server | the minimal recipe above, then `(cd && npm install express && npm install --save-dev @types/express)` | no | `src/` | +| Node library | the minimal recipe above | no | `src/` | +| pnpm / yarn / bun instead of npm | replace `npm create` with `pnpm create` / `yarn create` / `bun create`, and the install command accordingly; `/pipelex-integrate` reads the lockfile to pick the package manager for what it adds | — | — | + +`npm create @latest -- `: the `--` is what passes the flags to the initializer rather than to npm. The Next.js `--yes` accepts the initializer's defaults for every prompt not covered by a flag. + +**Every follow-on `npm install` above is parenthesised for the same reason every `uv add` is, and npm is the worse of the two.** `npm install` resolves the project it writes to from its *working* directory upwards, so run from the parent it adds the dependency to the user's own `package.json` and puts `node_modules/` in the user's tree — and where `uv add` at least fails outright when it finds no project nearby, npm finds the parent and silently succeeds, leaving the new project with nothing installed and no error to read. The `cd ` inside the minimal recipe's own `&&` chain does not reach a follow-on issued as a separate command, because shell state does not survive from one command to the next. Use the subshell, or `npm install --prefix `. + +**The minimal recipe's `--module nodenext` plus `"type": "module"` is exactly the shape that meets the ts-zod emitter's extensionless-import defect** (`pipelex-integrate`'s `references/typescript.md`, "Known defect"): the generated `binder.ts` fails the type check with `TS2835` and will not load at runtime. So say so when you hand a minimal TypeScript project to `/pipelex-integrate`, and when the user has no reason to prefer Node's own resolution, prefer a bundler-backed setup (Vite, Next.js) or `--module esnext --moduleResolution bundler`, which the defect does not touch. The two recipes also need `node_modules/` and `dist/` in a `.gitignore` before the pristine commit — neither `npm init -y` nor `tsc --init` writes one. + +**And `tsc --init` writes `"types": []` as an active key, which switches `@types/node` off on the line after the recipe installed it.** That empty array is the current `tsc --init` template's own default (TypeScript 7 writes it, with `// "types": ["node"],` commented out three lines below), and it means no `@types` package is loaded at all — so the call site `/pipelex-integrate` writes next fails with `TS2591` on `node:fs/promises`, on `node:path` and on `process`, and then cascading `TS7006` implicit-any on the `readdir` callback. Nothing about the errors points at the tsconfig, so set `"types": ["node"]` as part of the recipe rather than leaving it for the integration to discover. `npm install --save-dev typescript` also resolves to TypeScript 7 now; say which major the project got, because `tsc --init`'s defaults moved with it. + +## What every branch-B project shares afterwards + +- One commit, the pristine scaffold, so the user's first real change is a clean diff. +- `.env.example` and `.env` written, `.env` ignored, `PIPELEX_API_KEY` filled only from the shell environment. Both land *after* the pristine commit, which holds the initializer's output as it came, so they stay untracked for the user to review and commit — the same posture branch A leaves the bootstrap's edits in. +- Nothing else Pipelex-shaped: the SDK dependency, the `methods/` directory and the generated tree arrive with the first `/pipelex-integrate`. diff --git a/pipelex-codex/skills/pipelex-scaffold/references/starters.md b/pipelex-codex/skills/pipelex-scaffold/references/starters.md new file mode 100644 index 0000000..08550a3 --- /dev/null +++ b/pipelex-codex/skills/pipelex-scaffold/references/starters.md @@ -0,0 +1,76 @@ +# The two Pipelex starters + +Both are GitHub **template repositories** under the `Pipelex` organization. Each is a real, CI-tested application against the hosted Pipelex API, not a parameterized template: the identity you see in a fresh clone (`pipelex-starter-js` / `Pipelex Starter`, or `piper` / `Piper`) is a placeholder that the starter's own `bootstrap` skill rewrites. Read the clone's `README.md` after acquiring it — the sections named below are where the details live, and they move as the starters evolve. + +## Side by side + +| | `pipelex-starter-js` | `pipelex-starter-python` | +|---|---|---| +| **Shape** | Next.js (App Router), React, TypeScript strict, Tailwind; a web app with one tab per method whose input form is rendered from the method's own contract by `@pipelex/mthds-form` | A Typer CLI with one command per method, printing JSON on stdout and a cost report on stderr; three execution modes (`blocking`, `attended`, `detached`) as separate sub-packages | +| **Pick it when** | people will use the methods in a browser: forms, uploads, live run status | the methods run from a terminal, a script, a batch job or a service, and the user wants Python | +| **SDK** | `@pipelex/sdk` | `pipelex-sdk` (import package `pipelex_sdk`) — the `pipelex` runtime is **not** a dependency | +| **Methods live in** | `methods//main.mthds`, or `methods//method.json` for a method that lives elsewhere (a catalog id or a published address) | `/methods//main.mthds` | +| **Generated types** | `src/generated//` — `types.ts`, `binder.ts`, `contracts.ts`, `codegen.lock`, `sources.json` | `/generated//` — `models.py`, `codegen.lock` | +| **Codegen harness** | `npm run codegen` (keyed, dev), `npm run codegen:check` (offline, in `make check`), `npm run codegen:verify` (keyed, semantic); `make add-method METHOD=` scaffolds a remote method end to end | `make codegen` / `make codegen-check` — **both shell out to a `pipelex` CLI the starter does not depend on** (`PIPELEX=` in the Makefile); `/pipelex-integrate` knows this and writes into the same layout when that CLI is absent | +| **Toolchain floor** | Node ≥ the `engines.node` field of `package.json` (22.12 at writing); `npm` | `uv`; a Python inside `requires-python` of `pyproject.toml` (3.11–3.14 at writing) | +| **Env file** | `.env.local`, from `.env.example`: `PIPELEX_BASE_URL`, `PIPELEX_API_KEY`, `NEXT_PUBLIC_EXECUTION_MODE` | `.env`, from `.env.example`: `PIPELEX_BASE_URL`, `PIPELEX_API_KEY` | +| **Checks the bootstrap runs** | `npm install --package-lock-only`, then `make all` (lint, format check, typecheck, unit tests, build) | `make li` (lock + sync), then `make agent-check` and `make agent-test` | +| **Agent-facing files** | `CLAUDE.md`, `AGENTS.md`; skills `bootstrap`, `release`, `bump-sdk`, `bump-mthds-form` | `CLAUDE.md`; skills `bootstrap`, `release` | +| **Docs worth reading after bootstrap** | `docs/codegen.md`, `docs/add-method.md`, `docs/input-form.md`, `docs/adopt-in-an-existing-project.md`; README → "Swap in your own pipeline" and "Remove an example" | `docs/codegen.md`, `docs/cli-architecture.md`; README → the per-command sections | +| **Demos it carries** | several demo methods, one tab each; keep them as references or strip them with the README's "Remove an example" checklist | several demo methods, one CLI command each; keep them as references or remove the command and its method directory together | + +## Acquisition + +`` below is the one the choice above settled — `pipelex-starter-js` or `pipelex-starter-python` — and each block is a single chain for that one starter, never a menu to run top to bottom. + +Local clone with fresh history (the default — it produces what GitHub's "Use this template" button produces, a copy with no history and no remote): + +```bash +git clone --depth 1 https://github.com/Pipelex/.git || exit +git -C rev-parse HEAD +rm -rf /.git && git -C init -b main +git -C add -A -- . && git -C commit -m "Start from Pipelex/ ()" -- . +``` + +The version comes from `package.json` (`"version"`) on JS and from `pyproject.toml` (`version =`) on Python, read before the commit. **The `-- .` pathspec is on the commit as well as on the staging**, for the reason `/pipelex-scaffold`'s Step 3 gives in full: `add -A -- .` bounds what is staged, but a bare `git commit` then commits the whole index, so anything the user had staged elsewhere in an enclosing repository rides along under this skill's message. + +**The `|| exit` on the clone is load-bearing and is not decoration**, for the reason `/pipelex-scaffold`'s Step 2 gives in full: the line below it deletes a `.git` directory, and a clone that never ran — a network failure, or `` already existing — leaves that `rm -rf` to find whatever `.git` is actually at that path, destroying a repository of the user's irrecoverably. The guard only holds inside one shell, so when the two lines go out as separate commands, check the clone's exit status yourself before typing the `rm -rf`, and never type it on a path you have not just created. + +Into a directory whose only entry is `.git` — the one shape the "Where" rule reads as empty and `git clone` still refuses — acquire beside it and move in, so the user's own repository stands and the end state is the same as above: + +```bash +dir=$(cd && pwd) || exit 1 +tmp=$(mktemp -d "$(dirname "$dir")/.pipelex-starter-XXXXXX") || exit 1 +git clone --depth 1 https://github.com/Pipelex/.git "$tmp" || { rm -rf "$tmp"; exit 1; } +git -C "$tmp" rev-parse HEAD +rm -rf "$tmp/.git" || { rm -rf "$tmp"; exit 1; } +[ "$(ls -A "$dir")" = ".git" ] || { rm -rf "$tmp"; exit 1; } +cp -R "$tmp"/. "$dir"/ || { rm -rf "$tmp"; exit 1; } +rm -rf "$tmp" +``` + +Every part of that chain is load-bearing, and `/pipelex-scaffold`'s Step 2 gives each in full. **The first line resolves `` to an absolute path before the parent is computed from it**, without which a destination spelled `.` — the ordinary spelling, since the user is usually standing in the directory they just `git init`-ed — puts the temporary directory inside the destination and the `ls -A` line then refuses every time. **No `rm -rf` in it addresses a path under ``**: the template's history is discarded while the clone is still at a path `mktemp` made for this command, before anything moves, so the guarded-deletion problem above does not arise here at all. **`cp -R "$tmp"/. "$dir"/` carries the entries beginning with a dot** — `.gitignore`, `.env.example`, `.github/`, `.claude/` — every one of which `mv "$tmp"/* "$dir"/` leaves behind while exiting `0`. And **the `ls -A` line admits exactly one entry**, `.git`, which the clone no longer has, so the template can only add to the directory and anything else stops the run with nothing copied and the temporary path removed. It is one chain and goes out as one command, for the reason the paragraph above gives. + +GitHub repository, on request and after confirmation (visibility asked, default private; GitHub makes the initial commit, so no pristine commit of your own): + +```bash +gh auth status +gh repo create / --template Pipelex/ --private --clone +``` + +## The bootstrap you delegate to + +Both starters carry `.claude/skills/bootstrap/SKILL.md` with a bundled script (`scripts/bootstrap.mjs` / `scripts/bootstrap.py`). Read the file in the clone and follow it; the shape is the same on both: + +1. **Preflight** — confirms the identity is still the template's (`package.json` name `pipelex-starter-js`; `pyproject.toml` `name = "piper"`), notes a dirty tree, and on JS makes sure `node_modules/` exists (`make install`). +2. **Collect** — the package name (kebab on JS, underscores on Python, everything else derives from it), a display title, a one-line description; optionally author name **and** email (never one without the other), the repository URL, and the license (MIT kept, proprietary, or another SPDX id; the copyright holder and year). Pass what the conversation already holds so it asks once for the rest. +3. **Dry run** — the script with `--dry-run` prints the plan; the user confirms. +4. **Run** — the same command without `--dry-run`; on Python the package directory is renamed with `git mv`, which is why the pristine commit must exist first. `--clean` strips the template-only prose; keep it unless the user wants the template charter kept. +5. **Verify** — the lock file is re-synced and the project's own checks run; red is fixed, not skipped. +6. **Self-removal** — `rm -rf .claude/skills/bootstrap`, unstaged like everything else; the user reviews with `git status` and `git diff` and commits when ready. + +The starter's rules are yours while you run it: never commit on the user's behalf, always dry-run first, never touch `.github/` or the `release` skill's logic. + +## What `/pipelex-integrate` finds afterwards + +A bootstrapped starter is a project that **owns a codegen harness**, and `/pipelex-integrate` defers to it: on JS it drops a bundle under `methods//` and runs `npm run codegen`, or runs `make add-method METHOD=…` for a catalog or published method, then follows `docs/codegen.md` and the existing actions for the fan-out; on Python it places the bundle under `/methods//` and runs `make codegen` when a `pipelex` CLI is available, writing into `/generated//` through the Pipelex workshop when it is not. It never writes a second generated layout beside the starter's own. diff --git a/pipelex-vibe/skills/pipelex-design/SKILL.md b/pipelex-vibe/skills/pipelex-design/SKILL.md index 50cf5c2..d5d690a 100644 --- a/pipelex-vibe/skills/pipelex-design/SKILL.md +++ b/pipelex-vibe/skills/pipelex-design/SKILL.md @@ -182,7 +182,7 @@ After the gate: 1. **Organize only when the layout needs it.** A direct result that is already coherent skips `/pipelex-organize`. A converged stepwise result normally invokes it automatically because one-definition-per-file construction history and satisfied headers need regrouping. A naturally coherent result in either mode does not take an organization round trip solely for process compliance. 2. **Project the input schema.** Call `mthds_inputs_template` with the final whole-bundle `files` submission plus `explicit: false`. Show the returned compact template, but **do not save it as `inputs.json`** — input preparation belongs exclusively to `/pipelex-inputs`. 3. **Present the flow.** Point to the interactive method graph where the host rendered the valid verdict's view; in terminal hosts, present a concise text flow of the final structure. -4. **Hand off inputs — and the code.** Suggest preparing real inputs with `/pipelex-inputs`. Then, when the workspace holds a codebase (a `package.json` or a `pyproject.toml`), say that `/pipelex-integrate` wires the method into it with generated types and a typed call site. +4. **Hand off inputs — and the code.** Suggest preparing real inputs with `/pipelex-inputs`. Then, when the workspace holds a codebase (a `package.json` or a `pyproject.toml`), say that `/pipelex-integrate` wires the method into it with generated types and a typed call site; when it holds none and the user wants an application around the method, `/pipelex-scaffold` creates one and hands it to `/pipelex-integrate`. > **NEVER write `inputs.json` manually.** If the user provides files, paths, or wants to run with real data, invoke `/pipelex-inputs` — it handles the template, path resolution, placeholder formatting, and file copying. diff --git a/pipelex-vibe/skills/pipelex-integrate/SKILL.md b/pipelex-vibe/skills/pipelex-integrate/SKILL.md index daf08e3..c4a94d2 100644 --- a/pipelex-vibe/skills/pipelex-integrate/SKILL.md +++ b/pipelex-vibe/skills/pipelex-integrate/SKILL.md @@ -1,6 +1,6 @@ --- name: pipelex-integrate -description: Wire an MTHDS method into a Python or TypeScript codebase with generated, drift-proof types and one typed call site that runs it through @pipelex/sdk or pipelex-sdk. Use when the user says "use this method in my app", "call this from my code", "generate types for this method", "wire the method into my project", "add this pipeline to my service", "typed client for this method", "integrate the method", "refresh the generated types", "regenerate the types", "the types are stale", or wants application code that runs a .mthds method — from a local bundle, a catalog id (mt_…) or a published method_ref address. Also the refresh path after a bundle edit. Not for authoring or editing the method itself (/pipelex-design, /pipelex-edit), and not for a workspace with no project in it yet, where the project has to exist first. +description: Wire an MTHDS method into a Python or TypeScript codebase with generated, drift-proof types and one typed call site that runs it through @pipelex/sdk or pipelex-sdk. Use when the user says "use this method in my app", "call this from my code", "generate types for this method", "wire the method into my project", "add this pipeline to my service", "typed client for this method", "integrate the method", "refresh the generated types", "regenerate the types", "the types are stale", or wants application code that runs a .mthds method — from a local bundle, a catalog id (mt_…) or a published method_ref address. Also the refresh path after a bundle edit. Not for authoring or editing the method itself (/pipelex-design, /pipelex-edit), and not for a project that does not exist yet (/pipelex-scaffold). --- @@ -52,7 +52,7 @@ Automatic by default: state the target, the destination and the generator in one - **A published address** — `method_ref: "github.com//[/]@"`. The tag is the pin: a `method_ref` **without a tag** floats and is refused for a committed integration; ask for the tag. - **A catalog id** — `method_id: "mt_…"` (resolve a name through `mthds_list_methods` when present). The catalog is unversioned: an edit to the stored method silently invalidates committed types with nothing offline to detect it. Say so in one line, recommend committing the source or publishing an address, and proceed only on the user's say-so. -**The project** is the nearest directory holding a `package.json` or a `pyproject.toml` (or `setup.py` / `requirements.txt`) above the working area. A workspace holding several (a monorepo, a full-stack repo) is a question — which app? — never a guess. **No project at all** → this is not an integration yet: say so and stop, because there is nothing to wire the method into. Creating the project is the user's own step — a starter, their framework's initializer, their own layout — and integration begins once it exists. +**The project** is the nearest directory holding a `package.json` or a `pyproject.toml` (or `setup.py` / `requirements.txt`) above the working area. A workspace holding several (a monorepo, a full-stack repo) is a question — which app? — never a guess. **No project at all** → this is not an integration yet: offer the `pipelex-scaffold` skill (open `../pipelex-scaffold/SKILL.md`), which creates one and hands it back here. Then look for a **codegen harness**: a `codegen` script in `package.json` or a `codegen` Makefile target, a `sources.json` carrying a `derived` map, `docs/codegen.md` or `docs/add-method.md`, a `methods/` directory beside `src/generated/` or `/generated/`. Either of the first two decides; the rest only corroborate. If the project has one, follow [A project that owns a codegen harness](#a-project-that-owns-a-codegen-harness) from here. diff --git a/pipelex-vibe/skills/pipelex-scaffold/SKILL.md b/pipelex-vibe/skills/pipelex-scaffold/SKILL.md new file mode 100644 index 0000000..067bdde --- /dev/null +++ b/pipelex-vibe/skills/pipelex-scaffold/SKILL.md @@ -0,0 +1,205 @@ +--- +name: pipelex-scaffold +description: Start a new project that will call MTHDS methods through Pipelex, in TypeScript or Python — from one of the Pipelex starter templates or from the ecosystem's own initializer — and hand it to /pipelex-integrate. Use when the user says "start a new project with Pipelex", "I have a method and need an app around it", "create a Next.js app that runs my method", "set up a Pipelex project from scratch", "new Python CLI for this method", "which starter should I use", "bootstrap a Pipelex project", or wants a codebase where none exists yet. Also use when the user is standing in a freshly cloned pipelex-starter-js or pipelex-starter-python that has not been renamed yet — this skill runs the template's own bootstrap for them. Not for adding Pipelex to code that already exists: that is /pipelex-integrate. + +--- + +# Scaffold a project for Pipelex methods + +Give a user who has no project yet a project that is ready for `/pipelex-integrate`. This skill has exactly two branches and carries no templates of its own: + +- **One of the Pipelex starters** when the user wants the opinionated shape: `pipelex-starter-js` for a web app whose forms are rendered from the methods' own contracts, `pipelex-starter-python` for a CLI or service that runs methods in the three execution modes. You acquire the template, commit it once as it came, then run the clone's **own** `bootstrap` skill — the rename logic lives in the starters and is never reimplemented here. +- **The ecosystem's own initializer** when the user wants their framework or a minimal project: `uv init --package`, `npm create next-app@latest`, `django-admin startproject`, whatever the framework documents. You run it; you never assemble a project by hand. + +Both branches end the same way: an env file that follows the starters' convention, one pristine commit that makes everything after it reviewable, and the hand-off — to `/pipelex-integrate` when a method exists, to `/pipelex-design` first when none does. + +**What this skill is not.** Not a template engine (no cookiecutter, no copier, no framework matrix of its own), not a bootstrap (the starters own theirs), not a runner or a dev-server launcher, not a deployer. It needs no MCP tool and no API key: git, the starters' scripts and the ecosystem's initializers are all it uses. + +## Choosing the branch + +A cheap, reliable signal decides; an inconclusive one asks one question; nothing is guessed twice. + +| Question | Signals, in order | When inconclusive | +|---|---|---| +| **Language** | the user's word; the language the method's consumer is written in; a framework the user named | ask | +| **Which branch** | a **named framework** the starters do not carry (FastAPI, Django, Express, Hono, a plain library, a Lambda) → the initializer; **"minimal"**, **"no demo code"**, **"just a project"** → the initializer; a **web app people use in a browser**, forms, an upload flow → the JS starter; a **CLI, script, batch job, worker or service** in Python → the Python starter | one question offering the matching starter first, saying what it brings (durable runs, forms or CLI modes, codegen wiring, CI, its own `release` skill) and what it costs (demos to keep as references or to strip) | +| **Where** | the directory the user named; **"here"** when the working directory is empty — **and a directory whose only entry is `.git` is empty for this rule**, because `mkdir my-app && cd my-app && git init` is an ordinary way to arrive here and a repository the user made is not work of theirs to write over: branch B runs its initializer in the directory as it stands and leaves that repository alone — it does **not** re-run `git init` there, since Step 3's test finds `` is already its own repository — while branch A acquires beside it and moves in, so the repository already there goes on standing either way (Step 2); else a kebab-case directory named after the project | ask; never write into a directory that exists and is not empty, and never offer to move, delete or merge what it holds to make room — the answer is another directory. **A lone `.git` is the only entry that does not make a directory non-empty, and that is a ruling about `.git` and nothing else** — not a class of files you may decide to overlook. Every other entry still refuses, `.DS_Store`, `.idea/`, `.vscode/` and `Thumbs.db` included: judging which of a user's files matter is the thing this rule exists to forbid, and a list that grows by guesswork is how it would come back. The rule is about **a directory you are creating a project in**, which is why the fresh-clone shortcut below is not an exception to it: there the project is already there and you are finishing it, not writing over someone's work | +| **GitHub or local** | the user asked for a GitHub repository → `gh repo create --template`, after confirmation; otherwise a local clone with fresh history | local | + +**The fresh-clone shortcut.** A starter clone already in the working directory that has not been bootstrapped — `package.json` still says `pipelex-starter-js`, or `pyproject.toml` still says `name = "piper"` — **and whose `origin` does not point at `Pipelex/pipelex-starter-…`** is branch A entered at step 4: acquisition already happened, so go straight to running the clone's bootstrap. Do not clone again. + +[references/starters.md](references/starters.md) compares the two starters and carries every command below; [references/initializers.md](references/initializers.md) carries the initializers. + +## Mode + +Automatic by default, with the plugin's usual rules: an explicit user signal wins ("just do it" → automatic; "walk me through" → interactive); a genuinely ambiguous branch is one question, asked once; a request that gave every input up front proceeds without re-asking. Two things always confirm, in every mode: **`gh repo create`**, because it creates a repository on GitHub, and whatever the clone's bootstrap skill confirms on its own account. The pristine commit does not need confirmation **on a directory this skill created** — it holds the template as it came, and no user content is at stake. **The acquisition into a directory that already held a repository is the exception**, and it is a third thing that always confirms: there the commit lands on the user's branch, on top of their history, and `add -A -- .` records whatever their worktree was already showing along with the template (Step 3). None of the three grounds above holds, so state what will be staged and what it will land on, and ask — in every mode. + +## Branch A — one of the starters + +### Step 1: Prerequisites + +Check before touching anything, and **stop** on a missing piece with the exact thing missing and the starter README's own line about it. Never install a toolchain — but a runtime the machine already has and only the `PATH` is missing is not a missing piece: when `node` or `uv` is absent while a version manager on the machine carries one (`nvm`, `fnm`, `volta`, `asdf`, `mise`), activate it for this work and say in the report which one you used and that the user's own shell may not have it. Stop only when no usable runtime can be reached that way. + +**Activating it means resolving it to an absolute path, not sourcing a shell.** Your shell state does not survive from one command to the next — each one starts again from the user's profile, which is the profile that did not have the runtime — so `. nvm.sh` or `eval "$(fnm env)"` in one call buys nothing in the next. Resolve the binary once (`ls "$NVM_DIR"/versions/node/*/bin/node`, `volta which node`, `mise which node`, `asdf which node`, `fnm exec --using= -- which node`), keep that directory, and prefix **every** later command with it — `PATH=":$PATH" …` — the clone's bootstrap and its `make all` / `make agent-check` included, because those are separate commands too. Verify the runtime answers under that prefix **before** step 2, so a machine you cannot actually reach stops while nothing has been created; discovering it at step 4 has already spent the pristine commit. + +Three things this clause does not license. **A shim is not a runtime**: `asdf` and `mise` put a `node` on the `PATH` that exists and then fails with "no version set", so the test is that `node --version` *answers*, not that the binary resolves — and that case is a stop, not a manager to activate. **The floor still applies**: a manager holding Node 18 does not satisfy the starter's `engines` floor, and "a runtime the machine already has" never means a version below it. And **`volta` and `mise` install on first use** — `volta run`, `mise x` and `mise use` will fetch a version they do not have — which is the toolchain install this step forbids: use only a version the manager already holds, and stop rather than let it download one. Note too that `nvm`, `fnm` and `volta` manage Node alone and can never supply `uv`. + +- **JavaScript**: Node at or above the floor the starter's `package.json` `engines` field names (`node --version`; 22.12 at writing — the SDK is ESM-only and the starter's e2e specs `require()` it), and `npm`. +- **Python**: `uv` on the PATH (the starter installs and locks with it) and a Python inside the starter's `requires-python` range that `uv python find` can see (3.11 to 3.14 at writing). +- **Both**: `git`. The GitHub form also needs `gh` authenticated — `gh auth status`. + +### Step 2: Acquire the template + +**Local, the default.** Clone shallow, read the template's identity, then detach from it: + +```bash +git clone --depth 1 https://github.com/Pipelex/.git || exit +git -C rev-parse HEAD # the template SHA, for the commit message +# the template version: package.json "version" (JS) or pyproject.toml version (Python) +rm -rf /.git && git -C init -b main +``` + +The `|| exit` on the clone is not decoration: the line below it deletes a `.git` directory, and if the clone never ran — a network failure, or `` already existing — that `rm -rf` finds whatever `.git` is actually at that path. On a directory the skill just created it destroys nothing; on a repository of the user's it destroys their history irrecoverably. Run the destructive line only behind a clone that succeeded, and never type it on a path you have not just created. + +The clone's `.git` is removed on purpose: it is the template's history and remote, and leaving it would make `git status` and a future `git push` belong to Pipelex's template rather than to the user's project. This is exactly what GitHub's "Use this template" button produces — a copy with no history and no remote — and it is why the starters' READMEs tell humans not to clone directly. Fresh history is how you honour that. + +**Local, into a directory that already holds a repository.** The "Where" rule reads a directory whose only entry is `.git` as empty, and `git clone` cannot serve it: git refuses any destination already holding a `.git` and stops with `destination path '' already exists and is not an empty directory`. So acquire **beside** the directory and move in. The end state is the one the default recipe reaches — the template in ``, no template history, no Pipelex remote — and the repository the user made goes on standing instead of being replaced: + +```bash +dir=$(cd && pwd) || exit 1 +tmp=$(mktemp -d "$(dirname "$dir")/.pipelex-starter-XXXXXX") || exit 1 +git clone --depth 1 https://github.com/Pipelex/.git "$tmp" || { rm -rf "$tmp"; exit 1; } +git -C "$tmp" rev-parse HEAD # the template SHA, for the commit message +# the template version: package.json "version" (JS) or pyproject.toml version (Python) +rm -rf "$tmp/.git" || { rm -rf "$tmp"; exit 1; } +[ "$(ls -A "$dir")" = ".git" ] || { rm -rf "$tmp"; exit 1; } +cp -R "$tmp"/. "$dir"/ || { rm -rf "$tmp"; exit 1; } +rm -rf "$tmp" +``` + +**The first line resolves the destination, and that is what keeps the temporary path a sibling rather than a child.** `` is very often `.` here: `mkdir my-app && cd my-app && git init` is the "Where" rule's own account of how a user arrives at a directory holding nothing but `.git`, and they then ask for the project *here*. `dirname .` is `.`, so deriving the parent from the spelling would put the temporary directory **inside** the destination, where the `ls -A` line below finds it sitting beside `.git` and refuses — every time, on exactly the case this section exists to serve. Resolving to an absolute path first also pins the destination for the rest of the chain, so no later line can be re-read against a working directory that has moved, and it is what lets every mention below be quoted: a name with a space reaches `cp` whole instead of arriving as two arguments. + +**No `rm -rf` here ever addresses a path under ``, and that is the ordering rather than a coincidence.** The template's history is discarded while the clone is still at a path `mktemp` made for this one command, so the destructive line is spent before anything moves: the only two paths any delete is pointed at are `"$tmp/.git"` and `"$tmp"`. Compare the default recipe, where the same line runs on `` itself and `|| exit` is the whole thing standing between it and a user's history. Here there is nothing for a guard to hold, which is what makes this the form you may aim at a directory holding somebody's repository. + +**`cp -R "$tmp"/. "$dir"/`, and never `mv "$tmp"/* "$dir"/`.** The glob matches no entry beginning with a dot, so the naive move leaves `.gitignore`, `.env.example`, `.github/` and `.claude/` behind, exits `0`, and the line after it deletes the temporary directory they are still sitting in — a starter arriving without its `.gitignore`, whose pristine commit then swallows `node_modules/`, reported as a success. The trailing `/.` copies the directory's *contents*, dotfiles included, with no shell globbing involved at all. Confirm it with `ls -A "$dir"` after the copy rather than trusting the form. + +**The `ls -A` line is the "Where" rule read again, against the copy.** It is not the decision — the "Where" question settled that — it is the last look before anything lands, put next to the copy so nothing can change between the two. It admits exactly one entry, `.git`, which the clone has not had since the line above: a collision is therefore impossible rather than merely unlikely, and the template can only add to the directory. Anything else — `.git` beside a file of the user's, a `.DS_Store`, a `README.md` they wrote — stops here with nothing copied, the temporary path removed and the directory as it was. A discarded shallow clone is the cheap half of that trade. Nothing of the user's is overwritten, moved or deleted to make room, here or anywhere. + +**The chain goes out as one command.** The guards hold only inside one shell — the same reason the `|| exit` above is load-bearing, stated in full in [references/starters.md](references/starters.md) — and split across separate calls this one loses its cleanup too, leaving the temporary directory beside the user's project with no line left to remove it. + +**Nothing is initialized here.** The default recipe ends `git init -b main` because it has just deleted the only repository at that path. This one ends on the user's repository, their branch and their remote, which is the whole point of taking the long way round. + +**GitHub, on request.** When the user asked for a repository on GitHub: + +```bash +gh repo create / --template Pipelex/ --private --clone +``` + +`--clone` takes no destination argument: it clones into `./` under the current working directory, so for this form `` **is** `` — either choose the repository name to match the directory the "Where" question settled, or rebind `` to `./` before step 4, because every step after this one addresses `` literally. Visibility is the user's call: ask, default `--private`. Creating a repository on GitHub is outward-facing, so **state the exact command and confirm before running it**, in every mode. GitHub writes the initial commit itself — skip step 3 and continue at step 4. If `gh` is absent or not authenticated, fall back to the local clone and say the repository can be created later with `gh repo create --source .`. + +Both forms take the template's default-branch head. Do not offer a release tag unless the user asks for one. + +### Step 3: Commit the pristine template — exactly once + +```bash +git -C add -A -- . && git -C commit -m "Start from Pipelex/ ()" -- . +``` + +This is the **one commit this skill makes**, and it is load-bearing twice over: the Python starter's bootstrap renames the package directory with `git mv`, which refuses a path git does not track, and a committed baseline is what turns the bootstrap's edits into a diff the user can read before committing them. Nothing of the user's is in it — it is the template as it came. One qualification on the acquisition into a directory that already held a repository: the commit lands on the user's branch, on top of their history rather than opening a new one, and `add -A -- .` also records whatever deletion their worktree was already showing — their own pending change and not one this skill made, so name it in the report instead of undoing it. That is the commit the Mode section sends back for confirmation, and this is what to put in front of the user: `git -C status --short` before staging says what will ride along, and it is the difference between a baseline commit and a line in their history that says "Start from Pipelex/…" over a change they made. + +### Step 4: Run the clone's own bootstrap + +Read `/.claude/skills/bootstrap/SKILL.md` and follow it as written. The project's skills are not loaded in this session — it began elsewhere — so read the file; do not look for a `/bootstrap` command. Run every command it gives from inside the project directory (`cd && …`, or `-C `), because that skill assumes it is standing in the repo root. + +Feed it what the conversation already holds — the project name, title, description, author, repository URL, license — so that it asks once, consolidated, for whatever is left, exactly as its own Step 2 says. It dry-runs, previews, runs, re-syncs the lock file, runs the project's own checks (`make all` on JS; `make agent-check` and `make agent-test` on Python), and removes itself. Its rules stand unchanged: it never commits, its edits stay uncommitted for the user's review (the Python renames are staged by `git mv`, which its skill explains), and a red check is fixed, never skipped. **Add nothing to that procedure and reimplement none of it.** If the clone carries no bootstrap skill — a future template dropped it — follow the README's "manual equivalent" list and say that the template changed. + +### Step 5: The env file + +```bash +cp -n /.env.example /.env.local # JS: Next.js reads .env.local +cp -n /.env.example /.env # Python: python-dotenv reads .env +``` + +`-n` because this is the one step that can destroy something of the user's. On the fresh-clone shortcut the directory is one they were already working in, and a plain `cp` would overwrite an `.env.local` they had filled with their own key — the skill's whole posture is that nothing of the user's is ever cleared, and an env file is the most expensive thing in the tree to lose. An existing env file is left exactly as it is; read whether it already carries a key with the file-side test below, and say in the report that you kept theirs. + +Fill `PIPELEX_API_KEY` **from the shell environment when it is set there**, and leave it empty otherwise, telling the user where a key comes from (`app.pipelex.com`) and that this file is where it goes. **Never print a key, and never ask for one in the conversation.** Test for it without printing it — `[ -n "${PIPELEX_API_KEY:-}" ] && echo set || echo unset`. + +**The value moves only through a shell that expands the variable itself, and never through you.** A redirection is safe precisely because the shell does the expanding and only the variable's *name* is transcribed. **Append only when the file does not already carry a key**, which is the other half of the `-n` above: `grep -q '^PIPELEX_API_KEY=.\+' /.env.local || printf 'PIPELEX_API_KEY=%s\n' "$PIPELEX_API_KEY" >> /.env.local`. On the fresh-clone shortcut the file is one the user may have filled themselves, and an unconditional append puts a second assignment *after* theirs — every dotenv reader resolves a repeated name to the later line, so their working key is silently replaced by whatever the shell happened to export, while the report tells them you kept theirs. A file-editing tool is the one form that cannot be made safe, whatever it is called — it takes a **literal** string, so you would have to know the value to pass it, and a tool call's parameters are the transcript. So: no file-editing tool on a line carrying the key, no `env | grep PIPELEX`, no `echo $PIPELEX_API_KEY`, no command substitution in a message, and **no reading the env file back** once written — `cat .env.local`, a `grep` over it, or opening it to check your work is the reflex after writing and the first move when a later step fails, and it puts the key in the transcript just as surely. To confirm the write landed, test the file the same way you tested the environment: `grep -q '^PIPELEX_API_KEY=.\+' /.env.local && echo filled || echo empty`. A key in the transcript is a key to rotate, and it is not yours to spend. + +A value that passed the test is copied verbatim and never inspected, so say in the report that it was taken from the environment **and not validated** — a placeholder someone exported once passes a presence test and fails the first run, and this skill never calls the API, so it cannot tell the difference. `PIPELEX_BASE_URL` stays as the example ships it. Confirm the file is gitignored before writing a key into it — both starters ignore it, but check. + +### Step 6: Verify and hand off + +The bootstrap's own checks are the verification; do not start `make dev`. Write the report (below), then hand the user's method to `/pipelex-integrate`, which recognizes the starter's codegen harness (`npm run codegen`, `make codegen`, `make add-method`) and defers to it rather than writing a second one. + +## Branch B — the ecosystem's initializer + +### Step 1: Prerequisites + +As in branch A, for the language chosen. + +### Step 2: Run the initializer — never assemble by hand + +- **A named framework** uses its documented initializer with its non-interactive flags: `npm create next-app@latest -- --ts --app --src-dir --eslint --use-npm --yes`, where the `--` is what passes the flags to the initializer instead of to npm and without it `create-next-app` prompts; `uv init --package --no-workspace ` then `uv add "fastapi[standard]"` **from inside ``**, because `uv add` writes to whatever project its working directory resolves to and from the parent that is the user's, not the new one; and so on — [references/initializers.md](references/initializers.md) carries the common ones. An initializer that only runs interactively is handed to the user to run, and you resume when it is done. +- **No framework named** takes the language's own minimal initializer: Python → `uv init --package --no-workspace `, which gives the import package `/pipelex-integrate` wants and a console-script entry; TypeScript → `npm init -y`, then `npm install --save-dev typescript @types/node` and `npx tsc --init` with strict mode, ES modules and a `src/` root. **Read [references/initializers.md](references/initializers.md) before running either**, and not only for the flags: it is where the two costs of the TypeScript default are written down — it is the resolution that meets the emitter's extensionless-import defect, and `tsc --init` switches off the `@types/node` the line before it installed — and both are the kind of thing the integration, not the scaffold, gets blamed for. + +Nothing beyond what the initializer writes is authored by this skill: no example code, no folder layout of its own, no opinion the framework did not ship. + +**On Python, finish with `uv sync` from inside ``.** `uv init` writes a `pyproject.toml` and stops: no lock file, no environment. `/pipelex-integrate` picks the package manager off the lock file and reads no lock file as `pip install` into the active environment, so a project handed over without one is a uv project installed into with pip — and `uv init` left no environment for pip to find either. The recipes that end in a `uv add` are locked by that command; the minimal and script forms, which are exactly what "no framework named" selects, are locked only by this line. [references/initializers.md](references/initializers.md) carries it with the rest of the post-initializer sequence. + +### Step 3: Version control and the pristine commit + +**Test whether `` is its own repository; never infer it from which initializer ran.** `git -C rev-parse --show-toplevel` must print `` itself, and when it does not, run `git init -b main` in the directory before staging anything. The list of initializers that `git init` on their own is not a substitute for that test, because membership in it is conditional: `uv init` initializes a repository when it creates a standalone project and **does not** when the parent directory already holds one, where it makes `` a workspace member of the enclosing project instead. A `` with no `.git` of its own is governed by whatever repository encloses it — the user's — and `git -C ` sets git's working directory without scoping anything, so the staging below would sweep that whole worktree: the user's unrelated untracked files, wherever they sit, committed into their repository under this skill's message. That is the one outcome this step exists to prevent, and the read-back catches it only if you read the paths and not just the count. + +**Then confirm there is a `.gitignore` covering the dependency tree and the build output**: `npm init -y` and `tsc --init` write none, so the minimal TypeScript recipe — and the Express and library recipes built on it — reach this step with a populated `node_modules/` and nothing excluding it, and the staging below would commit the whole dependency tree into the one commit that is supposed to be a readable baseline. Write `node_modules/`, `dist/` and `.env` into a `.gitignore` first where the initializer left none, and read back what is staged — `git -C diff --cached --name-only` over the paths themselves, not a `--stat | tail -1` whose count cannot tell a correct scaffold from a swept-up worktree — before committing. Then the one commit, for the same reason as branch A: + +```bash +git -C add -A -- . && git -C commit -m "Scaffold project" -- . +``` + +The `-- .` pathspec is the second half of the guard, and it is on **both** commands for a reason: `add -A -- .` bounds what this command stages, but a bare `git commit` then commits the *whole index*, so anything the user had staged elsewhere in an enclosing repository before the session goes into the commit under this skill's message. With the pathspec on the commit too, the staging and the commit are both held to `` and below even when the repository turns out to be an enclosing one, and the user's own staged work is left staged and uncommitted where they put it. + +**An initializer that commits as well as `git init`s has already made this commit.** `create-next-app` is one: it runs `git init`, stages everything and commits, so the tree is clean and the command above stops with `nothing to commit` — which is the initializer having done the job, not a failure of it. Take that commit as the pristine one, exactly as branch A takes GitHub's, and name it and its message in the report. Never force a second empty commit on top of it. + +### Step 4: The env file + +Write `.env.example` with the two lines the starters share, make sure `.env` is gitignored, and copy the example to `.env` under the same key rule as branch A: + +``` +PIPELEX_BASE_URL=https://api.pipelex.com +PIPELEX_API_KEY= +``` + +### Step 5: Hand off + +Add **no** SDK dependency and create **no** empty `methods/` directory: `/pipelex-integrate` adds `@pipelex/sdk` or `pipelex-sdk` when it writes the first call site, and creates `methods//` when it places the first bundle. A project with nothing to integrate yet has nothing Pipelex-shaped in it beyond the env convention, and that is correct. + +## The report + +Say, in this order: what was created and where; which template or initializer it came from, at which version and SHA; that this skill made exactly one commit and what it holds; what the bootstrap changed and that those changes are uncommitted for review, in the bootstrap's own words (branch A); which env file was written and whether the key was filled from the environment or left for the user; the demos the starter still carries and where the README's removal checklist is (branch A); and the hand-off. + +Two lines are easy to forget and matter: + +- **The project's own instructions and skills load in a session started inside it.** Its `CLAUDE.md` / `AGENTS.md` and its `release` and `bump-*` skills are not in the current session; `cd `, then starting Mistral Vibe there, is how they arrive. +- **`/pipelex-integrate` still works from here meanwhile**, because the Pipelex workshop writes anywhere under the directory the harness was launched in, and the new project sits there. One exception, and it is the version-manager machine of step 1: the workshop is spawned with `npx` on the **harness's** own `PATH`, which no activation of yours reaches, so a `node` only reachable through `nvm` or `fnm` means no workshop at all. Say so there instead — the hand-off needs the harness restarted from a shell where the runtime is active. Hand the method to it by opening `../pipelex-integrate/SKILL.md` and following it; a bundle that lives elsewhere on disk is copied into the project by that skill. No method yet → `/pipelex-design` first. + +## When something goes wrong + +| Condition | Do this | +|---|---| +| A toolchain piece is missing (Node below the floor, no `uv`, no git) | STOP, name the exact missing piece and the starter README's line about it; never install a toolchain — first check a version manager the machine already has (`nvm`, `fnm`, `volta`, `asdf`, `mise`) and use its runtime, saying so | +| The target directory exists and is not empty — anything at all beyond a lone `.git` | STOP, ask for another; never delete, move or write into it, and never offer to. **A directory holding nothing but `.git` is empty here and is written into**; that exception is the one directory entry by name and not a class, so `.DS_Store`, `.idea/`, `.vscode/`, `Thumbs.db` and anything else still refuse. On branch A that directory is served by the acquisition beside it (Step 2) and never by a `git clone` into it, which git refuses outright | +| `git clone` fails (network, permissions) | report git's error verbatim; nothing to clean up beyond an empty directory, and on the acquisition beside an existing repository the chain removes its own temporary path and copies nothing | +| `gh` is absent or not authenticated | fall back to the local clone; say the GitHub repository can be created later with `gh repo create --source .` | +| The clone carries no `bootstrap` skill | follow the README's manual list; say the template changed | +| The bootstrap's checks are red | its own rule: fix the cause and re-run; never hand off on red | +| An initializer is interactive with no non-interactive form | hand the command to the user to run in the session; resume after | +| `PIPELEX_API_KEY` is not in the shell environment | leave the value empty in the env file; say where a key comes from and where it goes; never ask for it in the conversation | +| you need to know whether a key is set | test it without printing it (`[ -n "${PIPELEX_API_KEY:-}" ] && echo set`); never `env \| grep PIPELEX`, never echo the value, never move it with a file-editing tool, never read the env file back — a key in the transcript is a key to rotate | +| The working directory is the template's own checkout (its `origin` remote points at `Pipelex/pipelex-starter-…`) | STOP: this is the template, not a copy of it — acquire a copy in another directory. This is what separates it from the fresh-clone shortcut, which is a copy of the template and carries somebody else's `origin` or none | + +## Reference + +- [references/starters.md](references/starters.md) — the two starters side by side: what each brings, its prerequisite floors, the acquisition commands, its env file, its bootstrap, its demos and their removal checklist, and the codegen harness `/pipelex-integrate` will find. +- [references/initializers.md](references/initializers.md) — per language, the minimal default and the common frameworks' non-interactive initializers, whether each runs `git init`, and where the import package or `src/` root lands. +- `/pipelex-integrate` — the skill this one hands every project to. diff --git a/pipelex-vibe/skills/pipelex-scaffold/references/initializers.md b/pipelex-vibe/skills/pipelex-scaffold/references/initializers.md new file mode 100644 index 0000000..bd2e5c0 --- /dev/null +++ b/pipelex-vibe/skills/pipelex-scaffold/references/initializers.md @@ -0,0 +1,50 @@ +# Ecosystem initializers + +Branch B of `/pipelex-scaffold` runs an initializer that already exists and authors nothing of its own beyond what that initializer writes. This file lists the common ones with their non-interactive forms. Flags change between versions: when a command below prompts anyway or rejects a flag, read its `--help` and prefer its own current non-interactive form over improvising a layout by hand. An initializer with no non-interactive form is handed to the user to run in the session. + +After the initializer: `git init -b main` only if `git -C rev-parse --show-toplevel` does not print `` itself — **the test, never the inference from which initializer ran**, because the `git init?` columns below are conditional and `/pipelex-scaffold`'s Step 3 gives the case that makes them so (`uv init` initializes nothing when the parent directory already holds a project, and staging in a `` governed by an enclosing repository sweeps the user's whole worktree into this skill's commit). Then one pristine commit (`Scaffold project`), the two-line `.env.example` (`PIPELEX_BASE_URL=https://api.pipelex.com`, `PIPELEX_API_KEY=`), `.env` gitignored and copied from it. No SDK dependency — `/pipelex-integrate` adds it. + +**On Python, run `uv sync` from inside `` before the pristine commit, and commit the `uv.lock` it writes.** `uv init` writes a `pyproject.toml` and nothing else: no lock file and no environment. That matters at the hand-off rather than here, because `/pipelex-integrate` reads the lock file to decide how to install — `uv.lock` selects uv, and **nothing** selects `pip install` into whatever environment happens to be active. A project handed over without its lock is therefore a uv project that the next skill installs into with pip, or cannot install into at all, because `uv init` left no environment for pip to find either. The recipes below that end in a `uv add` get a lock from that command; the minimal and script forms, which are the defaults when no framework is named, get one only from this step. + +## Python + +| Want | Command | `git init`? | Where the import package lands | +|---|---|---|---| +| **Minimal (the default when no framework is named)** | `uv init --package --no-workspace ` | yes (`--vcs none` to skip) | `src//` with `__init__.py` and a console-script entry in `pyproject.toml` | +| A script-style app rather than a package | `uv init --app --no-workspace ` | yes | `main.py` at the root — no import package; `/pipelex-integrate` will put the generated tree under `generated/` at the root | +| FastAPI service | `uv init --package --no-workspace && (cd && uv add "fastapi[standard]")` | yes | as minimal | +| Django project | `uv init --package --no-workspace && (cd && uv add django && uv run django-admin startproject config .)` | yes (from `uv init`) | the Django project package `config/` plus `src//`; ask which one owns the Pipelex call sites | +| Typer CLI (the Python starter's shape, without the starter) | `uv init --package --no-workspace && (cd && uv add typer)` | yes | as minimal | +| An existing `pyproject.toml` layout the user prefers (poetry, pdm, hatch) | the tool the user names: `poetry new `, `pdm init --non-interactive`, `hatch new ` | poetry: no; pdm: no; hatch: no | per tool — `poetry new` and `hatch new` make `/` or `src//` | + +`uv init` refuses a directory that already holds a project; on an empty "here" directory use `uv init --package --no-workspace .` — it names the package after the directory. + +**`--no-workspace` is on every `uv init` above, and it is the same hazard the `uv add` parentheses below address, one command earlier.** Run inside a directory that already holds a `pyproject.toml` — the scaffold being made inside an existing project, which is the common case — a bare `uv init --package ` does not create a standalone project at all. It prints `Adding as member of workspace …`, appends a `[tool.uv.workspace]` table naming `` to **the user's own `pyproject.toml`**, and then the first `uv add` writes the lockfile at the *parent* root, so the new project has no `uv.lock` of its own and does not resolve standalone. Editing a file of the user's is exactly what this skill does not do, and a project that needs its parent to resolve is not the project the report says was handed over. `--no-workspace` leaves the parent untouched and gives `` its own lock. With no parent project it changes nothing, so it is safe to pass always, which is why it is not conditional. + +**Every `uv add` above runs inside ``, and the parentheses are why.** `uv add` resolves the project from its *working* directory upwards, so run from the parent it writes the dependency into whatever project it finds there — the user's own `pyproject.toml` and lockfile, when the scaffold is being made inside an existing workspace — or fails outright when it finds none. Neither is the new project. `uv add --directory ` is equivalent if you prefer a flag to a subshell. + +## TypeScript / JavaScript + +| Want | Command | `git init`? | Where `src/` lands | +|---|---|---|---| +| **Minimal (the default when no framework is named)** | `mkdir -p && cd && npm init -y && npm install --save-dev typescript @types/node && npx tsc --init --strict --module nodenext --target es2022 --rootDir src --outDir dist` then set `"type": "module"` in `package.json` — **but read the resolution note below first: prefer `--module esnext --moduleResolution bundler` unless the user actually wants Node's own resolution**, because `nodenext` is the shape that meets the emitter defect | no | `src/` (create it); `/pipelex-integrate` puts the generated tree under `src/generated/` | +| Next.js app (the JS starter's shape, without the starter) | `npm create next-app@latest -- --ts --app --src-dir --eslint --use-npm --yes` | yes (`--disable-git` to skip) | `src/app/`; generated tree under `src/generated/` | +| Vite + React | `npm create vite@latest -- --template react-ts` then `(cd && npm install)` | no | `src/` | +| Hono server | `npm create hono@latest -- --template nodejs --pm npm --install` | no | `src/` | +| Express server | the minimal recipe above, then `(cd && npm install express && npm install --save-dev @types/express)` | no | `src/` | +| Node library | the minimal recipe above | no | `src/` | +| pnpm / yarn / bun instead of npm | replace `npm create` with `pnpm create` / `yarn create` / `bun create`, and the install command accordingly; `/pipelex-integrate` reads the lockfile to pick the package manager for what it adds | — | — | + +`npm create @latest -- `: the `--` is what passes the flags to the initializer rather than to npm. The Next.js `--yes` accepts the initializer's defaults for every prompt not covered by a flag. + +**Every follow-on `npm install` above is parenthesised for the same reason every `uv add` is, and npm is the worse of the two.** `npm install` resolves the project it writes to from its *working* directory upwards, so run from the parent it adds the dependency to the user's own `package.json` and puts `node_modules/` in the user's tree — and where `uv add` at least fails outright when it finds no project nearby, npm finds the parent and silently succeeds, leaving the new project with nothing installed and no error to read. The `cd ` inside the minimal recipe's own `&&` chain does not reach a follow-on issued as a separate command, because shell state does not survive from one command to the next. Use the subshell, or `npm install --prefix `. + +**The minimal recipe's `--module nodenext` plus `"type": "module"` is exactly the shape that meets the ts-zod emitter's extensionless-import defect** (`pipelex-integrate`'s `references/typescript.md`, "Known defect"): the generated `binder.ts` fails the type check with `TS2835` and will not load at runtime. So say so when you hand a minimal TypeScript project to `/pipelex-integrate`, and when the user has no reason to prefer Node's own resolution, prefer a bundler-backed setup (Vite, Next.js) or `--module esnext --moduleResolution bundler`, which the defect does not touch. The two recipes also need `node_modules/` and `dist/` in a `.gitignore` before the pristine commit — neither `npm init -y` nor `tsc --init` writes one. + +**And `tsc --init` writes `"types": []` as an active key, which switches `@types/node` off on the line after the recipe installed it.** That empty array is the current `tsc --init` template's own default (TypeScript 7 writes it, with `// "types": ["node"],` commented out three lines below), and it means no `@types` package is loaded at all — so the call site `/pipelex-integrate` writes next fails with `TS2591` on `node:fs/promises`, on `node:path` and on `process`, and then cascading `TS7006` implicit-any on the `readdir` callback. Nothing about the errors points at the tsconfig, so set `"types": ["node"]` as part of the recipe rather than leaving it for the integration to discover. `npm install --save-dev typescript` also resolves to TypeScript 7 now; say which major the project got, because `tsc --init`'s defaults moved with it. + +## What every branch-B project shares afterwards + +- One commit, the pristine scaffold, so the user's first real change is a clean diff. +- `.env.example` and `.env` written, `.env` ignored, `PIPELEX_API_KEY` filled only from the shell environment. Both land *after* the pristine commit, which holds the initializer's output as it came, so they stay untracked for the user to review and commit — the same posture branch A leaves the bootstrap's edits in. +- Nothing else Pipelex-shaped: the SDK dependency, the `methods/` directory and the generated tree arrive with the first `/pipelex-integrate`. diff --git a/pipelex-vibe/skills/pipelex-scaffold/references/starters.md b/pipelex-vibe/skills/pipelex-scaffold/references/starters.md new file mode 100644 index 0000000..08550a3 --- /dev/null +++ b/pipelex-vibe/skills/pipelex-scaffold/references/starters.md @@ -0,0 +1,76 @@ +# The two Pipelex starters + +Both are GitHub **template repositories** under the `Pipelex` organization. Each is a real, CI-tested application against the hosted Pipelex API, not a parameterized template: the identity you see in a fresh clone (`pipelex-starter-js` / `Pipelex Starter`, or `piper` / `Piper`) is a placeholder that the starter's own `bootstrap` skill rewrites. Read the clone's `README.md` after acquiring it — the sections named below are where the details live, and they move as the starters evolve. + +## Side by side + +| | `pipelex-starter-js` | `pipelex-starter-python` | +|---|---|---| +| **Shape** | Next.js (App Router), React, TypeScript strict, Tailwind; a web app with one tab per method whose input form is rendered from the method's own contract by `@pipelex/mthds-form` | A Typer CLI with one command per method, printing JSON on stdout and a cost report on stderr; three execution modes (`blocking`, `attended`, `detached`) as separate sub-packages | +| **Pick it when** | people will use the methods in a browser: forms, uploads, live run status | the methods run from a terminal, a script, a batch job or a service, and the user wants Python | +| **SDK** | `@pipelex/sdk` | `pipelex-sdk` (import package `pipelex_sdk`) — the `pipelex` runtime is **not** a dependency | +| **Methods live in** | `methods//main.mthds`, or `methods//method.json` for a method that lives elsewhere (a catalog id or a published address) | `/methods//main.mthds` | +| **Generated types** | `src/generated//` — `types.ts`, `binder.ts`, `contracts.ts`, `codegen.lock`, `sources.json` | `/generated//` — `models.py`, `codegen.lock` | +| **Codegen harness** | `npm run codegen` (keyed, dev), `npm run codegen:check` (offline, in `make check`), `npm run codegen:verify` (keyed, semantic); `make add-method METHOD=` scaffolds a remote method end to end | `make codegen` / `make codegen-check` — **both shell out to a `pipelex` CLI the starter does not depend on** (`PIPELEX=` in the Makefile); `/pipelex-integrate` knows this and writes into the same layout when that CLI is absent | +| **Toolchain floor** | Node ≥ the `engines.node` field of `package.json` (22.12 at writing); `npm` | `uv`; a Python inside `requires-python` of `pyproject.toml` (3.11–3.14 at writing) | +| **Env file** | `.env.local`, from `.env.example`: `PIPELEX_BASE_URL`, `PIPELEX_API_KEY`, `NEXT_PUBLIC_EXECUTION_MODE` | `.env`, from `.env.example`: `PIPELEX_BASE_URL`, `PIPELEX_API_KEY` | +| **Checks the bootstrap runs** | `npm install --package-lock-only`, then `make all` (lint, format check, typecheck, unit tests, build) | `make li` (lock + sync), then `make agent-check` and `make agent-test` | +| **Agent-facing files** | `CLAUDE.md`, `AGENTS.md`; skills `bootstrap`, `release`, `bump-sdk`, `bump-mthds-form` | `CLAUDE.md`; skills `bootstrap`, `release` | +| **Docs worth reading after bootstrap** | `docs/codegen.md`, `docs/add-method.md`, `docs/input-form.md`, `docs/adopt-in-an-existing-project.md`; README → "Swap in your own pipeline" and "Remove an example" | `docs/codegen.md`, `docs/cli-architecture.md`; README → the per-command sections | +| **Demos it carries** | several demo methods, one tab each; keep them as references or strip them with the README's "Remove an example" checklist | several demo methods, one CLI command each; keep them as references or remove the command and its method directory together | + +## Acquisition + +`` below is the one the choice above settled — `pipelex-starter-js` or `pipelex-starter-python` — and each block is a single chain for that one starter, never a menu to run top to bottom. + +Local clone with fresh history (the default — it produces what GitHub's "Use this template" button produces, a copy with no history and no remote): + +```bash +git clone --depth 1 https://github.com/Pipelex/.git || exit +git -C rev-parse HEAD +rm -rf /.git && git -C init -b main +git -C add -A -- . && git -C commit -m "Start from Pipelex/ ()" -- . +``` + +The version comes from `package.json` (`"version"`) on JS and from `pyproject.toml` (`version =`) on Python, read before the commit. **The `-- .` pathspec is on the commit as well as on the staging**, for the reason `/pipelex-scaffold`'s Step 3 gives in full: `add -A -- .` bounds what is staged, but a bare `git commit` then commits the whole index, so anything the user had staged elsewhere in an enclosing repository rides along under this skill's message. + +**The `|| exit` on the clone is load-bearing and is not decoration**, for the reason `/pipelex-scaffold`'s Step 2 gives in full: the line below it deletes a `.git` directory, and a clone that never ran — a network failure, or `` already existing — leaves that `rm -rf` to find whatever `.git` is actually at that path, destroying a repository of the user's irrecoverably. The guard only holds inside one shell, so when the two lines go out as separate commands, check the clone's exit status yourself before typing the `rm -rf`, and never type it on a path you have not just created. + +Into a directory whose only entry is `.git` — the one shape the "Where" rule reads as empty and `git clone` still refuses — acquire beside it and move in, so the user's own repository stands and the end state is the same as above: + +```bash +dir=$(cd && pwd) || exit 1 +tmp=$(mktemp -d "$(dirname "$dir")/.pipelex-starter-XXXXXX") || exit 1 +git clone --depth 1 https://github.com/Pipelex/.git "$tmp" || { rm -rf "$tmp"; exit 1; } +git -C "$tmp" rev-parse HEAD +rm -rf "$tmp/.git" || { rm -rf "$tmp"; exit 1; } +[ "$(ls -A "$dir")" = ".git" ] || { rm -rf "$tmp"; exit 1; } +cp -R "$tmp"/. "$dir"/ || { rm -rf "$tmp"; exit 1; } +rm -rf "$tmp" +``` + +Every part of that chain is load-bearing, and `/pipelex-scaffold`'s Step 2 gives each in full. **The first line resolves `` to an absolute path before the parent is computed from it**, without which a destination spelled `.` — the ordinary spelling, since the user is usually standing in the directory they just `git init`-ed — puts the temporary directory inside the destination and the `ls -A` line then refuses every time. **No `rm -rf` in it addresses a path under ``**: the template's history is discarded while the clone is still at a path `mktemp` made for this command, before anything moves, so the guarded-deletion problem above does not arise here at all. **`cp -R "$tmp"/. "$dir"/` carries the entries beginning with a dot** — `.gitignore`, `.env.example`, `.github/`, `.claude/` — every one of which `mv "$tmp"/* "$dir"/` leaves behind while exiting `0`. And **the `ls -A` line admits exactly one entry**, `.git`, which the clone no longer has, so the template can only add to the directory and anything else stops the run with nothing copied and the temporary path removed. It is one chain and goes out as one command, for the reason the paragraph above gives. + +GitHub repository, on request and after confirmation (visibility asked, default private; GitHub makes the initial commit, so no pristine commit of your own): + +```bash +gh auth status +gh repo create / --template Pipelex/ --private --clone +``` + +## The bootstrap you delegate to + +Both starters carry `.claude/skills/bootstrap/SKILL.md` with a bundled script (`scripts/bootstrap.mjs` / `scripts/bootstrap.py`). Read the file in the clone and follow it; the shape is the same on both: + +1. **Preflight** — confirms the identity is still the template's (`package.json` name `pipelex-starter-js`; `pyproject.toml` `name = "piper"`), notes a dirty tree, and on JS makes sure `node_modules/` exists (`make install`). +2. **Collect** — the package name (kebab on JS, underscores on Python, everything else derives from it), a display title, a one-line description; optionally author name **and** email (never one without the other), the repository URL, and the license (MIT kept, proprietary, or another SPDX id; the copyright holder and year). Pass what the conversation already holds so it asks once for the rest. +3. **Dry run** — the script with `--dry-run` prints the plan; the user confirms. +4. **Run** — the same command without `--dry-run`; on Python the package directory is renamed with `git mv`, which is why the pristine commit must exist first. `--clean` strips the template-only prose; keep it unless the user wants the template charter kept. +5. **Verify** — the lock file is re-synced and the project's own checks run; red is fixed, not skipped. +6. **Self-removal** — `rm -rf .claude/skills/bootstrap`, unstaged like everything else; the user reviews with `git status` and `git diff` and commits when ready. + +The starter's rules are yours while you run it: never commit on the user's behalf, always dry-run first, never touch `.github/` or the `release` skill's logic. + +## What `/pipelex-integrate` finds afterwards + +A bootstrapped starter is a project that **owns a codegen harness**, and `/pipelex-integrate` defers to it: on JS it drops a bundle under `methods//` and runs `npm run codegen`, or runs `make add-method METHOD=…` for a catalog or published method, then follows `docs/codegen.md` and the existing actions for the fan-out; on Python it places the bundle under `/methods//` and runs `make codegen` when a `pipelex` CLI is available, writing into `/generated//` through the Pipelex workshop when it is not. It never writes a second generated layout beside the starter's own. diff --git a/pipelex/skills/pipelex-design/SKILL.md b/pipelex/skills/pipelex-design/SKILL.md index c169860..5f5d3d0 100644 --- a/pipelex/skills/pipelex-design/SKILL.md +++ b/pipelex/skills/pipelex-design/SKILL.md @@ -191,7 +191,7 @@ After the gate: 1. **Organize only when the layout needs it.** A direct result that is already coherent skips `/pipelex-organize`. A converged stepwise result normally invokes it automatically because one-definition-per-file construction history and satisfied headers need regrouping. A naturally coherent result in either mode does not take an organization round trip solely for process compliance. 2. **Project the input schema.** Call `mthds_inputs_template` with the final whole-bundle `files` submission plus `explicit: false`. Show the returned compact template, but **do not save it as `inputs.json`** — input preparation belongs exclusively to `/pipelex-inputs`. 3. **Present the flow.** Point to the interactive method graph where the host rendered the valid verdict's view; in terminal hosts, present a concise text flow of the final structure. -4. **Hand off inputs — and the code.** Suggest preparing real inputs with `/pipelex-inputs`. Then, when the workspace holds a codebase (a `package.json` or a `pyproject.toml`), say that `/pipelex-integrate` wires the method into it with generated types and a typed call site. +4. **Hand off inputs — and the code.** Suggest preparing real inputs with `/pipelex-inputs`. Then, when the workspace holds a codebase (a `package.json` or a `pyproject.toml`), say that `/pipelex-integrate` wires the method into it with generated types and a typed call site; when it holds none and the user wants an application around the method, `/pipelex-scaffold` creates one and hands it to `/pipelex-integrate`. > **NEVER write `inputs.json` manually.** If the user provides files, paths, or wants to run with real data, invoke `/pipelex-inputs` — it handles the template, path resolution, placeholder formatting, and file copying. diff --git a/pipelex/skills/pipelex-integrate/SKILL.md b/pipelex/skills/pipelex-integrate/SKILL.md index 5f517fd..81f8aa6 100644 --- a/pipelex/skills/pipelex-integrate/SKILL.md +++ b/pipelex/skills/pipelex-integrate/SKILL.md @@ -1,6 +1,6 @@ --- name: pipelex-integrate -description: Wire an MTHDS method into a Python or TypeScript codebase with generated, drift-proof types and one typed call site that runs it through @pipelex/sdk or pipelex-sdk. Use when the user says "use this method in my app", "call this from my code", "generate types for this method", "wire the method into my project", "add this pipeline to my service", "typed client for this method", "integrate the method", "refresh the generated types", "regenerate the types", "the types are stale", or wants application code that runs a .mthds method — from a local bundle, a catalog id (mt_…) or a published method_ref address. Also the refresh path after a bundle edit. Not for authoring or editing the method itself (/pipelex-design, /pipelex-edit), and not for a workspace with no project in it yet, where the project has to exist first. +description: Wire an MTHDS method into a Python or TypeScript codebase with generated, drift-proof types and one typed call site that runs it through @pipelex/sdk or pipelex-sdk. Use when the user says "use this method in my app", "call this from my code", "generate types for this method", "wire the method into my project", "add this pipeline to my service", "typed client for this method", "integrate the method", "refresh the generated types", "regenerate the types", "the types are stale", or wants application code that runs a .mthds method — from a local bundle, a catalog id (mt_…) or a published method_ref address. Also the refresh path after a bundle edit. Not for authoring or editing the method itself (/pipelex-design, /pipelex-edit), and not for a project that does not exist yet (/pipelex-scaffold). allowed-tools: - Bash - Read @@ -63,7 +63,7 @@ Automatic by default: state the target, the destination and the generator in one - **A published address** — `method_ref: "github.com//[/]@"`. The tag is the pin: a `method_ref` **without a tag** floats and is refused for a committed integration; ask for the tag. - **A catalog id** — `method_id: "mt_…"` (resolve a name through `mthds_list_methods` when present). The catalog is unversioned: an edit to the stored method silently invalidates committed types with nothing offline to detect it. Say so in one line, recommend committing the source or publishing an address, and proceed only on the user's say-so. -**The project** is the nearest directory holding a `package.json` or a `pyproject.toml` (or `setup.py` / `requirements.txt`) above the working area. A workspace holding several (a monorepo, a full-stack repo) is a question — which app? — never a guess. **No project at all** → this is not an integration yet: say so and stop, because there is nothing to wire the method into. Creating the project is the user's own step — a starter, their framework's initializer, their own layout — and integration begins once it exists. +**The project** is the nearest directory holding a `package.json` or a `pyproject.toml` (or `setup.py` / `requirements.txt`) above the working area. A workspace holding several (a monorepo, a full-stack repo) is a question — which app? — never a guess. **No project at all** → this is not an integration yet: offer `/pipelex-scaffold`, which creates one and hands it back here. Then look for a **codegen harness**: a `codegen` script in `package.json` or a `codegen` Makefile target, a `sources.json` carrying a `derived` map, `docs/codegen.md` or `docs/add-method.md`, a `methods/` directory beside `src/generated/` or `/generated/`. Either of the first two decides; the rest only corroborate. If the project has one, follow [A project that owns a codegen harness](#a-project-that-owns-a-codegen-harness) from here. diff --git a/pipelex/skills/pipelex-scaffold/SKILL.md b/pipelex/skills/pipelex-scaffold/SKILL.md new file mode 100644 index 0000000..daab34a --- /dev/null +++ b/pipelex/skills/pipelex-scaffold/SKILL.md @@ -0,0 +1,212 @@ +--- +name: pipelex-scaffold +description: Start a new project that will call MTHDS methods through Pipelex, in TypeScript or Python — from one of the Pipelex starter templates or from the ecosystem's own initializer — and hand it to /pipelex-integrate. Use when the user says "start a new project with Pipelex", "I have a method and need an app around it", "create a Next.js app that runs my method", "set up a Pipelex project from scratch", "new Python CLI for this method", "which starter should I use", "bootstrap a Pipelex project", or wants a codebase where none exists yet. Also use when the user is standing in a freshly cloned pipelex-starter-js or pipelex-starter-python that has not been renamed yet — this skill runs the template's own bootstrap for them. Not for adding Pipelex to code that already exists: that is /pipelex-integrate. +allowed-tools: + - Bash + - Read + - Write + - Edit + - Grep + - Glob + +--- + +# Scaffold a project for Pipelex methods + +Give a user who has no project yet a project that is ready for `/pipelex-integrate`. This skill has exactly two branches and carries no templates of its own: + +- **One of the Pipelex starters** when the user wants the opinionated shape: `pipelex-starter-js` for a web app whose forms are rendered from the methods' own contracts, `pipelex-starter-python` for a CLI or service that runs methods in the three execution modes. You acquire the template, commit it once as it came, then run the clone's **own** `bootstrap` skill — the rename logic lives in the starters and is never reimplemented here. +- **The ecosystem's own initializer** when the user wants their framework or a minimal project: `uv init --package`, `npm create next-app@latest`, `django-admin startproject`, whatever the framework documents. You run it; you never assemble a project by hand. + +Both branches end the same way: an env file that follows the starters' convention, one pristine commit that makes everything after it reviewable, and the hand-off — to `/pipelex-integrate` when a method exists, to `/pipelex-design` first when none does. + +**What this skill is not.** Not a template engine (no cookiecutter, no copier, no framework matrix of its own), not a bootstrap (the starters own theirs), not a runner or a dev-server launcher, not a deployer. It needs no MCP tool and no API key: git, the starters' scripts and the ecosystem's initializers are all it uses. + +## Choosing the branch + +A cheap, reliable signal decides; an inconclusive one asks one question; nothing is guessed twice. + +| Question | Signals, in order | When inconclusive | +|---|---|---| +| **Language** | the user's word; the language the method's consumer is written in; a framework the user named | ask | +| **Which branch** | a **named framework** the starters do not carry (FastAPI, Django, Express, Hono, a plain library, a Lambda) → the initializer; **"minimal"**, **"no demo code"**, **"just a project"** → the initializer; a **web app people use in a browser**, forms, an upload flow → the JS starter; a **CLI, script, batch job, worker or service** in Python → the Python starter | one question offering the matching starter first, saying what it brings (durable runs, forms or CLI modes, codegen wiring, CI, its own `release` skill) and what it costs (demos to keep as references or to strip) | +| **Where** | the directory the user named; **"here"** when the working directory is empty — **and a directory whose only entry is `.git` is empty for this rule**, because `mkdir my-app && cd my-app && git init` is an ordinary way to arrive here and a repository the user made is not work of theirs to write over: branch B runs its initializer in the directory as it stands and leaves that repository alone — it does **not** re-run `git init` there, since Step 3's test finds `` is already its own repository — while branch A acquires beside it and moves in, so the repository already there goes on standing either way (Step 2); else a kebab-case directory named after the project | ask; never write into a directory that exists and is not empty, and never offer to move, delete or merge what it holds to make room — the answer is another directory. **A lone `.git` is the only entry that does not make a directory non-empty, and that is a ruling about `.git` and nothing else** — not a class of files you may decide to overlook. Every other entry still refuses, `.DS_Store`, `.idea/`, `.vscode/` and `Thumbs.db` included: judging which of a user's files matter is the thing this rule exists to forbid, and a list that grows by guesswork is how it would come back. The rule is about **a directory you are creating a project in**, which is why the fresh-clone shortcut below is not an exception to it: there the project is already there and you are finishing it, not writing over someone's work | +| **GitHub or local** | the user asked for a GitHub repository → `gh repo create --template`, after confirmation; otherwise a local clone with fresh history | local | + +**The fresh-clone shortcut.** A starter clone already in the working directory that has not been bootstrapped — `package.json` still says `pipelex-starter-js`, or `pyproject.toml` still says `name = "piper"` — **and whose `origin` does not point at `Pipelex/pipelex-starter-…`** is branch A entered at step 4: acquisition already happened, so go straight to running the clone's bootstrap. Do not clone again. + +[references/starters.md](references/starters.md) compares the two starters and carries every command below; [references/initializers.md](references/initializers.md) carries the initializers. + +## Mode + +Automatic by default, with the plugin's usual rules: an explicit user signal wins ("just do it" → automatic; "walk me through" → interactive); a genuinely ambiguous branch is one question, asked once; a request that gave every input up front proceeds without re-asking. Two things always confirm, in every mode: **`gh repo create`**, because it creates a repository on GitHub, and whatever the clone's bootstrap skill confirms on its own account. The pristine commit does not need confirmation **on a directory this skill created** — it holds the template as it came, and no user content is at stake. **The acquisition into a directory that already held a repository is the exception**, and it is a third thing that always confirms: there the commit lands on the user's branch, on top of their history, and `add -A -- .` records whatever their worktree was already showing along with the template (Step 3). None of the three grounds above holds, so state what will be staged and what it will land on, and ask — in every mode. + +## Branch A — one of the starters + +### Step 1: Prerequisites + +Check before touching anything, and **stop** on a missing piece with the exact thing missing and the starter README's own line about it. Never install a toolchain — but a runtime the machine already has and only the `PATH` is missing is not a missing piece: when `node` or `uv` is absent while a version manager on the machine carries one (`nvm`, `fnm`, `volta`, `asdf`, `mise`), activate it for this work and say in the report which one you used and that the user's own shell may not have it. Stop only when no usable runtime can be reached that way. + +**Activating it means resolving it to an absolute path, not sourcing a shell.** Your shell state does not survive from one command to the next — each one starts again from the user's profile, which is the profile that did not have the runtime — so `. nvm.sh` or `eval "$(fnm env)"` in one call buys nothing in the next. Resolve the binary once (`ls "$NVM_DIR"/versions/node/*/bin/node`, `volta which node`, `mise which node`, `asdf which node`, `fnm exec --using= -- which node`), keep that directory, and prefix **every** later command with it — `PATH=":$PATH" …` — the clone's bootstrap and its `make all` / `make agent-check` included, because those are separate commands too. Verify the runtime answers under that prefix **before** step 2, so a machine you cannot actually reach stops while nothing has been created; discovering it at step 4 has already spent the pristine commit. + +Three things this clause does not license. **A shim is not a runtime**: `asdf` and `mise` put a `node` on the `PATH` that exists and then fails with "no version set", so the test is that `node --version` *answers*, not that the binary resolves — and that case is a stop, not a manager to activate. **The floor still applies**: a manager holding Node 18 does not satisfy the starter's `engines` floor, and "a runtime the machine already has" never means a version below it. And **`volta` and `mise` install on first use** — `volta run`, `mise x` and `mise use` will fetch a version they do not have — which is the toolchain install this step forbids: use only a version the manager already holds, and stop rather than let it download one. Note too that `nvm`, `fnm` and `volta` manage Node alone and can never supply `uv`. + +- **JavaScript**: Node at or above the floor the starter's `package.json` `engines` field names (`node --version`; 22.12 at writing — the SDK is ESM-only and the starter's e2e specs `require()` it), and `npm`. +- **Python**: `uv` on the PATH (the starter installs and locks with it) and a Python inside the starter's `requires-python` range that `uv python find` can see (3.11 to 3.14 at writing). +- **Both**: `git`. The GitHub form also needs `gh` authenticated — `gh auth status`. + +### Step 2: Acquire the template + +**Local, the default.** Clone shallow, read the template's identity, then detach from it: + +```bash +git clone --depth 1 https://github.com/Pipelex/.git || exit +git -C rev-parse HEAD # the template SHA, for the commit message +# the template version: package.json "version" (JS) or pyproject.toml version (Python) +rm -rf /.git && git -C init -b main +``` + +The `|| exit` on the clone is not decoration: the line below it deletes a `.git` directory, and if the clone never ran — a network failure, or `` already existing — that `rm -rf` finds whatever `.git` is actually at that path. On a directory the skill just created it destroys nothing; on a repository of the user's it destroys their history irrecoverably. Run the destructive line only behind a clone that succeeded, and never type it on a path you have not just created. + +The clone's `.git` is removed on purpose: it is the template's history and remote, and leaving it would make `git status` and a future `git push` belong to Pipelex's template rather than to the user's project. This is exactly what GitHub's "Use this template" button produces — a copy with no history and no remote — and it is why the starters' READMEs tell humans not to clone directly. Fresh history is how you honour that. + +**Local, into a directory that already holds a repository.** The "Where" rule reads a directory whose only entry is `.git` as empty, and `git clone` cannot serve it: git refuses any destination already holding a `.git` and stops with `destination path '' already exists and is not an empty directory`. So acquire **beside** the directory and move in. The end state is the one the default recipe reaches — the template in ``, no template history, no Pipelex remote — and the repository the user made goes on standing instead of being replaced: + +```bash +dir=$(cd && pwd) || exit 1 +tmp=$(mktemp -d "$(dirname "$dir")/.pipelex-starter-XXXXXX") || exit 1 +git clone --depth 1 https://github.com/Pipelex/.git "$tmp" || { rm -rf "$tmp"; exit 1; } +git -C "$tmp" rev-parse HEAD # the template SHA, for the commit message +# the template version: package.json "version" (JS) or pyproject.toml version (Python) +rm -rf "$tmp/.git" || { rm -rf "$tmp"; exit 1; } +[ "$(ls -A "$dir")" = ".git" ] || { rm -rf "$tmp"; exit 1; } +cp -R "$tmp"/. "$dir"/ || { rm -rf "$tmp"; exit 1; } +rm -rf "$tmp" +``` + +**The first line resolves the destination, and that is what keeps the temporary path a sibling rather than a child.** `` is very often `.` here: `mkdir my-app && cd my-app && git init` is the "Where" rule's own account of how a user arrives at a directory holding nothing but `.git`, and they then ask for the project *here*. `dirname .` is `.`, so deriving the parent from the spelling would put the temporary directory **inside** the destination, where the `ls -A` line below finds it sitting beside `.git` and refuses — every time, on exactly the case this section exists to serve. Resolving to an absolute path first also pins the destination for the rest of the chain, so no later line can be re-read against a working directory that has moved, and it is what lets every mention below be quoted: a name with a space reaches `cp` whole instead of arriving as two arguments. + +**No `rm -rf` here ever addresses a path under ``, and that is the ordering rather than a coincidence.** The template's history is discarded while the clone is still at a path `mktemp` made for this one command, so the destructive line is spent before anything moves: the only two paths any delete is pointed at are `"$tmp/.git"` and `"$tmp"`. Compare the default recipe, where the same line runs on `` itself and `|| exit` is the whole thing standing between it and a user's history. Here there is nothing for a guard to hold, which is what makes this the form you may aim at a directory holding somebody's repository. + +**`cp -R "$tmp"/. "$dir"/`, and never `mv "$tmp"/* "$dir"/`.** The glob matches no entry beginning with a dot, so the naive move leaves `.gitignore`, `.env.example`, `.github/` and `.claude/` behind, exits `0`, and the line after it deletes the temporary directory they are still sitting in — a starter arriving without its `.gitignore`, whose pristine commit then swallows `node_modules/`, reported as a success. The trailing `/.` copies the directory's *contents*, dotfiles included, with no shell globbing involved at all. Confirm it with `ls -A "$dir"` after the copy rather than trusting the form. + +**The `ls -A` line is the "Where" rule read again, against the copy.** It is not the decision — the "Where" question settled that — it is the last look before anything lands, put next to the copy so nothing can change between the two. It admits exactly one entry, `.git`, which the clone has not had since the line above: a collision is therefore impossible rather than merely unlikely, and the template can only add to the directory. Anything else — `.git` beside a file of the user's, a `.DS_Store`, a `README.md` they wrote — stops here with nothing copied, the temporary path removed and the directory as it was. A discarded shallow clone is the cheap half of that trade. Nothing of the user's is overwritten, moved or deleted to make room, here or anywhere. + +**The chain goes out as one command.** The guards hold only inside one shell — the same reason the `|| exit` above is load-bearing, stated in full in [references/starters.md](references/starters.md) — and split across separate calls this one loses its cleanup too, leaving the temporary directory beside the user's project with no line left to remove it. + +**Nothing is initialized here.** The default recipe ends `git init -b main` because it has just deleted the only repository at that path. This one ends on the user's repository, their branch and their remote, which is the whole point of taking the long way round. + +**GitHub, on request.** When the user asked for a repository on GitHub: + +```bash +gh repo create / --template Pipelex/ --private --clone +``` + +`--clone` takes no destination argument: it clones into `./` under the current working directory, so for this form `` **is** `` — either choose the repository name to match the directory the "Where" question settled, or rebind `` to `./` before step 4, because every step after this one addresses `` literally. Visibility is the user's call: ask, default `--private`. Creating a repository on GitHub is outward-facing, so **state the exact command and confirm before running it**, in every mode. GitHub writes the initial commit itself — skip step 3 and continue at step 4. If `gh` is absent or not authenticated, fall back to the local clone and say the repository can be created later with `gh repo create --source .`. + +Both forms take the template's default-branch head. Do not offer a release tag unless the user asks for one. + +### Step 3: Commit the pristine template — exactly once + +```bash +git -C add -A -- . && git -C commit -m "Start from Pipelex/ ()" -- . +``` + +This is the **one commit this skill makes**, and it is load-bearing twice over: the Python starter's bootstrap renames the package directory with `git mv`, which refuses a path git does not track, and a committed baseline is what turns the bootstrap's edits into a diff the user can read before committing them. Nothing of the user's is in it — it is the template as it came. One qualification on the acquisition into a directory that already held a repository: the commit lands on the user's branch, on top of their history rather than opening a new one, and `add -A -- .` also records whatever deletion their worktree was already showing — their own pending change and not one this skill made, so name it in the report instead of undoing it. That is the commit the Mode section sends back for confirmation, and this is what to put in front of the user: `git -C status --short` before staging says what will ride along, and it is the difference between a baseline commit and a line in their history that says "Start from Pipelex/…" over a change they made. + +### Step 4: Run the clone's own bootstrap + +Read `/.claude/skills/bootstrap/SKILL.md` and follow it as written. The project's skills are not loaded in this session — it began elsewhere — so read the file; do not look for a `/bootstrap` command. Run every command it gives from inside the project directory (`cd && …`, or `-C `), because that skill assumes it is standing in the repo root. + +Feed it what the conversation already holds — the project name, title, description, author, repository URL, license — so that it asks once, consolidated, for whatever is left, exactly as its own Step 2 says. It dry-runs, previews, runs, re-syncs the lock file, runs the project's own checks (`make all` on JS; `make agent-check` and `make agent-test` on Python), and removes itself. Its rules stand unchanged: it never commits, its edits stay uncommitted for the user's review (the Python renames are staged by `git mv`, which its skill explains), and a red check is fixed, never skipped. **Add nothing to that procedure and reimplement none of it.** If the clone carries no bootstrap skill — a future template dropped it — follow the README's "manual equivalent" list and say that the template changed. + +### Step 5: The env file + +```bash +cp -n /.env.example /.env.local # JS: Next.js reads .env.local +cp -n /.env.example /.env # Python: python-dotenv reads .env +``` + +`-n` because this is the one step that can destroy something of the user's. On the fresh-clone shortcut the directory is one they were already working in, and a plain `cp` would overwrite an `.env.local` they had filled with their own key — the skill's whole posture is that nothing of the user's is ever cleared, and an env file is the most expensive thing in the tree to lose. An existing env file is left exactly as it is; read whether it already carries a key with the file-side test below, and say in the report that you kept theirs. + +Fill `PIPELEX_API_KEY` **from the shell environment when it is set there**, and leave it empty otherwise, telling the user where a key comes from (`app.pipelex.com`) and that this file is where it goes. **Never print a key, and never ask for one in the conversation.** Test for it without printing it — `[ -n "${PIPELEX_API_KEY:-}" ] && echo set || echo unset`. + +**The value moves only through a shell that expands the variable itself, and never through you.** A redirection is safe precisely because the shell does the expanding and only the variable's *name* is transcribed. **Append only when the file does not already carry a key**, which is the other half of the `-n` above: `grep -q '^PIPELEX_API_KEY=.\+' /.env.local || printf 'PIPELEX_API_KEY=%s\n' "$PIPELEX_API_KEY" >> /.env.local`. On the fresh-clone shortcut the file is one the user may have filled themselves, and an unconditional append puts a second assignment *after* theirs — every dotenv reader resolves a repeated name to the later line, so their working key is silently replaced by whatever the shell happened to export, while the report tells them you kept theirs. A file-editing tool is the one form that cannot be made safe, whatever it is called — it takes a **literal** string, so you would have to know the value to pass it, and a tool call's parameters are the transcript. So: no file-editing tool on a line carrying the key, no `env | grep PIPELEX`, no `echo $PIPELEX_API_KEY`, no command substitution in a message, and **no reading the env file back** once written — `cat .env.local`, a `grep` over it, or opening it to check your work is the reflex after writing and the first move when a later step fails, and it puts the key in the transcript just as surely. To confirm the write landed, test the file the same way you tested the environment: `grep -q '^PIPELEX_API_KEY=.\+' /.env.local && echo filled || echo empty`. A key in the transcript is a key to rotate, and it is not yours to spend. + +A value that passed the test is copied verbatim and never inspected, so say in the report that it was taken from the environment **and not validated** — a placeholder someone exported once passes a presence test and fails the first run, and this skill never calls the API, so it cannot tell the difference. `PIPELEX_BASE_URL` stays as the example ships it. Confirm the file is gitignored before writing a key into it — both starters ignore it, but check. + +### Step 6: Verify and hand off + +The bootstrap's own checks are the verification; do not start `make dev`. Write the report (below), then hand the user's method to `/pipelex-integrate`, which recognizes the starter's codegen harness (`npm run codegen`, `make codegen`, `make add-method`) and defers to it rather than writing a second one. + +## Branch B — the ecosystem's initializer + +### Step 1: Prerequisites + +As in branch A, for the language chosen. + +### Step 2: Run the initializer — never assemble by hand + +- **A named framework** uses its documented initializer with its non-interactive flags: `npm create next-app@latest -- --ts --app --src-dir --eslint --use-npm --yes`, where the `--` is what passes the flags to the initializer instead of to npm and without it `create-next-app` prompts; `uv init --package --no-workspace ` then `uv add "fastapi[standard]"` **from inside ``**, because `uv add` writes to whatever project its working directory resolves to and from the parent that is the user's, not the new one; and so on — [references/initializers.md](references/initializers.md) carries the common ones. An initializer that only runs interactively is handed to the user to run (typing `! ` in the prompt runs it inside this session), and you resume when it is done. +- **No framework named** takes the language's own minimal initializer: Python → `uv init --package --no-workspace `, which gives the import package `/pipelex-integrate` wants and a console-script entry; TypeScript → `npm init -y`, then `npm install --save-dev typescript @types/node` and `npx tsc --init` with strict mode, ES modules and a `src/` root. **Read [references/initializers.md](references/initializers.md) before running either**, and not only for the flags: it is where the two costs of the TypeScript default are written down — it is the resolution that meets the emitter's extensionless-import defect, and `tsc --init` switches off the `@types/node` the line before it installed — and both are the kind of thing the integration, not the scaffold, gets blamed for. + +Nothing beyond what the initializer writes is authored by this skill: no example code, no folder layout of its own, no opinion the framework did not ship. + +**On Python, finish with `uv sync` from inside ``.** `uv init` writes a `pyproject.toml` and stops: no lock file, no environment. `/pipelex-integrate` picks the package manager off the lock file and reads no lock file as `pip install` into the active environment, so a project handed over without one is a uv project installed into with pip — and `uv init` left no environment for pip to find either. The recipes that end in a `uv add` are locked by that command; the minimal and script forms, which are exactly what "no framework named" selects, are locked only by this line. [references/initializers.md](references/initializers.md) carries it with the rest of the post-initializer sequence. + +### Step 3: Version control and the pristine commit + +**Test whether `` is its own repository; never infer it from which initializer ran.** `git -C rev-parse --show-toplevel` must print `` itself, and when it does not, run `git init -b main` in the directory before staging anything. The list of initializers that `git init` on their own is not a substitute for that test, because membership in it is conditional: `uv init` initializes a repository when it creates a standalone project and **does not** when the parent directory already holds one, where it makes `` a workspace member of the enclosing project instead. A `` with no `.git` of its own is governed by whatever repository encloses it — the user's — and `git -C ` sets git's working directory without scoping anything, so the staging below would sweep that whole worktree: the user's unrelated untracked files, wherever they sit, committed into their repository under this skill's message. That is the one outcome this step exists to prevent, and the read-back catches it only if you read the paths and not just the count. + +**Then confirm there is a `.gitignore` covering the dependency tree and the build output**: `npm init -y` and `tsc --init` write none, so the minimal TypeScript recipe — and the Express and library recipes built on it — reach this step with a populated `node_modules/` and nothing excluding it, and the staging below would commit the whole dependency tree into the one commit that is supposed to be a readable baseline. Write `node_modules/`, `dist/` and `.env` into a `.gitignore` first where the initializer left none, and read back what is staged — `git -C diff --cached --name-only` over the paths themselves, not a `--stat | tail -1` whose count cannot tell a correct scaffold from a swept-up worktree — before committing. Then the one commit, for the same reason as branch A: + +```bash +git -C add -A -- . && git -C commit -m "Scaffold project" -- . +``` + +The `-- .` pathspec is the second half of the guard, and it is on **both** commands for a reason: `add -A -- .` bounds what this command stages, but a bare `git commit` then commits the *whole index*, so anything the user had staged elsewhere in an enclosing repository before the session goes into the commit under this skill's message. With the pathspec on the commit too, the staging and the commit are both held to `` and below even when the repository turns out to be an enclosing one, and the user's own staged work is left staged and uncommitted where they put it. + +**An initializer that commits as well as `git init`s has already made this commit.** `create-next-app` is one: it runs `git init`, stages everything and commits, so the tree is clean and the command above stops with `nothing to commit` — which is the initializer having done the job, not a failure of it. Take that commit as the pristine one, exactly as branch A takes GitHub's, and name it and its message in the report. Never force a second empty commit on top of it. + +### Step 4: The env file + +Write `.env.example` with the two lines the starters share, make sure `.env` is gitignored, and copy the example to `.env` under the same key rule as branch A: + +``` +PIPELEX_BASE_URL=https://api.pipelex.com +PIPELEX_API_KEY= +``` + +### Step 5: Hand off + +Add **no** SDK dependency and create **no** empty `methods/` directory: `/pipelex-integrate` adds `@pipelex/sdk` or `pipelex-sdk` when it writes the first call site, and creates `methods//` when it places the first bundle. A project with nothing to integrate yet has nothing Pipelex-shaped in it beyond the env convention, and that is correct. + +## The report + +Say, in this order: what was created and where; which template or initializer it came from, at which version and SHA; that this skill made exactly one commit and what it holds; what the bootstrap changed and that those changes are uncommitted for review, in the bootstrap's own words (branch A); which env file was written and whether the key was filled from the environment or left for the user; the demos the starter still carries and where the README's removal checklist is (branch A); and the hand-off. + +Two lines are easy to forget and matter: + +- **The project's own instructions and skills load in a session started inside it.** Its `CLAUDE.md` / `AGENTS.md` and its `release` and `bump-*` skills are not in the current session; `cd && claude` is how they arrive. +- **`/pipelex-integrate` still works from here meanwhile**, because the Pipelex workshop writes anywhere under the directory the harness was launched in, and the new project sits there. One exception, and it is the version-manager machine of step 1: the workshop is spawned with `npx` on the **harness's** own `PATH`, which no activation of yours reaches, so a `node` only reachable through `nvm` or `fnm` means no workshop at all. Say so there instead — the hand-off needs the harness restarted from a shell where the runtime is active. Hand the method to it with `/pipelex-integrate`; a bundle that lives elsewhere on disk is copied into the project by that skill. No method yet → `/pipelex-design` first. + +## When something goes wrong + +| Condition | Do this | +|---|---| +| A toolchain piece is missing (Node below the floor, no `uv`, no git) | STOP, name the exact missing piece and the starter README's line about it; never install a toolchain — first check a version manager the machine already has (`nvm`, `fnm`, `volta`, `asdf`, `mise`) and use its runtime, saying so | +| The target directory exists and is not empty — anything at all beyond a lone `.git` | STOP, ask for another; never delete, move or write into it, and never offer to. **A directory holding nothing but `.git` is empty here and is written into**; that exception is the one directory entry by name and not a class, so `.DS_Store`, `.idea/`, `.vscode/`, `Thumbs.db` and anything else still refuse. On branch A that directory is served by the acquisition beside it (Step 2) and never by a `git clone` into it, which git refuses outright | +| `git clone` fails (network, permissions) | report git's error verbatim; nothing to clean up beyond an empty directory, and on the acquisition beside an existing repository the chain removes its own temporary path and copies nothing | +| `gh` is absent or not authenticated | fall back to the local clone; say the GitHub repository can be created later with `gh repo create --source .` | +| The clone carries no `bootstrap` skill | follow the README's manual list; say the template changed | +| The bootstrap's checks are red | its own rule: fix the cause and re-run; never hand off on red | +| An initializer is interactive with no non-interactive form | hand the command to the user to run in the session; resume after | +| `PIPELEX_API_KEY` is not in the shell environment | leave the value empty in the env file; say where a key comes from and where it goes; never ask for it in the conversation | +| you need to know whether a key is set | test it without printing it (`[ -n "${PIPELEX_API_KEY:-}" ] && echo set`); never `env \| grep PIPELEX`, never echo the value, never move it with a file-editing tool, never read the env file back — a key in the transcript is a key to rotate | +| The working directory is the template's own checkout (its `origin` remote points at `Pipelex/pipelex-starter-…`) | STOP: this is the template, not a copy of it — acquire a copy in another directory. This is what separates it from the fresh-clone shortcut, which is a copy of the template and carries somebody else's `origin` or none | + +## Reference + +- [references/starters.md](references/starters.md) — the two starters side by side: what each brings, its prerequisite floors, the acquisition commands, its env file, its bootstrap, its demos and their removal checklist, and the codegen harness `/pipelex-integrate` will find. +- [references/initializers.md](references/initializers.md) — per language, the minimal default and the common frameworks' non-interactive initializers, whether each runs `git init`, and where the import package or `src/` root lands. +- `/pipelex-integrate` — the skill this one hands every project to. diff --git a/pipelex/skills/pipelex-scaffold/references/initializers.md b/pipelex/skills/pipelex-scaffold/references/initializers.md new file mode 100644 index 0000000..bd2e5c0 --- /dev/null +++ b/pipelex/skills/pipelex-scaffold/references/initializers.md @@ -0,0 +1,50 @@ +# Ecosystem initializers + +Branch B of `/pipelex-scaffold` runs an initializer that already exists and authors nothing of its own beyond what that initializer writes. This file lists the common ones with their non-interactive forms. Flags change between versions: when a command below prompts anyway or rejects a flag, read its `--help` and prefer its own current non-interactive form over improvising a layout by hand. An initializer with no non-interactive form is handed to the user to run in the session. + +After the initializer: `git init -b main` only if `git -C rev-parse --show-toplevel` does not print `` itself — **the test, never the inference from which initializer ran**, because the `git init?` columns below are conditional and `/pipelex-scaffold`'s Step 3 gives the case that makes them so (`uv init` initializes nothing when the parent directory already holds a project, and staging in a `` governed by an enclosing repository sweeps the user's whole worktree into this skill's commit). Then one pristine commit (`Scaffold project`), the two-line `.env.example` (`PIPELEX_BASE_URL=https://api.pipelex.com`, `PIPELEX_API_KEY=`), `.env` gitignored and copied from it. No SDK dependency — `/pipelex-integrate` adds it. + +**On Python, run `uv sync` from inside `` before the pristine commit, and commit the `uv.lock` it writes.** `uv init` writes a `pyproject.toml` and nothing else: no lock file and no environment. That matters at the hand-off rather than here, because `/pipelex-integrate` reads the lock file to decide how to install — `uv.lock` selects uv, and **nothing** selects `pip install` into whatever environment happens to be active. A project handed over without its lock is therefore a uv project that the next skill installs into with pip, or cannot install into at all, because `uv init` left no environment for pip to find either. The recipes below that end in a `uv add` get a lock from that command; the minimal and script forms, which are the defaults when no framework is named, get one only from this step. + +## Python + +| Want | Command | `git init`? | Where the import package lands | +|---|---|---|---| +| **Minimal (the default when no framework is named)** | `uv init --package --no-workspace ` | yes (`--vcs none` to skip) | `src//` with `__init__.py` and a console-script entry in `pyproject.toml` | +| A script-style app rather than a package | `uv init --app --no-workspace ` | yes | `main.py` at the root — no import package; `/pipelex-integrate` will put the generated tree under `generated/` at the root | +| FastAPI service | `uv init --package --no-workspace && (cd && uv add "fastapi[standard]")` | yes | as minimal | +| Django project | `uv init --package --no-workspace && (cd && uv add django && uv run django-admin startproject config .)` | yes (from `uv init`) | the Django project package `config/` plus `src//`; ask which one owns the Pipelex call sites | +| Typer CLI (the Python starter's shape, without the starter) | `uv init --package --no-workspace && (cd && uv add typer)` | yes | as minimal | +| An existing `pyproject.toml` layout the user prefers (poetry, pdm, hatch) | the tool the user names: `poetry new `, `pdm init --non-interactive`, `hatch new ` | poetry: no; pdm: no; hatch: no | per tool — `poetry new` and `hatch new` make `/` or `src//` | + +`uv init` refuses a directory that already holds a project; on an empty "here" directory use `uv init --package --no-workspace .` — it names the package after the directory. + +**`--no-workspace` is on every `uv init` above, and it is the same hazard the `uv add` parentheses below address, one command earlier.** Run inside a directory that already holds a `pyproject.toml` — the scaffold being made inside an existing project, which is the common case — a bare `uv init --package ` does not create a standalone project at all. It prints `Adding as member of workspace …`, appends a `[tool.uv.workspace]` table naming `` to **the user's own `pyproject.toml`**, and then the first `uv add` writes the lockfile at the *parent* root, so the new project has no `uv.lock` of its own and does not resolve standalone. Editing a file of the user's is exactly what this skill does not do, and a project that needs its parent to resolve is not the project the report says was handed over. `--no-workspace` leaves the parent untouched and gives `` its own lock. With no parent project it changes nothing, so it is safe to pass always, which is why it is not conditional. + +**Every `uv add` above runs inside ``, and the parentheses are why.** `uv add` resolves the project from its *working* directory upwards, so run from the parent it writes the dependency into whatever project it finds there — the user's own `pyproject.toml` and lockfile, when the scaffold is being made inside an existing workspace — or fails outright when it finds none. Neither is the new project. `uv add --directory ` is equivalent if you prefer a flag to a subshell. + +## TypeScript / JavaScript + +| Want | Command | `git init`? | Where `src/` lands | +|---|---|---|---| +| **Minimal (the default when no framework is named)** | `mkdir -p && cd && npm init -y && npm install --save-dev typescript @types/node && npx tsc --init --strict --module nodenext --target es2022 --rootDir src --outDir dist` then set `"type": "module"` in `package.json` — **but read the resolution note below first: prefer `--module esnext --moduleResolution bundler` unless the user actually wants Node's own resolution**, because `nodenext` is the shape that meets the emitter defect | no | `src/` (create it); `/pipelex-integrate` puts the generated tree under `src/generated/` | +| Next.js app (the JS starter's shape, without the starter) | `npm create next-app@latest -- --ts --app --src-dir --eslint --use-npm --yes` | yes (`--disable-git` to skip) | `src/app/`; generated tree under `src/generated/` | +| Vite + React | `npm create vite@latest -- --template react-ts` then `(cd && npm install)` | no | `src/` | +| Hono server | `npm create hono@latest -- --template nodejs --pm npm --install` | no | `src/` | +| Express server | the minimal recipe above, then `(cd && npm install express && npm install --save-dev @types/express)` | no | `src/` | +| Node library | the minimal recipe above | no | `src/` | +| pnpm / yarn / bun instead of npm | replace `npm create` with `pnpm create` / `yarn create` / `bun create`, and the install command accordingly; `/pipelex-integrate` reads the lockfile to pick the package manager for what it adds | — | — | + +`npm create @latest -- `: the `--` is what passes the flags to the initializer rather than to npm. The Next.js `--yes` accepts the initializer's defaults for every prompt not covered by a flag. + +**Every follow-on `npm install` above is parenthesised for the same reason every `uv add` is, and npm is the worse of the two.** `npm install` resolves the project it writes to from its *working* directory upwards, so run from the parent it adds the dependency to the user's own `package.json` and puts `node_modules/` in the user's tree — and where `uv add` at least fails outright when it finds no project nearby, npm finds the parent and silently succeeds, leaving the new project with nothing installed and no error to read. The `cd ` inside the minimal recipe's own `&&` chain does not reach a follow-on issued as a separate command, because shell state does not survive from one command to the next. Use the subshell, or `npm install --prefix `. + +**The minimal recipe's `--module nodenext` plus `"type": "module"` is exactly the shape that meets the ts-zod emitter's extensionless-import defect** (`pipelex-integrate`'s `references/typescript.md`, "Known defect"): the generated `binder.ts` fails the type check with `TS2835` and will not load at runtime. So say so when you hand a minimal TypeScript project to `/pipelex-integrate`, and when the user has no reason to prefer Node's own resolution, prefer a bundler-backed setup (Vite, Next.js) or `--module esnext --moduleResolution bundler`, which the defect does not touch. The two recipes also need `node_modules/` and `dist/` in a `.gitignore` before the pristine commit — neither `npm init -y` nor `tsc --init` writes one. + +**And `tsc --init` writes `"types": []` as an active key, which switches `@types/node` off on the line after the recipe installed it.** That empty array is the current `tsc --init` template's own default (TypeScript 7 writes it, with `// "types": ["node"],` commented out three lines below), and it means no `@types` package is loaded at all — so the call site `/pipelex-integrate` writes next fails with `TS2591` on `node:fs/promises`, on `node:path` and on `process`, and then cascading `TS7006` implicit-any on the `readdir` callback. Nothing about the errors points at the tsconfig, so set `"types": ["node"]` as part of the recipe rather than leaving it for the integration to discover. `npm install --save-dev typescript` also resolves to TypeScript 7 now; say which major the project got, because `tsc --init`'s defaults moved with it. + +## What every branch-B project shares afterwards + +- One commit, the pristine scaffold, so the user's first real change is a clean diff. +- `.env.example` and `.env` written, `.env` ignored, `PIPELEX_API_KEY` filled only from the shell environment. Both land *after* the pristine commit, which holds the initializer's output as it came, so they stay untracked for the user to review and commit — the same posture branch A leaves the bootstrap's edits in. +- Nothing else Pipelex-shaped: the SDK dependency, the `methods/` directory and the generated tree arrive with the first `/pipelex-integrate`. diff --git a/pipelex/skills/pipelex-scaffold/references/starters.md b/pipelex/skills/pipelex-scaffold/references/starters.md new file mode 100644 index 0000000..08550a3 --- /dev/null +++ b/pipelex/skills/pipelex-scaffold/references/starters.md @@ -0,0 +1,76 @@ +# The two Pipelex starters + +Both are GitHub **template repositories** under the `Pipelex` organization. Each is a real, CI-tested application against the hosted Pipelex API, not a parameterized template: the identity you see in a fresh clone (`pipelex-starter-js` / `Pipelex Starter`, or `piper` / `Piper`) is a placeholder that the starter's own `bootstrap` skill rewrites. Read the clone's `README.md` after acquiring it — the sections named below are where the details live, and they move as the starters evolve. + +## Side by side + +| | `pipelex-starter-js` | `pipelex-starter-python` | +|---|---|---| +| **Shape** | Next.js (App Router), React, TypeScript strict, Tailwind; a web app with one tab per method whose input form is rendered from the method's own contract by `@pipelex/mthds-form` | A Typer CLI with one command per method, printing JSON on stdout and a cost report on stderr; three execution modes (`blocking`, `attended`, `detached`) as separate sub-packages | +| **Pick it when** | people will use the methods in a browser: forms, uploads, live run status | the methods run from a terminal, a script, a batch job or a service, and the user wants Python | +| **SDK** | `@pipelex/sdk` | `pipelex-sdk` (import package `pipelex_sdk`) — the `pipelex` runtime is **not** a dependency | +| **Methods live in** | `methods//main.mthds`, or `methods//method.json` for a method that lives elsewhere (a catalog id or a published address) | `/methods//main.mthds` | +| **Generated types** | `src/generated//` — `types.ts`, `binder.ts`, `contracts.ts`, `codegen.lock`, `sources.json` | `/generated//` — `models.py`, `codegen.lock` | +| **Codegen harness** | `npm run codegen` (keyed, dev), `npm run codegen:check` (offline, in `make check`), `npm run codegen:verify` (keyed, semantic); `make add-method METHOD=` scaffolds a remote method end to end | `make codegen` / `make codegen-check` — **both shell out to a `pipelex` CLI the starter does not depend on** (`PIPELEX=` in the Makefile); `/pipelex-integrate` knows this and writes into the same layout when that CLI is absent | +| **Toolchain floor** | Node ≥ the `engines.node` field of `package.json` (22.12 at writing); `npm` | `uv`; a Python inside `requires-python` of `pyproject.toml` (3.11–3.14 at writing) | +| **Env file** | `.env.local`, from `.env.example`: `PIPELEX_BASE_URL`, `PIPELEX_API_KEY`, `NEXT_PUBLIC_EXECUTION_MODE` | `.env`, from `.env.example`: `PIPELEX_BASE_URL`, `PIPELEX_API_KEY` | +| **Checks the bootstrap runs** | `npm install --package-lock-only`, then `make all` (lint, format check, typecheck, unit tests, build) | `make li` (lock + sync), then `make agent-check` and `make agent-test` | +| **Agent-facing files** | `CLAUDE.md`, `AGENTS.md`; skills `bootstrap`, `release`, `bump-sdk`, `bump-mthds-form` | `CLAUDE.md`; skills `bootstrap`, `release` | +| **Docs worth reading after bootstrap** | `docs/codegen.md`, `docs/add-method.md`, `docs/input-form.md`, `docs/adopt-in-an-existing-project.md`; README → "Swap in your own pipeline" and "Remove an example" | `docs/codegen.md`, `docs/cli-architecture.md`; README → the per-command sections | +| **Demos it carries** | several demo methods, one tab each; keep them as references or strip them with the README's "Remove an example" checklist | several demo methods, one CLI command each; keep them as references or remove the command and its method directory together | + +## Acquisition + +`` below is the one the choice above settled — `pipelex-starter-js` or `pipelex-starter-python` — and each block is a single chain for that one starter, never a menu to run top to bottom. + +Local clone with fresh history (the default — it produces what GitHub's "Use this template" button produces, a copy with no history and no remote): + +```bash +git clone --depth 1 https://github.com/Pipelex/.git || exit +git -C rev-parse HEAD +rm -rf /.git && git -C init -b main +git -C add -A -- . && git -C commit -m "Start from Pipelex/ ()" -- . +``` + +The version comes from `package.json` (`"version"`) on JS and from `pyproject.toml` (`version =`) on Python, read before the commit. **The `-- .` pathspec is on the commit as well as on the staging**, for the reason `/pipelex-scaffold`'s Step 3 gives in full: `add -A -- .` bounds what is staged, but a bare `git commit` then commits the whole index, so anything the user had staged elsewhere in an enclosing repository rides along under this skill's message. + +**The `|| exit` on the clone is load-bearing and is not decoration**, for the reason `/pipelex-scaffold`'s Step 2 gives in full: the line below it deletes a `.git` directory, and a clone that never ran — a network failure, or `` already existing — leaves that `rm -rf` to find whatever `.git` is actually at that path, destroying a repository of the user's irrecoverably. The guard only holds inside one shell, so when the two lines go out as separate commands, check the clone's exit status yourself before typing the `rm -rf`, and never type it on a path you have not just created. + +Into a directory whose only entry is `.git` — the one shape the "Where" rule reads as empty and `git clone` still refuses — acquire beside it and move in, so the user's own repository stands and the end state is the same as above: + +```bash +dir=$(cd && pwd) || exit 1 +tmp=$(mktemp -d "$(dirname "$dir")/.pipelex-starter-XXXXXX") || exit 1 +git clone --depth 1 https://github.com/Pipelex/.git "$tmp" || { rm -rf "$tmp"; exit 1; } +git -C "$tmp" rev-parse HEAD +rm -rf "$tmp/.git" || { rm -rf "$tmp"; exit 1; } +[ "$(ls -A "$dir")" = ".git" ] || { rm -rf "$tmp"; exit 1; } +cp -R "$tmp"/. "$dir"/ || { rm -rf "$tmp"; exit 1; } +rm -rf "$tmp" +``` + +Every part of that chain is load-bearing, and `/pipelex-scaffold`'s Step 2 gives each in full. **The first line resolves `` to an absolute path before the parent is computed from it**, without which a destination spelled `.` — the ordinary spelling, since the user is usually standing in the directory they just `git init`-ed — puts the temporary directory inside the destination and the `ls -A` line then refuses every time. **No `rm -rf` in it addresses a path under ``**: the template's history is discarded while the clone is still at a path `mktemp` made for this command, before anything moves, so the guarded-deletion problem above does not arise here at all. **`cp -R "$tmp"/. "$dir"/` carries the entries beginning with a dot** — `.gitignore`, `.env.example`, `.github/`, `.claude/` — every one of which `mv "$tmp"/* "$dir"/` leaves behind while exiting `0`. And **the `ls -A` line admits exactly one entry**, `.git`, which the clone no longer has, so the template can only add to the directory and anything else stops the run with nothing copied and the temporary path removed. It is one chain and goes out as one command, for the reason the paragraph above gives. + +GitHub repository, on request and after confirmation (visibility asked, default private; GitHub makes the initial commit, so no pristine commit of your own): + +```bash +gh auth status +gh repo create / --template Pipelex/ --private --clone +``` + +## The bootstrap you delegate to + +Both starters carry `.claude/skills/bootstrap/SKILL.md` with a bundled script (`scripts/bootstrap.mjs` / `scripts/bootstrap.py`). Read the file in the clone and follow it; the shape is the same on both: + +1. **Preflight** — confirms the identity is still the template's (`package.json` name `pipelex-starter-js`; `pyproject.toml` `name = "piper"`), notes a dirty tree, and on JS makes sure `node_modules/` exists (`make install`). +2. **Collect** — the package name (kebab on JS, underscores on Python, everything else derives from it), a display title, a one-line description; optionally author name **and** email (never one without the other), the repository URL, and the license (MIT kept, proprietary, or another SPDX id; the copyright holder and year). Pass what the conversation already holds so it asks once for the rest. +3. **Dry run** — the script with `--dry-run` prints the plan; the user confirms. +4. **Run** — the same command without `--dry-run`; on Python the package directory is renamed with `git mv`, which is why the pristine commit must exist first. `--clean` strips the template-only prose; keep it unless the user wants the template charter kept. +5. **Verify** — the lock file is re-synced and the project's own checks run; red is fixed, not skipped. +6. **Self-removal** — `rm -rf .claude/skills/bootstrap`, unstaged like everything else; the user reviews with `git status` and `git diff` and commits when ready. + +The starter's rules are yours while you run it: never commit on the user's behalf, always dry-run first, never touch `.github/` or the `release` skill's logic. + +## What `/pipelex-integrate` finds afterwards + +A bootstrapped starter is a project that **owns a codegen harness**, and `/pipelex-integrate` defers to it: on JS it drops a bundle under `methods//` and runs `npm run codegen`, or runs `make add-method METHOD=…` for a catalog or published method, then follows `docs/codegen.md` and the existing actions for the fan-out; on Python it places the bundle under `/methods//` and runs `make codegen` when a `pipelex` CLI is available, writing into `/generated//` through the Pipelex workshop when it is not. It never writes a second generated layout beside the starter's own. diff --git a/skills/pipelex-scaffold/references/initializers.md b/skills/pipelex-scaffold/references/initializers.md new file mode 100644 index 0000000..bd2e5c0 --- /dev/null +++ b/skills/pipelex-scaffold/references/initializers.md @@ -0,0 +1,50 @@ +# Ecosystem initializers + +Branch B of `/pipelex-scaffold` runs an initializer that already exists and authors nothing of its own beyond what that initializer writes. This file lists the common ones with their non-interactive forms. Flags change between versions: when a command below prompts anyway or rejects a flag, read its `--help` and prefer its own current non-interactive form over improvising a layout by hand. An initializer with no non-interactive form is handed to the user to run in the session. + +After the initializer: `git init -b main` only if `git -C rev-parse --show-toplevel` does not print `` itself — **the test, never the inference from which initializer ran**, because the `git init?` columns below are conditional and `/pipelex-scaffold`'s Step 3 gives the case that makes them so (`uv init` initializes nothing when the parent directory already holds a project, and staging in a `` governed by an enclosing repository sweeps the user's whole worktree into this skill's commit). Then one pristine commit (`Scaffold project`), the two-line `.env.example` (`PIPELEX_BASE_URL=https://api.pipelex.com`, `PIPELEX_API_KEY=`), `.env` gitignored and copied from it. No SDK dependency — `/pipelex-integrate` adds it. + +**On Python, run `uv sync` from inside `` before the pristine commit, and commit the `uv.lock` it writes.** `uv init` writes a `pyproject.toml` and nothing else: no lock file and no environment. That matters at the hand-off rather than here, because `/pipelex-integrate` reads the lock file to decide how to install — `uv.lock` selects uv, and **nothing** selects `pip install` into whatever environment happens to be active. A project handed over without its lock is therefore a uv project that the next skill installs into with pip, or cannot install into at all, because `uv init` left no environment for pip to find either. The recipes below that end in a `uv add` get a lock from that command; the minimal and script forms, which are the defaults when no framework is named, get one only from this step. + +## Python + +| Want | Command | `git init`? | Where the import package lands | +|---|---|---|---| +| **Minimal (the default when no framework is named)** | `uv init --package --no-workspace ` | yes (`--vcs none` to skip) | `src//` with `__init__.py` and a console-script entry in `pyproject.toml` | +| A script-style app rather than a package | `uv init --app --no-workspace ` | yes | `main.py` at the root — no import package; `/pipelex-integrate` will put the generated tree under `generated/` at the root | +| FastAPI service | `uv init --package --no-workspace && (cd && uv add "fastapi[standard]")` | yes | as minimal | +| Django project | `uv init --package --no-workspace && (cd && uv add django && uv run django-admin startproject config .)` | yes (from `uv init`) | the Django project package `config/` plus `src//`; ask which one owns the Pipelex call sites | +| Typer CLI (the Python starter's shape, without the starter) | `uv init --package --no-workspace && (cd && uv add typer)` | yes | as minimal | +| An existing `pyproject.toml` layout the user prefers (poetry, pdm, hatch) | the tool the user names: `poetry new `, `pdm init --non-interactive`, `hatch new ` | poetry: no; pdm: no; hatch: no | per tool — `poetry new` and `hatch new` make `/` or `src//` | + +`uv init` refuses a directory that already holds a project; on an empty "here" directory use `uv init --package --no-workspace .` — it names the package after the directory. + +**`--no-workspace` is on every `uv init` above, and it is the same hazard the `uv add` parentheses below address, one command earlier.** Run inside a directory that already holds a `pyproject.toml` — the scaffold being made inside an existing project, which is the common case — a bare `uv init --package ` does not create a standalone project at all. It prints `Adding as member of workspace …`, appends a `[tool.uv.workspace]` table naming `` to **the user's own `pyproject.toml`**, and then the first `uv add` writes the lockfile at the *parent* root, so the new project has no `uv.lock` of its own and does not resolve standalone. Editing a file of the user's is exactly what this skill does not do, and a project that needs its parent to resolve is not the project the report says was handed over. `--no-workspace` leaves the parent untouched and gives `` its own lock. With no parent project it changes nothing, so it is safe to pass always, which is why it is not conditional. + +**Every `uv add` above runs inside ``, and the parentheses are why.** `uv add` resolves the project from its *working* directory upwards, so run from the parent it writes the dependency into whatever project it finds there — the user's own `pyproject.toml` and lockfile, when the scaffold is being made inside an existing workspace — or fails outright when it finds none. Neither is the new project. `uv add --directory ` is equivalent if you prefer a flag to a subshell. + +## TypeScript / JavaScript + +| Want | Command | `git init`? | Where `src/` lands | +|---|---|---|---| +| **Minimal (the default when no framework is named)** | `mkdir -p && cd && npm init -y && npm install --save-dev typescript @types/node && npx tsc --init --strict --module nodenext --target es2022 --rootDir src --outDir dist` then set `"type": "module"` in `package.json` — **but read the resolution note below first: prefer `--module esnext --moduleResolution bundler` unless the user actually wants Node's own resolution**, because `nodenext` is the shape that meets the emitter defect | no | `src/` (create it); `/pipelex-integrate` puts the generated tree under `src/generated/` | +| Next.js app (the JS starter's shape, without the starter) | `npm create next-app@latest -- --ts --app --src-dir --eslint --use-npm --yes` | yes (`--disable-git` to skip) | `src/app/`; generated tree under `src/generated/` | +| Vite + React | `npm create vite@latest -- --template react-ts` then `(cd && npm install)` | no | `src/` | +| Hono server | `npm create hono@latest -- --template nodejs --pm npm --install` | no | `src/` | +| Express server | the minimal recipe above, then `(cd && npm install express && npm install --save-dev @types/express)` | no | `src/` | +| Node library | the minimal recipe above | no | `src/` | +| pnpm / yarn / bun instead of npm | replace `npm create` with `pnpm create` / `yarn create` / `bun create`, and the install command accordingly; `/pipelex-integrate` reads the lockfile to pick the package manager for what it adds | — | — | + +`npm create @latest -- `: the `--` is what passes the flags to the initializer rather than to npm. The Next.js `--yes` accepts the initializer's defaults for every prompt not covered by a flag. + +**Every follow-on `npm install` above is parenthesised for the same reason every `uv add` is, and npm is the worse of the two.** `npm install` resolves the project it writes to from its *working* directory upwards, so run from the parent it adds the dependency to the user's own `package.json` and puts `node_modules/` in the user's tree — and where `uv add` at least fails outright when it finds no project nearby, npm finds the parent and silently succeeds, leaving the new project with nothing installed and no error to read. The `cd ` inside the minimal recipe's own `&&` chain does not reach a follow-on issued as a separate command, because shell state does not survive from one command to the next. Use the subshell, or `npm install --prefix `. + +**The minimal recipe's `--module nodenext` plus `"type": "module"` is exactly the shape that meets the ts-zod emitter's extensionless-import defect** (`pipelex-integrate`'s `references/typescript.md`, "Known defect"): the generated `binder.ts` fails the type check with `TS2835` and will not load at runtime. So say so when you hand a minimal TypeScript project to `/pipelex-integrate`, and when the user has no reason to prefer Node's own resolution, prefer a bundler-backed setup (Vite, Next.js) or `--module esnext --moduleResolution bundler`, which the defect does not touch. The two recipes also need `node_modules/` and `dist/` in a `.gitignore` before the pristine commit — neither `npm init -y` nor `tsc --init` writes one. + +**And `tsc --init` writes `"types": []` as an active key, which switches `@types/node` off on the line after the recipe installed it.** That empty array is the current `tsc --init` template's own default (TypeScript 7 writes it, with `// "types": ["node"],` commented out three lines below), and it means no `@types` package is loaded at all — so the call site `/pipelex-integrate` writes next fails with `TS2591` on `node:fs/promises`, on `node:path` and on `process`, and then cascading `TS7006` implicit-any on the `readdir` callback. Nothing about the errors points at the tsconfig, so set `"types": ["node"]` as part of the recipe rather than leaving it for the integration to discover. `npm install --save-dev typescript` also resolves to TypeScript 7 now; say which major the project got, because `tsc --init`'s defaults moved with it. + +## What every branch-B project shares afterwards + +- One commit, the pristine scaffold, so the user's first real change is a clean diff. +- `.env.example` and `.env` written, `.env` ignored, `PIPELEX_API_KEY` filled only from the shell environment. Both land *after* the pristine commit, which holds the initializer's output as it came, so they stay untracked for the user to review and commit — the same posture branch A leaves the bootstrap's edits in. +- Nothing else Pipelex-shaped: the SDK dependency, the `methods/` directory and the generated tree arrive with the first `/pipelex-integrate`. diff --git a/skills/pipelex-scaffold/references/starters.md b/skills/pipelex-scaffold/references/starters.md new file mode 100644 index 0000000..08550a3 --- /dev/null +++ b/skills/pipelex-scaffold/references/starters.md @@ -0,0 +1,76 @@ +# The two Pipelex starters + +Both are GitHub **template repositories** under the `Pipelex` organization. Each is a real, CI-tested application against the hosted Pipelex API, not a parameterized template: the identity you see in a fresh clone (`pipelex-starter-js` / `Pipelex Starter`, or `piper` / `Piper`) is a placeholder that the starter's own `bootstrap` skill rewrites. Read the clone's `README.md` after acquiring it — the sections named below are where the details live, and they move as the starters evolve. + +## Side by side + +| | `pipelex-starter-js` | `pipelex-starter-python` | +|---|---|---| +| **Shape** | Next.js (App Router), React, TypeScript strict, Tailwind; a web app with one tab per method whose input form is rendered from the method's own contract by `@pipelex/mthds-form` | A Typer CLI with one command per method, printing JSON on stdout and a cost report on stderr; three execution modes (`blocking`, `attended`, `detached`) as separate sub-packages | +| **Pick it when** | people will use the methods in a browser: forms, uploads, live run status | the methods run from a terminal, a script, a batch job or a service, and the user wants Python | +| **SDK** | `@pipelex/sdk` | `pipelex-sdk` (import package `pipelex_sdk`) — the `pipelex` runtime is **not** a dependency | +| **Methods live in** | `methods//main.mthds`, or `methods//method.json` for a method that lives elsewhere (a catalog id or a published address) | `/methods//main.mthds` | +| **Generated types** | `src/generated//` — `types.ts`, `binder.ts`, `contracts.ts`, `codegen.lock`, `sources.json` | `/generated//` — `models.py`, `codegen.lock` | +| **Codegen harness** | `npm run codegen` (keyed, dev), `npm run codegen:check` (offline, in `make check`), `npm run codegen:verify` (keyed, semantic); `make add-method METHOD=` scaffolds a remote method end to end | `make codegen` / `make codegen-check` — **both shell out to a `pipelex` CLI the starter does not depend on** (`PIPELEX=` in the Makefile); `/pipelex-integrate` knows this and writes into the same layout when that CLI is absent | +| **Toolchain floor** | Node ≥ the `engines.node` field of `package.json` (22.12 at writing); `npm` | `uv`; a Python inside `requires-python` of `pyproject.toml` (3.11–3.14 at writing) | +| **Env file** | `.env.local`, from `.env.example`: `PIPELEX_BASE_URL`, `PIPELEX_API_KEY`, `NEXT_PUBLIC_EXECUTION_MODE` | `.env`, from `.env.example`: `PIPELEX_BASE_URL`, `PIPELEX_API_KEY` | +| **Checks the bootstrap runs** | `npm install --package-lock-only`, then `make all` (lint, format check, typecheck, unit tests, build) | `make li` (lock + sync), then `make agent-check` and `make agent-test` | +| **Agent-facing files** | `CLAUDE.md`, `AGENTS.md`; skills `bootstrap`, `release`, `bump-sdk`, `bump-mthds-form` | `CLAUDE.md`; skills `bootstrap`, `release` | +| **Docs worth reading after bootstrap** | `docs/codegen.md`, `docs/add-method.md`, `docs/input-form.md`, `docs/adopt-in-an-existing-project.md`; README → "Swap in your own pipeline" and "Remove an example" | `docs/codegen.md`, `docs/cli-architecture.md`; README → the per-command sections | +| **Demos it carries** | several demo methods, one tab each; keep them as references or strip them with the README's "Remove an example" checklist | several demo methods, one CLI command each; keep them as references or remove the command and its method directory together | + +## Acquisition + +`` below is the one the choice above settled — `pipelex-starter-js` or `pipelex-starter-python` — and each block is a single chain for that one starter, never a menu to run top to bottom. + +Local clone with fresh history (the default — it produces what GitHub's "Use this template" button produces, a copy with no history and no remote): + +```bash +git clone --depth 1 https://github.com/Pipelex/.git || exit +git -C rev-parse HEAD +rm -rf /.git && git -C init -b main +git -C add -A -- . && git -C commit -m "Start from Pipelex/ ()" -- . +``` + +The version comes from `package.json` (`"version"`) on JS and from `pyproject.toml` (`version =`) on Python, read before the commit. **The `-- .` pathspec is on the commit as well as on the staging**, for the reason `/pipelex-scaffold`'s Step 3 gives in full: `add -A -- .` bounds what is staged, but a bare `git commit` then commits the whole index, so anything the user had staged elsewhere in an enclosing repository rides along under this skill's message. + +**The `|| exit` on the clone is load-bearing and is not decoration**, for the reason `/pipelex-scaffold`'s Step 2 gives in full: the line below it deletes a `.git` directory, and a clone that never ran — a network failure, or `` already existing — leaves that `rm -rf` to find whatever `.git` is actually at that path, destroying a repository of the user's irrecoverably. The guard only holds inside one shell, so when the two lines go out as separate commands, check the clone's exit status yourself before typing the `rm -rf`, and never type it on a path you have not just created. + +Into a directory whose only entry is `.git` — the one shape the "Where" rule reads as empty and `git clone` still refuses — acquire beside it and move in, so the user's own repository stands and the end state is the same as above: + +```bash +dir=$(cd && pwd) || exit 1 +tmp=$(mktemp -d "$(dirname "$dir")/.pipelex-starter-XXXXXX") || exit 1 +git clone --depth 1 https://github.com/Pipelex/.git "$tmp" || { rm -rf "$tmp"; exit 1; } +git -C "$tmp" rev-parse HEAD +rm -rf "$tmp/.git" || { rm -rf "$tmp"; exit 1; } +[ "$(ls -A "$dir")" = ".git" ] || { rm -rf "$tmp"; exit 1; } +cp -R "$tmp"/. "$dir"/ || { rm -rf "$tmp"; exit 1; } +rm -rf "$tmp" +``` + +Every part of that chain is load-bearing, and `/pipelex-scaffold`'s Step 2 gives each in full. **The first line resolves `` to an absolute path before the parent is computed from it**, without which a destination spelled `.` — the ordinary spelling, since the user is usually standing in the directory they just `git init`-ed — puts the temporary directory inside the destination and the `ls -A` line then refuses every time. **No `rm -rf` in it addresses a path under ``**: the template's history is discarded while the clone is still at a path `mktemp` made for this command, before anything moves, so the guarded-deletion problem above does not arise here at all. **`cp -R "$tmp"/. "$dir"/` carries the entries beginning with a dot** — `.gitignore`, `.env.example`, `.github/`, `.claude/` — every one of which `mv "$tmp"/* "$dir"/` leaves behind while exiting `0`. And **the `ls -A` line admits exactly one entry**, `.git`, which the clone no longer has, so the template can only add to the directory and anything else stops the run with nothing copied and the temporary path removed. It is one chain and goes out as one command, for the reason the paragraph above gives. + +GitHub repository, on request and after confirmation (visibility asked, default private; GitHub makes the initial commit, so no pristine commit of your own): + +```bash +gh auth status +gh repo create / --template Pipelex/ --private --clone +``` + +## The bootstrap you delegate to + +Both starters carry `.claude/skills/bootstrap/SKILL.md` with a bundled script (`scripts/bootstrap.mjs` / `scripts/bootstrap.py`). Read the file in the clone and follow it; the shape is the same on both: + +1. **Preflight** — confirms the identity is still the template's (`package.json` name `pipelex-starter-js`; `pyproject.toml` `name = "piper"`), notes a dirty tree, and on JS makes sure `node_modules/` exists (`make install`). +2. **Collect** — the package name (kebab on JS, underscores on Python, everything else derives from it), a display title, a one-line description; optionally author name **and** email (never one without the other), the repository URL, and the license (MIT kept, proprietary, or another SPDX id; the copyright holder and year). Pass what the conversation already holds so it asks once for the rest. +3. **Dry run** — the script with `--dry-run` prints the plan; the user confirms. +4. **Run** — the same command without `--dry-run`; on Python the package directory is renamed with `git mv`, which is why the pristine commit must exist first. `--clean` strips the template-only prose; keep it unless the user wants the template charter kept. +5. **Verify** — the lock file is re-synced and the project's own checks run; red is fixed, not skipped. +6. **Self-removal** — `rm -rf .claude/skills/bootstrap`, unstaged like everything else; the user reviews with `git status` and `git diff` and commits when ready. + +The starter's rules are yours while you run it: never commit on the user's behalf, always dry-run first, never touch `.github/` or the `release` skill's logic. + +## What `/pipelex-integrate` finds afterwards + +A bootstrapped starter is a project that **owns a codegen harness**, and `/pipelex-integrate` defers to it: on JS it drops a bundle under `methods//` and runs `npm run codegen`, or runs `make add-method METHOD=…` for a catalog or published method, then follows `docs/codegen.md` and the existing actions for the fan-out; on Python it places the bundle under `/methods//` and runs `make codegen` when a `pipelex` CLI is available, writing into `/generated//` through the Pipelex workshop when it is not. It never writes a second generated layout beside the starter's own. diff --git a/templates/skills/pipelex-design/SKILL.md.j2 b/templates/skills/pipelex-design/SKILL.md.j2 index 00f15e5..10a8b84 100644 --- a/templates/skills/pipelex-design/SKILL.md.j2 +++ b/templates/skills/pipelex-design/SKILL.md.j2 @@ -186,7 +186,7 @@ After the gate: 1. **Organize only when the layout needs it.** A direct result that is already coherent skips `/pipelex-organize`. A converged stepwise result normally invokes it automatically because one-definition-per-file construction history and satisfied headers need regrouping. A naturally coherent result in either mode does not take an organization round trip solely for process compliance. 2. **Project the input schema.** Call `mthds_inputs_template` with the final whole-bundle `files` submission plus `explicit: false`. Show the returned compact template, but **do not save it as `inputs.json`** — input preparation belongs exclusively to `/pipelex-inputs`. 3. **Present the flow.** Point to the interactive method graph where the host rendered the valid verdict's view; in terminal hosts, present a concise text flow of the final structure. -4. **Hand off inputs — and the code.** Suggest preparing real inputs with `/pipelex-inputs`. Then, when the workspace holds a codebase (a `package.json` or a `pyproject.toml`), say that `/pipelex-integrate` wires the method into it with generated types and a typed call site. +4. **Hand off inputs — and the code.** Suggest preparing real inputs with `/pipelex-inputs`. Then, when the workspace holds a codebase (a `package.json` or a `pyproject.toml`), say that `/pipelex-integrate` wires the method into it with generated types and a typed call site; when it holds none and the user wants an application around the method, `/pipelex-scaffold` creates one and hands it to `/pipelex-integrate`. > **NEVER write `inputs.json` manually.** If the user provides files, paths, or wants to run with real data, invoke `/pipelex-inputs` — it handles the template, path resolution, placeholder formatting, and file copying. diff --git a/templates/skills/pipelex-integrate/SKILL.md.j2 b/templates/skills/pipelex-integrate/SKILL.md.j2 index d10dae4..b6d392c 100644 --- a/templates/skills/pipelex-integrate/SKILL.md.j2 +++ b/templates/skills/pipelex-integrate/SKILL.md.j2 @@ -1,6 +1,6 @@ --- name: pipelex-integrate -description: Wire an MTHDS method into a Python or TypeScript codebase with generated, drift-proof types and one typed call site that runs it through @pipelex/sdk or pipelex-sdk. Use when the user says "use this method in my app", "call this from my code", "generate types for this method", "wire the method into my project", "add this pipeline to my service", "typed client for this method", "integrate the method", "refresh the generated types", "regenerate the types", "the types are stale", or wants application code that runs a .mthds method — from a local bundle, a catalog id (mt_…) or a published method_ref address. Also the refresh path after a bundle edit. Not for authoring or editing the method itself (/pipelex-design, /pipelex-edit), and not for a workspace with no project in it yet, where the project has to exist first. +description: Wire an MTHDS method into a Python or TypeScript codebase with generated, drift-proof types and one typed call site that runs it through @pipelex/sdk or pipelex-sdk. Use when the user says "use this method in my app", "call this from my code", "generate types for this method", "wire the method into my project", "add this pipeline to my service", "typed client for this method", "integrate the method", "refresh the generated types", "regenerate the types", "the types are stale", or wants application code that runs a .mthds method — from a local bundle, a catalog id (mt_…) or a published method_ref address. Also the refresh path after a bundle edit. Not for authoring or editing the method itself (/pipelex-design, /pipelex-edit), and not for a project that does not exist yet (/pipelex-scaffold). {% include "skills/shared/frontmatter.md.j2" %} {%- if platform == "claude" %} - mcp__plugin_pipelex_pipelex__mthds_validate @@ -58,7 +58,7 @@ Automatic by default: state the target, the destination and the generator in one - **A published address** — `method_ref: "github.com//[/]@"`. The tag is the pin: a `method_ref` **without a tag** floats and is refused for a committed integration; ask for the tag. - **A catalog id** — `method_id: "mt_…"` (resolve a name through `mthds_list_methods` when present). The catalog is unversioned: an edit to the stored method silently invalidates committed types with nothing offline to detect it. Say so in one line, recommend committing the source or publishing an address, and proceed only on the user's say-so. -**The project** is the nearest directory holding a `package.json` or a `pyproject.toml` (or `setup.py` / `requirements.txt`) above the working area. A workspace holding several (a monorepo, a full-stack repo) is a question — which app? — never a guess. **No project at all** → this is not an integration yet: say so and stop, because there is nothing to wire the method into. Creating the project is the user's own step — a starter, their framework's initializer, their own layout — and integration begins once it exists. +**The project** is the nearest directory holding a `package.json` or a `pyproject.toml` (or `setup.py` / `requirements.txt`) above the working area. A workspace holding several (a monorepo, a full-stack repo) is a question — which app? — never a guess. **No project at all** → this is not an integration yet: offer {% if platform == "claude" %}`/pipelex-scaffold`{% else %}the `pipelex-scaffold` skill (open `../pipelex-scaffold/SKILL.md`){% endif %}, which creates one and hands it back here. Then look for a **codegen harness**: a `codegen` script in `package.json` or a `codegen` Makefile target, a `sources.json` carrying a `derived` map, `docs/codegen.md` or `docs/add-method.md`, a `methods/` directory beside `src/generated/` or `/generated/`. Either of the first two decides; the rest only corroborate. If the project has one, follow [A project that owns a codegen harness](#a-project-that-owns-a-codegen-harness) from here. diff --git a/templates/skills/pipelex-scaffold/SKILL.md.j2 b/templates/skills/pipelex-scaffold/SKILL.md.j2 new file mode 100644 index 0000000..37be71f --- /dev/null +++ b/templates/skills/pipelex-scaffold/SKILL.md.j2 @@ -0,0 +1,205 @@ +--- +name: pipelex-scaffold +description: Start a new project that will call MTHDS methods through Pipelex, in TypeScript or Python — from one of the Pipelex starter templates or from the ecosystem's own initializer — and hand it to /pipelex-integrate. Use when the user says "start a new project with Pipelex", "I have a method and need an app around it", "create a Next.js app that runs my method", "set up a Pipelex project from scratch", "new Python CLI for this method", "which starter should I use", "bootstrap a Pipelex project", or wants a codebase where none exists yet. Also use when the user is standing in a freshly cloned pipelex-starter-js or pipelex-starter-python that has not been renamed yet — this skill runs the template's own bootstrap for them. Not for adding Pipelex to code that already exists: that is /pipelex-integrate. +{% include "skills/shared/frontmatter.md.j2" %} +--- + +# Scaffold a project for Pipelex methods + +Give a user who has no project yet a project that is ready for `/pipelex-integrate`. This skill has exactly two branches and carries no templates of its own: + +- **One of the Pipelex starters** when the user wants the opinionated shape: `pipelex-starter-js` for a web app whose forms are rendered from the methods' own contracts, `pipelex-starter-python` for a CLI or service that runs methods in the three execution modes. You acquire the template, commit it once as it came, then run the clone's **own** `bootstrap` skill — the rename logic lives in the starters and is never reimplemented here. +- **The ecosystem's own initializer** when the user wants their framework or a minimal project: `uv init --package`, `npm create next-app@latest`, `django-admin startproject`, whatever the framework documents. You run it; you never assemble a project by hand. + +Both branches end the same way: an env file that follows the starters' convention, one pristine commit that makes everything after it reviewable, and the hand-off — to `/pipelex-integrate` when a method exists, to `/pipelex-design` first when none does. + +**What this skill is not.** Not a template engine (no cookiecutter, no copier, no framework matrix of its own), not a bootstrap (the starters own theirs), not a runner or a dev-server launcher, not a deployer. It needs no MCP tool and no API key: git, the starters' scripts and the ecosystem's initializers are all it uses. + +## Choosing the branch + +A cheap, reliable signal decides; an inconclusive one asks one question; nothing is guessed twice. + +| Question | Signals, in order | When inconclusive | +|---|---|---| +| **Language** | the user's word; the language the method's consumer is written in; a framework the user named | ask | +| **Which branch** | a **named framework** the starters do not carry (FastAPI, Django, Express, Hono, a plain library, a Lambda) → the initializer; **"minimal"**, **"no demo code"**, **"just a project"** → the initializer; a **web app people use in a browser**, forms, an upload flow → the JS starter; a **CLI, script, batch job, worker or service** in Python → the Python starter | one question offering the matching starter first, saying what it brings (durable runs, forms or CLI modes, codegen wiring, CI, its own `release` skill) and what it costs (demos to keep as references or to strip) | +| **Where** | the directory the user named; **"here"** when the working directory is empty — **and a directory whose only entry is `.git` is empty for this rule**, because `mkdir my-app && cd my-app && git init` is an ordinary way to arrive here and a repository the user made is not work of theirs to write over: branch B runs its initializer in the directory as it stands and leaves that repository alone — it does **not** re-run `git init` there, since Step 3's test finds `` is already its own repository — while branch A acquires beside it and moves in, so the repository already there goes on standing either way (Step 2); else a kebab-case directory named after the project | ask; never write into a directory that exists and is not empty, and never offer to move, delete or merge what it holds to make room — the answer is another directory. **A lone `.git` is the only entry that does not make a directory non-empty, and that is a ruling about `.git` and nothing else** — not a class of files you may decide to overlook. Every other entry still refuses, `.DS_Store`, `.idea/`, `.vscode/` and `Thumbs.db` included: judging which of a user's files matter is the thing this rule exists to forbid, and a list that grows by guesswork is how it would come back. The rule is about **a directory you are creating a project in**, which is why the fresh-clone shortcut below is not an exception to it: there the project is already there and you are finishing it, not writing over someone's work | +| **GitHub or local** | the user asked for a GitHub repository → `gh repo create --template`, after confirmation; otherwise a local clone with fresh history | local | + +**The fresh-clone shortcut.** A starter clone already in the working directory that has not been bootstrapped — `package.json` still says `pipelex-starter-js`, or `pyproject.toml` still says `name = "piper"` — **and whose `origin` does not point at `Pipelex/pipelex-starter-…`** is branch A entered at step 4: acquisition already happened, so go straight to running the clone's bootstrap. Do not clone again. + +[references/starters.md](references/starters.md) compares the two starters and carries every command below; [references/initializers.md](references/initializers.md) carries the initializers. + +## Mode + +Automatic by default, with the plugin's usual rules: an explicit user signal wins ("just do it" → automatic; "walk me through" → interactive); a genuinely ambiguous branch is one question, asked once; a request that gave every input up front proceeds without re-asking. Two things always confirm, in every mode: **`gh repo create`**, because it creates a repository on GitHub, and whatever the clone's bootstrap skill confirms on its own account. The pristine commit does not need confirmation **on a directory this skill created** — it holds the template as it came, and no user content is at stake. **The acquisition into a directory that already held a repository is the exception**, and it is a third thing that always confirms: there the commit lands on the user's branch, on top of their history, and `add -A -- .` records whatever their worktree was already showing along with the template (Step 3). None of the three grounds above holds, so state what will be staged and what it will land on, and ask — in every mode. + +## Branch A — one of the starters + +### Step 1: Prerequisites + +Check before touching anything, and **stop** on a missing piece with the exact thing missing and the starter README's own line about it. Never install a toolchain — but a runtime the machine already has and only the `PATH` is missing is not a missing piece: when `node` or `uv` is absent while a version manager on the machine carries one (`nvm`, `fnm`, `volta`, `asdf`, `mise`), activate it for this work and say in the report which one you used and that the user's own shell may not have it. Stop only when no usable runtime can be reached that way. + +**Activating it means resolving it to an absolute path, not sourcing a shell.** Your shell state does not survive from one command to the next — each one starts again from the user's profile, which is the profile that did not have the runtime — so `. nvm.sh` or `eval "$(fnm env)"` in one call buys nothing in the next. Resolve the binary once (`ls "$NVM_DIR"/versions/node/*/bin/node`, `volta which node`, `mise which node`, `asdf which node`, `fnm exec --using= -- which node`), keep that directory, and prefix **every** later command with it — `PATH=":$PATH" …` — the clone's bootstrap and its `make all` / `make agent-check` included, because those are separate commands too. Verify the runtime answers under that prefix **before** step 2, so a machine you cannot actually reach stops while nothing has been created; discovering it at step 4 has already spent the pristine commit. + +Three things this clause does not license. **A shim is not a runtime**: `asdf` and `mise` put a `node` on the `PATH` that exists and then fails with "no version set", so the test is that `node --version` *answers*, not that the binary resolves — and that case is a stop, not a manager to activate. **The floor still applies**: a manager holding Node 18 does not satisfy the starter's `engines` floor, and "a runtime the machine already has" never means a version below it. And **`volta` and `mise` install on first use** — `volta run`, `mise x` and `mise use` will fetch a version they do not have — which is the toolchain install this step forbids: use only a version the manager already holds, and stop rather than let it download one. Note too that `nvm`, `fnm` and `volta` manage Node alone and can never supply `uv`. + +- **JavaScript**: Node at or above the floor the starter's `package.json` `engines` field names (`node --version`; 22.12 at writing — the SDK is ESM-only and the starter's e2e specs `require()` it), and `npm`. +- **Python**: `uv` on the PATH (the starter installs and locks with it) and a Python inside the starter's `requires-python` range that `uv python find` can see (3.11 to 3.14 at writing). +- **Both**: `git`. The GitHub form also needs `gh` authenticated — `gh auth status`. + +### Step 2: Acquire the template + +**Local, the default.** Clone shallow, read the template's identity, then detach from it: + +```bash +git clone --depth 1 https://github.com/Pipelex/.git || exit +git -C rev-parse HEAD # the template SHA, for the commit message +# the template version: package.json "version" (JS) or pyproject.toml version (Python) +rm -rf /.git && git -C init -b main +``` + +The `|| exit` on the clone is not decoration: the line below it deletes a `.git` directory, and if the clone never ran — a network failure, or `` already existing — that `rm -rf` finds whatever `.git` is actually at that path. On a directory the skill just created it destroys nothing; on a repository of the user's it destroys their history irrecoverably. Run the destructive line only behind a clone that succeeded, and never type it on a path you have not just created. + +The clone's `.git` is removed on purpose: it is the template's history and remote, and leaving it would make `git status` and a future `git push` belong to Pipelex's template rather than to the user's project. This is exactly what GitHub's "Use this template" button produces — a copy with no history and no remote — and it is why the starters' READMEs tell humans not to clone directly. Fresh history is how you honour that. + +**Local, into a directory that already holds a repository.** The "Where" rule reads a directory whose only entry is `.git` as empty, and `git clone` cannot serve it: git refuses any destination already holding a `.git` and stops with `destination path '' already exists and is not an empty directory`. So acquire **beside** the directory and move in. The end state is the one the default recipe reaches — the template in ``, no template history, no Pipelex remote — and the repository the user made goes on standing instead of being replaced: + +```bash +dir=$(cd && pwd) || exit 1 +tmp=$(mktemp -d "$(dirname "$dir")/.pipelex-starter-XXXXXX") || exit 1 +git clone --depth 1 https://github.com/Pipelex/.git "$tmp" || { rm -rf "$tmp"; exit 1; } +git -C "$tmp" rev-parse HEAD # the template SHA, for the commit message +# the template version: package.json "version" (JS) or pyproject.toml version (Python) +rm -rf "$tmp/.git" || { rm -rf "$tmp"; exit 1; } +[ "$(ls -A "$dir")" = ".git" ] || { rm -rf "$tmp"; exit 1; } +cp -R "$tmp"/. "$dir"/ || { rm -rf "$tmp"; exit 1; } +rm -rf "$tmp" +``` + +**The first line resolves the destination, and that is what keeps the temporary path a sibling rather than a child.** `` is very often `.` here: `mkdir my-app && cd my-app && git init` is the "Where" rule's own account of how a user arrives at a directory holding nothing but `.git`, and they then ask for the project *here*. `dirname .` is `.`, so deriving the parent from the spelling would put the temporary directory **inside** the destination, where the `ls -A` line below finds it sitting beside `.git` and refuses — every time, on exactly the case this section exists to serve. Resolving to an absolute path first also pins the destination for the rest of the chain, so no later line can be re-read against a working directory that has moved, and it is what lets every mention below be quoted: a name with a space reaches `cp` whole instead of arriving as two arguments. + +**No `rm -rf` here ever addresses a path under ``, and that is the ordering rather than a coincidence.** The template's history is discarded while the clone is still at a path `mktemp` made for this one command, so the destructive line is spent before anything moves: the only two paths any delete is pointed at are `"$tmp/.git"` and `"$tmp"`. Compare the default recipe, where the same line runs on `` itself and `|| exit` is the whole thing standing between it and a user's history. Here there is nothing for a guard to hold, which is what makes this the form you may aim at a directory holding somebody's repository. + +**`cp -R "$tmp"/. "$dir"/`, and never `mv "$tmp"/* "$dir"/`.** The glob matches no entry beginning with a dot, so the naive move leaves `.gitignore`, `.env.example`, `.github/` and `.claude/` behind, exits `0`, and the line after it deletes the temporary directory they are still sitting in — a starter arriving without its `.gitignore`, whose pristine commit then swallows `node_modules/`, reported as a success. The trailing `/.` copies the directory's *contents*, dotfiles included, with no shell globbing involved at all. Confirm it with `ls -A "$dir"` after the copy rather than trusting the form. + +**The `ls -A` line is the "Where" rule read again, against the copy.** It is not the decision — the "Where" question settled that — it is the last look before anything lands, put next to the copy so nothing can change between the two. It admits exactly one entry, `.git`, which the clone has not had since the line above: a collision is therefore impossible rather than merely unlikely, and the template can only add to the directory. Anything else — `.git` beside a file of the user's, a `.DS_Store`, a `README.md` they wrote — stops here with nothing copied, the temporary path removed and the directory as it was. A discarded shallow clone is the cheap half of that trade. Nothing of the user's is overwritten, moved or deleted to make room, here or anywhere. + +**The chain goes out as one command.** The guards hold only inside one shell — the same reason the `|| exit` above is load-bearing, stated in full in [references/starters.md](references/starters.md) — and split across separate calls this one loses its cleanup too, leaving the temporary directory beside the user's project with no line left to remove it. + +**Nothing is initialized here.** The default recipe ends `git init -b main` because it has just deleted the only repository at that path. This one ends on the user's repository, their branch and their remote, which is the whole point of taking the long way round. + +**GitHub, on request.** When the user asked for a repository on GitHub: + +```bash +gh repo create / --template Pipelex/ --private --clone +``` + +`--clone` takes no destination argument: it clones into `./` under the current working directory, so for this form `` **is** `` — either choose the repository name to match the directory the "Where" question settled, or rebind `` to `./` before step 4, because every step after this one addresses `` literally. Visibility is the user's call: ask, default `--private`. Creating a repository on GitHub is outward-facing, so **state the exact command and confirm before running it**, in every mode. GitHub writes the initial commit itself — skip step 3 and continue at step 4. If `gh` is absent or not authenticated, fall back to the local clone and say the repository can be created later with `gh repo create --source .`. + +Both forms take the template's default-branch head. Do not offer a release tag unless the user asks for one. + +### Step 3: Commit the pristine template — exactly once + +```bash +git -C add -A -- . && git -C commit -m "Start from Pipelex/ ()" -- . +``` + +This is the **one commit this skill makes**, and it is load-bearing twice over: the Python starter's bootstrap renames the package directory with `git mv`, which refuses a path git does not track, and a committed baseline is what turns the bootstrap's edits into a diff the user can read before committing them. Nothing of the user's is in it — it is the template as it came. One qualification on the acquisition into a directory that already held a repository: the commit lands on the user's branch, on top of their history rather than opening a new one, and `add -A -- .` also records whatever deletion their worktree was already showing — their own pending change and not one this skill made, so name it in the report instead of undoing it. That is the commit the Mode section sends back for confirmation, and this is what to put in front of the user: `git -C status --short` before staging says what will ride along, and it is the difference between a baseline commit and a line in their history that says "Start from Pipelex/…" over a change they made. + +### Step 4: Run the clone's own bootstrap + +Read `/.claude/skills/bootstrap/SKILL.md` and follow it as written. The project's skills are not loaded in this session — it began elsewhere — so read the file; do not look for a `/bootstrap` command. Run every command it gives from inside the project directory (`cd && …`, or `-C `), because that skill assumes it is standing in the repo root. + +Feed it what the conversation already holds — the project name, title, description, author, repository URL, license — so that it asks once, consolidated, for whatever is left, exactly as its own Step 2 says. It dry-runs, previews, runs, re-syncs the lock file, runs the project's own checks (`make all` on JS; `make agent-check` and `make agent-test` on Python), and removes itself. Its rules stand unchanged: it never commits, its edits stay uncommitted for the user's review (the Python renames are staged by `git mv`, which its skill explains), and a red check is fixed, never skipped. **Add nothing to that procedure and reimplement none of it.** If the clone carries no bootstrap skill — a future template dropped it — follow the README's "manual equivalent" list and say that the template changed. + +### Step 5: The env file + +```bash +cp -n /.env.example /.env.local # JS: Next.js reads .env.local +cp -n /.env.example /.env # Python: python-dotenv reads .env +``` + +`-n` because this is the one step that can destroy something of the user's. On the fresh-clone shortcut the directory is one they were already working in, and a plain `cp` would overwrite an `.env.local` they had filled with their own key — the skill's whole posture is that nothing of the user's is ever cleared, and an env file is the most expensive thing in the tree to lose. An existing env file is left exactly as it is; read whether it already carries a key with the file-side test below, and say in the report that you kept theirs. + +Fill `PIPELEX_API_KEY` **from the shell environment when it is set there**, and leave it empty otherwise, telling the user where a key comes from (`app.pipelex.com`) and that this file is where it goes. **Never print a key, and never ask for one in the conversation.** Test for it without printing it — `[ -n "${PIPELEX_API_KEY:-}" ] && echo set || echo unset`. + +**The value moves only through a shell that expands the variable itself, and never through you.** A redirection is safe precisely because the shell does the expanding and only the variable's *name* is transcribed. **Append only when the file does not already carry a key**, which is the other half of the `-n` above: `grep -q '^PIPELEX_API_KEY=.\+' /.env.local || printf 'PIPELEX_API_KEY=%s\n' "$PIPELEX_API_KEY" >> /.env.local`. On the fresh-clone shortcut the file is one the user may have filled themselves, and an unconditional append puts a second assignment *after* theirs — every dotenv reader resolves a repeated name to the later line, so their working key is silently replaced by whatever the shell happened to export, while the report tells them you kept theirs. A file-editing tool is the one form that cannot be made safe, whatever it is called — it takes a **literal** string, so you would have to know the value to pass it, and a tool call's parameters are the transcript. So: no file-editing tool on a line carrying the key, no `env | grep PIPELEX`, no `echo $PIPELEX_API_KEY`, no command substitution in a message, and **no reading the env file back** once written — `cat .env.local`, a `grep` over it, or opening it to check your work is the reflex after writing and the first move when a later step fails, and it puts the key in the transcript just as surely. To confirm the write landed, test the file the same way you tested the environment: `grep -q '^PIPELEX_API_KEY=.\+' /.env.local && echo filled || echo empty`. A key in the transcript is a key to rotate, and it is not yours to spend. + +A value that passed the test is copied verbatim and never inspected, so say in the report that it was taken from the environment **and not validated** — a placeholder someone exported once passes a presence test and fails the first run, and this skill never calls the API, so it cannot tell the difference. `PIPELEX_BASE_URL` stays as the example ships it. Confirm the file is gitignored before writing a key into it — both starters ignore it, but check. + +### Step 6: Verify and hand off + +The bootstrap's own checks are the verification; do not start `make dev`. Write the report (below), then hand the user's method to `/pipelex-integrate`, which recognizes the starter's codegen harness (`npm run codegen`, `make codegen`, `make add-method`) and defers to it rather than writing a second one. + +## Branch B — the ecosystem's initializer + +### Step 1: Prerequisites + +As in branch A, for the language chosen. + +### Step 2: Run the initializer — never assemble by hand + +- **A named framework** uses its documented initializer with its non-interactive flags: `npm create next-app@latest -- --ts --app --src-dir --eslint --use-npm --yes`, where the `--` is what passes the flags to the initializer instead of to npm and without it `create-next-app` prompts; `uv init --package --no-workspace ` then `uv add "fastapi[standard]"` **from inside ``**, because `uv add` writes to whatever project its working directory resolves to and from the parent that is the user's, not the new one; and so on — [references/initializers.md](references/initializers.md) carries the common ones. An initializer that only runs interactively is handed to the user to run{% if platform == "claude" %} (typing `! ` in the prompt runs it inside this session){% endif %}, and you resume when it is done. +- **No framework named** takes the language's own minimal initializer: Python → `uv init --package --no-workspace `, which gives the import package `/pipelex-integrate` wants and a console-script entry; TypeScript → `npm init -y`, then `npm install --save-dev typescript @types/node` and `npx tsc --init` with strict mode, ES modules and a `src/` root. **Read [references/initializers.md](references/initializers.md) before running either**, and not only for the flags: it is where the two costs of the TypeScript default are written down — it is the resolution that meets the emitter's extensionless-import defect, and `tsc --init` switches off the `@types/node` the line before it installed — and both are the kind of thing the integration, not the scaffold, gets blamed for. + +Nothing beyond what the initializer writes is authored by this skill: no example code, no folder layout of its own, no opinion the framework did not ship. + +**On Python, finish with `uv sync` from inside ``.** `uv init` writes a `pyproject.toml` and stops: no lock file, no environment. `/pipelex-integrate` picks the package manager off the lock file and reads no lock file as `pip install` into the active environment, so a project handed over without one is a uv project installed into with pip — and `uv init` left no environment for pip to find either. The recipes that end in a `uv add` are locked by that command; the minimal and script forms, which are exactly what "no framework named" selects, are locked only by this line. [references/initializers.md](references/initializers.md) carries it with the rest of the post-initializer sequence. + +### Step 3: Version control and the pristine commit + +**Test whether `` is its own repository; never infer it from which initializer ran.** `git -C rev-parse --show-toplevel` must print `` itself, and when it does not, run `git init -b main` in the directory before staging anything. The list of initializers that `git init` on their own is not a substitute for that test, because membership in it is conditional: `uv init` initializes a repository when it creates a standalone project and **does not** when the parent directory already holds one, where it makes `` a workspace member of the enclosing project instead. A `` with no `.git` of its own is governed by whatever repository encloses it — the user's — and `git -C ` sets git's working directory without scoping anything, so the staging below would sweep that whole worktree: the user's unrelated untracked files, wherever they sit, committed into their repository under this skill's message. That is the one outcome this step exists to prevent, and the read-back catches it only if you read the paths and not just the count. + +**Then confirm there is a `.gitignore` covering the dependency tree and the build output**: `npm init -y` and `tsc --init` write none, so the minimal TypeScript recipe — and the Express and library recipes built on it — reach this step with a populated `node_modules/` and nothing excluding it, and the staging below would commit the whole dependency tree into the one commit that is supposed to be a readable baseline. Write `node_modules/`, `dist/` and `.env` into a `.gitignore` first where the initializer left none, and read back what is staged — `git -C diff --cached --name-only` over the paths themselves, not a `--stat | tail -1` whose count cannot tell a correct scaffold from a swept-up worktree — before committing. Then the one commit, for the same reason as branch A: + +```bash +git -C add -A -- . && git -C commit -m "Scaffold project" -- . +``` + +The `-- .` pathspec is the second half of the guard, and it is on **both** commands for a reason: `add -A -- .` bounds what this command stages, but a bare `git commit` then commits the *whole index*, so anything the user had staged elsewhere in an enclosing repository before the session goes into the commit under this skill's message. With the pathspec on the commit too, the staging and the commit are both held to `` and below even when the repository turns out to be an enclosing one, and the user's own staged work is left staged and uncommitted where they put it. + +**An initializer that commits as well as `git init`s has already made this commit.** `create-next-app` is one: it runs `git init`, stages everything and commits, so the tree is clean and the command above stops with `nothing to commit` — which is the initializer having done the job, not a failure of it. Take that commit as the pristine one, exactly as branch A takes GitHub's, and name it and its message in the report. Never force a second empty commit on top of it. + +### Step 4: The env file + +Write `.env.example` with the two lines the starters share, make sure `.env` is gitignored, and copy the example to `.env` under the same key rule as branch A: + +``` +PIPELEX_BASE_URL=https://api.pipelex.com +PIPELEX_API_KEY= +``` + +### Step 5: Hand off + +Add **no** SDK dependency and create **no** empty `methods/` directory: `/pipelex-integrate` adds `@pipelex/sdk` or `pipelex-sdk` when it writes the first call site, and creates `methods//` when it places the first bundle. A project with nothing to integrate yet has nothing Pipelex-shaped in it beyond the env convention, and that is correct. + +## The report + +Say, in this order: what was created and where; which template or initializer it came from, at which version and SHA; that this skill made exactly one commit and what it holds; what the bootstrap changed and that those changes are uncommitted for review, in the bootstrap's own words (branch A); which env file was written and whether the key was filled from the environment or left for the user; the demos the starter still carries and where the README's removal checklist is (branch A); and the hand-off. + +Two lines are easy to forget and matter: + +- **The project's own instructions and skills load in a session started inside it.** Its `CLAUDE.md` / `AGENTS.md` and its `release` and `bump-*` skills are not in the current session; {% if platform == "claude" %}`cd && claude`{% else %}`cd `, then starting {{ harness_name }} there,{% endif %} is how they arrive. +- **`/pipelex-integrate` still works from here meanwhile**, because the Pipelex workshop writes anywhere under the directory the harness was launched in, and the new project sits there. One exception, and it is the version-manager machine of step 1: the workshop is spawned with `npx` on the **harness's** own `PATH`, which no activation of yours reaches, so a `node` only reachable through `nvm` or `fnm` means no workshop at all. Say so there instead — the hand-off needs the harness restarted from a shell where the runtime is active. Hand the method to it{% if platform == "claude" %} with `/pipelex-integrate`{% else %} by opening `../pipelex-integrate/SKILL.md` and following it{% endif %}; a bundle that lives elsewhere on disk is copied into the project by that skill. No method yet → `/pipelex-design` first. + +## When something goes wrong + +| Condition | Do this | +|---|---| +| A toolchain piece is missing (Node below the floor, no `uv`, no git) | STOP, name the exact missing piece and the starter README's line about it; never install a toolchain — first check a version manager the machine already has (`nvm`, `fnm`, `volta`, `asdf`, `mise`) and use its runtime, saying so | +| The target directory exists and is not empty — anything at all beyond a lone `.git` | STOP, ask for another; never delete, move or write into it, and never offer to. **A directory holding nothing but `.git` is empty here and is written into**; that exception is the one directory entry by name and not a class, so `.DS_Store`, `.idea/`, `.vscode/`, `Thumbs.db` and anything else still refuse. On branch A that directory is served by the acquisition beside it (Step 2) and never by a `git clone` into it, which git refuses outright | +| `git clone` fails (network, permissions) | report git's error verbatim; nothing to clean up beyond an empty directory, and on the acquisition beside an existing repository the chain removes its own temporary path and copies nothing | +| `gh` is absent or not authenticated | fall back to the local clone; say the GitHub repository can be created later with `gh repo create --source .` | +| The clone carries no `bootstrap` skill | follow the README's manual list; say the template changed | +| The bootstrap's checks are red | its own rule: fix the cause and re-run; never hand off on red | +| An initializer is interactive with no non-interactive form | hand the command to the user to run in the session; resume after | +| `PIPELEX_API_KEY` is not in the shell environment | leave the value empty in the env file; say where a key comes from and where it goes; never ask for it in the conversation | +| you need to know whether a key is set | test it without printing it (`[ -n "${PIPELEX_API_KEY:-}" ] && echo set`); never `env \| grep PIPELEX`, never echo the value, never move it with a file-editing tool, never read the env file back — a key in the transcript is a key to rotate | +| The working directory is the template's own checkout (its `origin` remote points at `Pipelex/pipelex-starter-…`) | STOP: this is the template, not a copy of it — acquire a copy in another directory. This is what separates it from the fresh-clone shortcut, which is a copy of the template and carries somebody else's `origin` or none | + +## Reference + +- [references/starters.md](references/starters.md) — the two starters side by side: what each brings, its prerequisite floors, the acquisition commands, its env file, its bootstrap, its demos and their removal checklist, and the codegen harness `/pipelex-integrate` will find. +- [references/initializers.md](references/initializers.md) — per language, the minimal default and the common frameworks' non-interactive initializers, whether each runs `git init`, and where the import package or `src/` root lands. +- `/pipelex-integrate` — the skill this one hands every project to. diff --git a/tests/unit/test_pipelex_integrate_skill.py b/tests/unit/test_pipelex_integrate_skill.py index dcf51a9..27ebc44 100644 --- a/tests/unit/test_pipelex_integrate_skill.py +++ b/tests/unit/test_pipelex_integrate_skill.py @@ -526,12 +526,10 @@ def test_every_platform_renders_the_skill_and_its_references(self, target_name: assert rule in body, f"{target_name}: missing rule: {rule}" if target_name == "prod": assert "mcp__plugin_pipelex_pipelex__mthds_codegen" in body + assert "offer `/pipelex-scaffold`" in body else: assert "mcp__" not in body - # No project at all is a stop, not an offer: the greenfield front door is not in the - # published plugin, so naming a skill here would send the user to something absent. - assert "this is not an integration yet: say so and stop" in body - assert "pipelex-scaffold" not in body + assert "open `../pipelex-scaffold/SKILL.md`" in body references_dir = resolve_output_dir(self.REPO_ROOT, config.source) / "skills" / "pipelex-integrate" / "references" for reference in self.REFERENCES: diff --git a/tests/unit/test_pipelex_scaffold_skill.py b/tests/unit/test_pipelex_scaffold_skill.py new file mode 100644 index 0000000..b6451b0 --- /dev/null +++ b/tests/unit/test_pipelex_scaffold_skill.py @@ -0,0 +1,771 @@ +"""Pin the pipelex-scaffold skill: two branches, no templates of its own, one commit, delegated bootstrap.""" + +from __future__ import annotations + +import os +import re +import shutil +import subprocess +from pathlib import Path + +import pytest + +from scripts.gen_skill_docs import load_target_config, render_templates, resolve_output_dir + +REPO_ROOT = Path(__file__).parents[2] +SKILL_TEMPLATE = REPO_ROOT / "templates" / "skills" / "pipelex-scaffold" / "SKILL.md.j2" +STARTERS_REFERENCE = REPO_ROOT / "skills" / "pipelex-scaffold" / "references" / "starters.md" +INITIALIZERS_REFERENCE = REPO_ROOT / "skills" / "pipelex-scaffold" / "references" / "initializers.md" + +BASH_BLOCK = re.compile(r"```bash\n(.*?)```", re.DOTALL) +# The one string both acquisition recipes point at a real remote, swapped for a +# local repository so the recipes run as shipped without touching the network. +STARTER_URL = "https://github.com/Pipelex/.git" + +NEEDS_GIT = pytest.mark.skipif(shutil.which("git") is None, reason="the acquisition recipes are git") + + +def _bash_blocks(text: str) -> list[str]: + return [match.group(1) for match in BASH_BLOCK.finditer(text)] + + +def _recipe(text: str, marker: str) -> str: + """The one shipped bash block containing `marker`, verbatim. + + Pinned to exactly one so that a recipe split in two, or a second one written + beside it, fails here instead of letting this suite execute an arbitrary half. + """ + blocks = [block for block in _bash_blocks(text) if marker in block] + assert len(blocks) == 1, f"expected exactly one bash block containing {marker!r}, found {len(blocks)}" + return blocks[0] + + +class TestPipelexScaffoldSkill: + """The skill is executable guidance, so these tests guard what a user's new + project depends on: nothing is written into a non-empty directory, the skill + makes exactly one commit, the starters' bootstrap is delegated and never + reimplemented, the key never crosses the conversation, and no MCP tool is needed. + """ + + REPO_ROOT = Path(__file__).parents[2] + SKILLS = REPO_ROOT / "templates" / "skills" + TEMPLATE = SKILLS / "pipelex-scaffold" / "SKILL.md.j2" + REFERENCES_DIR = REPO_ROOT / "skills" / "pipelex-scaffold" / "references" + REFERENCES = ("starters.md", "initializers.md") + RULES = ( + "exactly two branches and carries no templates of its own", + "no cookiecutter, no copier, no framework matrix of its own", + "never write into a directory that exists and is not empty", + "never offer to move, delete or merge what it holds to make room", + "This is the **one commit this skill makes**", + "Read `/.claude/skills/bootstrap/SKILL.md` and follow it as written", + "**Add nothing to that procedure and reimplement none of it.**", + "**Never print a key, and never ask for one in the conversation.**", + "a key in the transcript is a key to rotate", + "**state the exact command and confirm before running it**", + "do not start `make dev`", + "Add **no** SDK dependency and create **no** empty `methods/` directory", + "Nothing beyond what the initializer writes is authored by this skill", + "a runtime the machine already has and only the `PATH` is missing is not a missing piece", + # A sourced shell does not survive the next command, so the runtime is resolved to a path. + "**Activating it means resolving it to an absolute path, not sourcing a shell.**", + "**A shim is not a runtime**", + # A file-editing tool needs the literal value in its parameters, which is the transcript. + "**The value moves only through a shell that expands the variable itself, and never through you.**", + "**no reading the env file back**", + "and not validated", + # The destructive line runs only behind a clone that succeeded. + "git clone --depth 1 https://github.com/Pipelex/.git || exit", + # cp -n: an env file the user already filled is the most expensive thing in the tree to lose. + "cp -n /.env.example /.env.local", + # npm eats the flags without the separator and create-next-app then prompts. + "npm create next-app@latest -- --ts --app --src-dir --eslint --use-npm --yes", + # uv add resolves the project from its working directory, which from the parent is the user's. + "**from inside ``**", + # An initializer that commits has already made the pristine commit. + "**An initializer that commits as well as `git init`s has already made this commit.**", + # The default TS branch is prescribed in the skill; both its costs live in the reference. + "**Read [references/initializers.md](references/initializers.md) before running either**", + # npm init -y and tsc --init write no .gitignore, so the staging would commit node_modules. + "**Then confirm there is a `.gitignore` covering the dependency tree and the build output**", + # with no .git of its own is governed by the repo enclosing it — the user's. + "**Test whether `` is its own repository; never infer it from which initializer ran.**", + ) + + @property + def scaffold(self) -> str: + return self.TEMPLATE.read_text(encoding="utf-8") + + def render(self, target_name: str) -> str: + """The skill as one target's users read it, rendered from `templates/` in memory. + + A rule asserted on the template alone is a rule that may never reach a user: the platform + conditionals are resolved here, and the committed trees under `pipelex*/` are built from + exactly this call. + """ + config = load_target_config(self.REPO_ROOT / "targets", target_name) + rendered = render_templates( + self.REPO_ROOT / "templates", + self.REPO_ROOT, + config.template_vars, + include_skills=["pipelex-scaffold"], + target_name=config.name, + ) + return next(content for path, content in rendered.items() if path.match("skills/pipelex-scaffold/SKILL.md")) + + def test_the_rules_are_stated(self) -> None: + body = self.scaffold + for rule in self.RULES: + assert rule in body, f"missing rule: {rule}" + + def test_the_skill_body_carries_the_guards_the_references_state(self) -> None: + """Two guards were stated in `references/initializers.md` and pinned only there. + + The skill body is the document an agent actually executes; a reference it is told to read + is a second hop. Round 2 added `--no-workspace` to every `uv init` in the reference and + asserted it there, while the named-framework recipe *in the skill* kept the bare form — so + the reference said "on every `uv init` above" and the executable line one hop away + contradicted it. These assertions run on the template and on all three renders, because a + rule that holds only in `templates/` is a rule no user reads. + """ + for body in [self.scaffold] + [self.render(target) for target in ("prod", "codex", "mistral-vibe")]: + # A bare `uv init --package ` appends a [tool.uv.workspace] table to the USER'S + # own pyproject.toml and leaves the new project without its own lock. + assert "uv init --package --no-workspace " in body + assert "uv init --package " not in body, "a bare uv init absorbs into the parent workspace" + # The append is gated: on the fresh-clone shortcut the env file may be one the user + # filled, and a second assignment after theirs is the one dotenv resolves to. + assert "grep -q '^PIPELEX_API_KEY=.\\+' /.env.local || printf" in body + assert ">> /.env.local`.\n" not in body, "an ungated append shadows a key the user already filled" + + def test_fresh_clone_shortcut_and_template_checkout_stop(self) -> None: + body = self.scaffold + assert "**The fresh-clone shortcut.**" in body + assert "Do not clone again." in body + assert "this is the template, not a copy of it" in body + + def test_only_a_lone_git_reads_as_empty_and_no_cruft_list_joins_it(self) -> None: + """`L-260912-724b71`, ruled 2026-09-13: a directory holding nothing but `.git` is empty. + + The refusal it narrows is the right one — the agent has no business deciding which of a + user's files matter — and the exception exists because `mkdir my-app && cd my-app && git + init` is an ordinary opening move and branch B runs `git init -b main` in the directory it + is working in one step later, so without it the skill refuses a state it produces itself. + + The cruft list was deliberately declined in the same ruling: `.DS_Store`, `.idea/`, + `.vscode/` and `Thumbs.db` keep refusing until a real report names one, because a list that + grows by guesswork is how this rule drifts back into the judgement it forbids. So this test + pins the exception as an entry named `.git` rather than as a predicate over ignorable + files, and pins the four declined names as still-refusing — adding any of them to the + exception means rewording a sentence asserted here, which is the point. + """ + for body in [self.scaffold] + [self.render(target) for target in ("prod", "codex", "mistral-vibe")]: + # The exception, at both sites, each stated as one named entry and not as a category. + assert "**and a directory whose only entry is `.git` is empty for this rule**" in body + assert "**A directory holding nothing but `.git` is empty here and is written into**" in body + assert "a repository the user made is not work of theirs to write over" in body + # And the refusal everything else still meets, with the declined names spelled out. + assert ( + "**A lone `.git` is the only entry that does not make a directory non-empty, and that is a ruling about `.git` and nothing else**" + in body + ) + assert "not a class of files you may decide to overlook" in body + assert "Every other entry still refuses, `.DS_Store`, `.idea/`, `.vscode/` and `Thumbs.db` included" in body + assert "that exception is the one directory entry by name and not a class" in body + assert "`.DS_Store`, `.idea/`, `.vscode/`, `Thumbs.db` and anything else still refuse" in body + # Untouched by the ruling: a directory read as empty is never cleared, so the skill + # still never offers to make room. Narrowing what counts as occupied is not permission + # to empty what is. + assert "never offer to move, delete or merge what it holds to make room" in body + assert "never delete, move or write into it, and never offer to" in body + + def test_branch_a_acquires_into_the_directory_the_lone_git_rule_admits(self) -> None: + """`L-260913-f28d9d`, ruled 2026-09-13: branch A serves the lone-`.git` directory too. + + The earlier ruling made a directory whose only entry is `.git` read as empty, which branch + B can honour because `uv init` accepts such a directory. Branch A could not: its first + command on the chosen directory is `git clone`, and git refuses a destination already + holding a `.git`. So the skill admitted a directory it then could not populate, and the + founder's own motivating case — `mkdir my-app && cd my-app && git init` — dead-ended for + anyone who wanted the starter rather than the initializer. Scoping the allowance to branch + B was rejected: it would answer that user with the refusal the exception was written to + remove, and the ruling would mean two different things depending on which branch they + landed in. + + Branch A now acquires beside the directory and moves in. These are the claims the recipe + one file over is executed against in `TestScaffoldAcquisitionRecipes`; asserted on the + template, on all three renders and on the reference, because round 1 found a guard that + had been pinned against the reference alone and was missing from the document an agent + actually executes. + """ + for body in [self.scaffold] + [self.render(target) for target in ("prod", "codex", "mistral-vibe")]: + assert "**Local, into a directory that already holds a repository.**" in body + # Both halves of the ruling: the same end state, and the user's repository left alone. + assert "The end state is the one the default recipe reaches" in body + assert "the repository the user made goes on standing instead of being replaced" in body + # Ordering: the destructive line is spent on the temporary path before anything moves. + assert "**No `rm -rf` here ever addresses a path under ``, and that is the ordering rather than a coincidence.**" in body + assert 'the only two paths any delete is pointed at are `"$tmp/.git"` and `"$tmp"`' in body + # Dotfiles: the naive glob drops them and still exits 0. + assert '**`cp -R "$tmp"/. "$dir"/`, and never `mv "$tmp"/* "$dir"/`.**' in body + assert "The glob matches no entry beginning with a dot" in body + # The destination is resolved before its parent is computed, so `.` cannot make the + # temporary path a child of the target. Round 2 found the unresolved form refusing + # every "scaffold here", which is the ruling's own motivating case. + assert "**The first line resolves the destination, and that is what keeps the temporary path a sibling rather than a child.**" in body + assert "would put the temporary directory **inside** the destination" in body + # Collision: one admitted entry, so the template can only add. + assert '**The `ls -A` line is the "Where" rule read again, against the copy.**' in body + assert "a collision is therefore impossible rather than merely unlikely" in body + assert "Nothing of the user's is overwritten, moved or deleted to make room" in body + # The guards and the cleanup hold only inside one shell. + assert "**The chain goes out as one command.**" in body + # The user's repository is not re-initialised; the pristine commit lands on their branch. + assert "**Nothing is initialized here.**" in body + assert "the commit lands on the user's branch, on top of their history" in body + # And the failure table sends branch A down this route rather than at a clone. + assert "On branch A that directory is served by the acquisition beside it (Step 2) and never by a `git clone` into it" in body + + # The reference is the file the skill names as carrying every command, so the recipe and + # the three properties that make it safe are stated there as well as in the skill body. + reference = STARTERS_REFERENCE.read_text(encoding="utf-8") + assert "dir=$(cd && pwd) || exit 1" in reference + assert 'tmp=$(mktemp -d "$(dirname "$dir")/.pipelex-starter-XXXXXX") || exit 1' in reference + assert "**No `rm -rf` in it addresses a path under ``**" in reference + assert '**`cp -R "$tmp"/. "$dir"/` carries the entries beginning with a dot**' in reference + assert "**the `ls -A` line admits exactly one entry**" in reference + assert "**The first line resolves `` to an absolute path before the parent is computed from it**" in reference + + def test_every_acquisition_block_names_one_starter(self) -> None: + """Round 2: the reference's blocks each carried both starters' URLs on consecutive lines. + + Read as the chain the prose calls them — "It is one chain and goes out as one command" — + the second clone lands on a destination the first has just filled, fails, and its handler + deletes the successful clone before anything is copied. The skill body had always used a + single `` placeholder, so this was the body-and-reference divergence round 1 was + told to watch for, reappearing on the other side. + """ + reference = STARTERS_REFERENCE.read_text(encoding="utf-8") + assert "pipelex-starter-js.git" not in reference + assert "pipelex-starter-python.git" not in reference + assert "--template Pipelex/pipelex-starter-js" not in reference + assert "--template Pipelex/pipelex-starter-python" not in reference + # The placeholder is bound where the blocks begin, so `` is not left dangling. + assert "`` below is the one the choice above settled" in reference + assert "never a menu to run top to bottom" in reference + + def test_the_lone_git_rule_does_not_promise_a_git_init_that_must_not_run(self) -> None: + """Round 2: the "Where" row justified the exception with behaviour that cannot occur there. + + It read "branch B runs `git init -b main` in the directory it is working in one step later" + — but in the lone-`.git` case the user has already run `git init`, so Step 3's own test finds + `` is its own repository and branch B must not re-run it. The rationale invited an agent + to expect the one command the step exists to gate. + """ + for body in [self.scaffold] + [self.render(target) for target in ("prod", "codex", "mistral-vibe")]: + assert "it does **not** re-run `git init` there" in body + assert "since Step 3's test finds `` is already its own repository" in body + assert "branch B runs `git init -b main` in the directory it is working in one step later" not in body + + def test_the_pristine_commit_confirms_when_it_lands_on_the_users_repository(self) -> None: + """Round 2: the Mode section's carve-out was made false by the preserving acquisition. + + It read "the pristine commit does not need confirmation — it is on a directory this skill + just created, holding the template as it came, and no user content is at stake". On the + preserving path none of those three grounds holds: the directory is the user's, the commit + lands on their branch, and `add -A -- .` sweeps in whatever their worktree was already + showing. Step 3 was qualified when the ruling landed and the Mode section was not, which is + the half-application this suite exists to catch. + """ + for body in [self.scaffold] + [self.render(target) for target in ("prod", "codex", "mistral-vibe")]: + assert "The pristine commit does not need confirmation **on a directory this skill created**" in body + assert "**The acquisition into a directory that already held a repository is the exception**" in body + assert "None of the three grounds above holds" in body + # And Step 3 says what to put in front of the user rather than only what to report. + assert "That is the commit the Mode section sends back for confirmation" in body + assert "`git -C status --short` before staging says what will ride along" in body + + def test_branch_b_locks_the_python_project_before_the_hand_off(self) -> None: + """Round 2: `uv init` writes a `pyproject.toml` and neither a lock file nor an environment. + + `/pipelex-integrate` picks the package manager off the lock file and reads its absence as + `pip install` into the active environment, so the default Python scaffold — the minimal and + script forms, which are what "no framework named" selects — handed over a uv project for the + next skill to install into with pip, and into no environment at all. Asserted on the body and + on the reference, because either alone is the half-application. + """ + for body in [self.scaffold] + [self.render(target) for target in ("prod", "codex", "mistral-vibe")]: + assert "**On Python, finish with `uv sync` from inside ``.**" in body + assert "reads no lock file as `pip install` into the active environment" in body + reference = INITIALIZERS_REFERENCE.read_text(encoding="utf-8") + assert "**On Python, run `uv sync` from inside `` before the pristine commit, and commit the `uv.lock` it writes.**" in reference + assert "`uv init` writes a `pyproject.toml` and nothing else: no lock file and no environment." in reference + + def test_declares_no_mcp_tool(self) -> None: + """The scaffold skill is MCP-free: no allowed-tools entry, no MCP-absent STOP message.""" + body = self.scaffold + assert "mcp__" not in body + assert "plugin manifest spawns" not in body + assert "It needs no MCP tool and no API key" in body + + def test_integrate_hands_a_missing_project_to_scaffold(self) -> None: + integrate = (self.SKILLS / "pipelex-integrate" / "SKILL.md.j2").read_text(encoding="utf-8") + assert "**No project at all** → this is not an integration yet: offer" in integrate + assert "pipelex-scaffold" in integrate + + def test_references_describe_both_starters_and_the_initializers(self) -> None: + starters = (self.REFERENCES_DIR / "starters.md").read_text(encoding="utf-8") + assert "pipelex-starter-js" in starters and "pipelex-starter-python" in starters + # The `|| exit` is the guard on the `rm -rf /.git` below it: a clone that never ran + # leaves that line to delete whatever `.git` is at that path — a user's history, if + # was theirs. The SKILL.md carries it; so must the reference the skill names as the source + # of every command, or the guard exists only in the copy nobody executes from. + assert "git clone --depth 1 https://github.com/Pipelex/.git || exit" in starters + assert "The `|| exit` on the clone is load-bearing" in starters + # Round 1 put the `-- .` pathspec on the commit as well as on the staging and wrote that it + # is on "both commands for a reason"; the reference kept the old bare commit, so the rule + # held only in the copy an agent does not read the commands out of. The mirror of the + # half-application round 1 was itself convened to fix. + assert 'git -C add -A -- . && git -C commit -m "Start from Pipelex/ ()" -- .' in starters + assert "**The `-- .` pathspec is on the commit as well as on the staging**" in starters + # And the initializers reference must send the reader to the repository test rather than to + # the inference the skill body forbids by name. + initializers = (self.REFERENCES_DIR / "initializers.md").read_text(encoding="utf-8") + assert "**the test, never the inference from which initializer ran**" in initializers + assert "`git init -b main` only if `git -C rev-parse --show-toplevel` does not print `` itself" in initializers + assert "gh repo create / --template Pipelex/ --private --clone" in starters + assert "shell out to a `pipelex` CLI the starter does not depend on" in starters + assert "npm create next-app@latest -- --ts --app --src-dir --eslint --use-npm --yes" in initializers + assert "No SDK dependency" in initializers + # Every `uv add` runs inside the new project: from the parent it writes to the user's own. + assert "**Every `uv add` above runs inside ``, and the parentheses are why.**" in initializers + for recipe in ( + '(cd && uv add "fastapi[standard]")', + "(cd && uv add typer)", + "(cd && uv add django && uv run django-admin startproject config .)", + ): + assert recipe in initializers, f"uv add not scoped to the project: {recipe}" + # Every `uv init` carries --no-workspace. Without it, run inside a directory that already + # holds a pyproject.toml, uv appends a [tool.uv.workspace] table to the USER'S file and puts + # the lock at the parent root, so the new project does not resolve standalone. Same hazard + # class as the `uv add` parentheses above, one command earlier. + assert "uv init --package --no-workspace " in initializers + assert "uv init --app --no-workspace " in initializers + assert "`--no-workspace` is on every `uv init` above" in initializers + assert "| `uv init --package `" not in initializers, "a bare uv init absorbs into the parent workspace" + assert "uv init --package &&" not in initializers, "a bare uv init absorbs into the parent workspace" + # npm resolves the project it writes to upward exactly as uv does, and unlike uv it finds the + # parent and silently succeeds, so every follow-on install is scoped too. + for recipe in ("(cd && npm install)", "(cd && npm install express && npm install --save-dev @types/express)"): + assert recipe in initializers, f"npm install not scoped to the project: {recipe}" + assert "**Every follow-on `npm install` above is parenthesised" in initializers + # The minimal TS default is the very resolution the emitter defect breaks. + assert "is exactly the shape that meets the ts-zod emitter's extensionless-import defect" in initializers + # `tsc --init` writes an active `"types": []`, which switches off the @types/node the line + # before it installed — the integrate call site then fails TS2591 on node:path and process. + assert '`tsc --init` writes `"types": []` as an active key' in initializers + assert 'set `"types": ["node"]` as part of the recipe' in initializers + + def test_the_pristine_commit_cannot_reach_an_enclosing_repository(self) -> None: + """`git -C ` sets git's working directory and scopes nothing. With no `.git` of its own, + is governed by whatever repo encloses it — the user's — and a pathspec-less `add -A` + stages that whole worktree, committing the user's unrelated files under this skill's message. + `uv init` is on the list of initializers that `git init`, but only when it creates a + standalone project; inside an existing one it makes a workspace member and no repo. + """ + body = self.TEMPLATE.read_text(encoding="utf-8") + assert "git -C rev-parse --show-toplevel` must print `` itself" in body + assert "never infer it from which initializer ran" in body + # Both pristine commits carry the pathspec, so the staging cannot escape . + assert body.count("git -C add -A -- .") == 2 + # `add -A -- .` bounds only the staging; a bare `git commit` commits the whole index, + # so anything the user had staged in an enclosing repo would ride along. Both commits + # carry the pathspec, which leaves their staged work staged. + assert 'commit -m "Start from Pipelex/ ()" -- .' in body + assert 'commit -m "Scaffold project" -- .' in body + assert "git -C add -A &&" not in body + # The read-back must name paths; a --stat count cannot tell a scaffold from a swept worktree. + assert "git -C diff --cached --name-only" in body + assert "git -C diff --cached --stat | tail -1" not in body + + @pytest.mark.parametrize("target_name", ["prod", "codex", "mistral-vibe"]) + def test_every_platform_renders_the_skill_and_its_references(self, target_name: str) -> None: + config = load_target_config(self.REPO_ROOT / "targets", target_name) + rendered = render_templates( + self.REPO_ROOT / "templates", + self.REPO_ROOT, + config.template_vars, + include_skills=["pipelex-scaffold"], + target_name=config.name, + ) + body = next(content for path, content in rendered.items() if path.match("skills/pipelex-scaffold/SKILL.md")) + assert "# Scaffold a project for Pipelex methods" in body + assert "{%" not in body + assert "{{" not in body + assert "mcp__" not in body + for rule in self.RULES: + assert rule in body, f"{target_name}: missing rule: {rule}" + if target_name == "prod": + assert "`cd && claude`" in body + assert "with `/pipelex-integrate`" in body + else: + assert "`cd && claude`" not in body + assert "opening `../pipelex-integrate/SKILL.md`" in body + + references_dir = resolve_output_dir(self.REPO_ROOT, config.source) / "skills" / "pipelex-scaffold" / "references" + for reference in self.REFERENCES: + assert (references_dir / reference).is_file(), f"{target_name}: missing references/{reference}" + + @pytest.mark.parametrize("target_name", ["prod", "codex", "mistral-vibe"]) + def test_the_committed_references_match_the_source_byte_for_byte(self, target_name: str) -> None: + """The references are executable know-how, and the committed target copies are the ones a + user installs — so compare against those, not against a fresh `copytree` into a tmp dir, + which only ever asserts that `shutil` copies bytes. A stale committed copy is the whole + failure mode, and it is invisible to any assertion that rebuilds its own expected side. + """ + config = load_target_config(self.REPO_ROOT / "targets", target_name) + installed = resolve_output_dir(self.REPO_ROOT, config.source) / "skills" / "pipelex-scaffold" / "references" + for reference in self.REFERENCES: + assert (installed / reference).is_file(), f"{target_name}: missing references/{reference}" + assert (installed / reference).read_bytes() == (self.REFERENCES_DIR / reference).read_bytes(), ( + f"{target_name}: references/{reference} is stale — run `make build`" + ) + + +@NEEDS_GIT +class TestScaffoldAcquisitionRecipes: + """Branch A's two acquisition recipes, extracted from the skill and executed. + + `L-260913-f28d9d` was ruled with a condition attached: the recipe is proven by being + run, not by being read. Three reasons, each specific. It touches the one area of this + skill that deletes; this campaign's history is that recipes composed during review + rounds became the next round's defects; and `references/starters.md` already warns that + the `|| exit` guard holds only inside one shell, which is the single thing standing + between a `rm -rf` and a user's repository. + + So these tests read the bash blocks out of the skill template, swap the starter's URL + for a local repository standing in for it, and run the bytes as shipped. Nothing is + mocked and nothing is paraphrased: a recipe reworded in the skill is the recipe that + runs here. No network, so this is part of the default suite rather than opt-in. + """ + + DEFAULT_MARKER = "rm -rf /.git && git -C init -b main" + PRESERVING_MARKER = "mktemp -d" + + @staticmethod + def _commit(repository: Path, message: str) -> None: + subprocess.run( + ["git", "-C", str(repository), "-c", "user.email=t@example.com", "-c", "user.name=Test", "commit", "-q", "-m", message], + check=True, + ) + + @pytest.fixture(scope="class") + def starter(self, tmp_path_factory: pytest.TempPathFactory) -> Path: + """A local repository standing in for a starter template. + + It carries what makes the move hard rather than what makes it look real: entries + beginning with a dot at the top level and nested inside one, which is the failure + the shipped `cp -R "$tmp"/.` form exists to avoid. + """ + template = tmp_path_factory.mktemp("starter-template") + (template / "src").mkdir() + (template / ".github" / "workflows").mkdir(parents=True) + (template / ".claude" / "skills" / "bootstrap").mkdir(parents=True) + (template / "package.json").write_text('{"name": "pipelex-starter-js", "version": "0.4.2"}\n', encoding="utf-8") + (template / "README.md").write_text("# Starter\n", encoding="utf-8") + (template / "src" / "index.ts").write_text("export const x = 1\n", encoding="utf-8") + (template / ".gitignore").write_text("node_modules/\n.env.local\n", encoding="utf-8") + (template / ".env.example").write_text("PIPELEX_BASE_URL=https://api.pipelex.com\nPIPELEX_API_KEY=\n", encoding="utf-8") + (template / ".github" / "workflows" / "ci.yml").write_text("name: ci\n", encoding="utf-8") + (template / ".claude" / "skills" / "bootstrap" / "SKILL.md").write_text("# bootstrap\n", encoding="utf-8") + subprocess.run(["git", "-C", str(template), "init", "-q", "-b", "main"], check=True) + subprocess.run(["git", "-C", str(template), "add", "-A"], check=True) + self._commit(template, "the template as it came") + return template + + # Every entry the stand-in starter ships, so a dropped one is named rather than counted. + TEMPLATE_ENTRIES = frozenset({"package.json", "README.md", "src", ".gitignore", ".env.example", ".github", ".claude"}) + DOTTED_ENTRIES = frozenset({".gitignore", ".env.example", ".github", ".claude"}) + + def _run( + self, + recipe: str, + *, + starter: Path, + target: Path, + path_prefix: Path | None = None, + dir_literal: str | None = None, + cwd: Path | None = None, + ) -> subprocess.CompletedProcess[str]: + """The recipe as shipped, with only the remote and `` bound. + + `dir_literal` binds `` to a spelling other than the target's absolute path — `.`, say — + and `cwd` is the directory the shell starts in, which is what makes such a spelling mean the + target at all. + """ + script = recipe.replace(STARTER_URL, f"file://{starter}").replace("", dir_literal or str(target)) + assert "" not in script and "github.com" not in script, "a placeholder survived the binding" + environment = dict(os.environ) + if path_prefix is not None: + environment["PATH"] = f"{path_prefix}{os.pathsep}{environment['PATH']}" + return subprocess.run( + ["bash", "-c", script], + capture_output=True, + text=True, + check=False, + env=environment, + cwd=None if cwd is None else str(cwd), + ) + + @property + def default_recipe(self) -> str: + return _recipe(SKILL_TEMPLATE.read_text(encoding="utf-8"), self.DEFAULT_MARKER) + + @property + def preserving_recipe(self) -> str: + return _recipe(SKILL_TEMPLATE.read_text(encoding="utf-8"), self.PRESERVING_MARKER) + + @staticmethod + def _entries(directory: Path) -> set[str]: + return {entry.name for entry in directory.iterdir()} + + @staticmethod + def _make_repository(directory: Path, branch: str) -> None: + directory.mkdir(parents=True, exist_ok=True) + subprocess.run(["git", "-C", str(directory), "init", "-q", "-b", branch], check=True) + + @staticmethod + def _temporaries_beside(target: Path) -> list[str]: + return [entry.name for entry in target.parent.iterdir() if entry.name.startswith(".pipelex-starter-")] + + def test_the_default_recipe_populates_a_directory_that_does_not_exist(self, starter: Path, tmp_path: Path) -> None: + target = tmp_path / "my-app" + result = self._run(self.default_recipe, starter=starter, target=target) + assert result.returncode == 0, result.stderr + assert self._entries(target) == self.TEMPLATE_ENTRIES | {".git"} + # Fresh history and no remote: what GitHub's "Use this template" button produces. + assert subprocess.run(["git", "-C", str(target), "remote"], capture_output=True, text=True, check=True).stdout == "" + assert subprocess.run(["git", "-C", str(target), "log", "-1"], capture_output=True, text=True, check=False).returncode != 0 + + def test_the_default_recipe_populates_an_empty_directory(self, starter: Path, tmp_path: Path) -> None: + target = tmp_path / "my-app" + target.mkdir() + result = self._run(self.default_recipe, starter=starter, target=target) + assert result.returncode == 0, result.stderr + assert self._entries(target) == self.TEMPLATE_ENTRIES | {".git"} + + def test_the_default_recipe_cannot_serve_a_lone_git_directory(self, starter: Path, tmp_path: Path) -> None: + """The reproduction the ruling was made on, kept executable. + + This is why the preserving recipe exists, and the assertion that would go green if + someone decided one recipe was enough after all. The `|| exit` holds, so the user's + repository is untouched — the cost is a dead end, not damage. + """ + target = tmp_path / "my-app" + self._make_repository(target, "main") + result = self._run(self.default_recipe, starter=starter, target=target) + assert result.returncode != 0 + assert "already exists and is not an empty directory" in result.stderr + assert self._entries(target) == {".git"} + + def test_the_preserving_recipe_leaves_the_users_repository_standing(self, starter: Path, tmp_path: Path) -> None: + """The ruling itself: their commit, their branch, their reflog, their remote. + + An assertion that the run succeeded is not the claim being made — the claim is that + the repository the user made survived it, so every part of it is read back. + """ + target = tmp_path / "my-app" + self._make_repository(target, "trunk") + (target / "NOTES.md").write_text("my notes\n", encoding="utf-8") + subprocess.run(["git", "-C", str(target), "add", "NOTES.md"], check=True) + self._commit(target, "my own first commit") + subprocess.run(["git", "-C", str(target), "rm", "-q", "NOTES.md"], check=True) + self._commit(target, "and then I emptied the worktree") + subprocess.run(["git", "-C", str(target), "remote", "add", "origin", "https://github.com/someone/theirs.git"], check=True) + + def read(*arguments: str) -> str: + return subprocess.run(["git", "-C", str(target), *arguments], capture_output=True, text=True, check=True).stdout + + commits_before, branch_before, reflog_before = read("log", "--format=%H"), read("rev-parse", "--abbrev-ref", "HEAD"), read("reflog") + assert self._entries(target) == {".git"}, "the fixture is not the lone-.git shape the ruling is about" + + result = self._run(self.preserving_recipe, starter=starter, target=target) + assert result.returncode == 0, result.stderr + + assert read("log", "--format=%H") == commits_before, "a commit of the user's did not survive" + assert read("rev-parse", "--abbrev-ref", "HEAD") == branch_before == "trunk\n" + assert read("reflog") == reflog_before, "the reflog was rewritten" + assert "https://github.com/someone/theirs.git" in read("remote", "-v"), "the user's remote is gone" + assert "pipelex-starter" not in read("remote", "-v"), "the template's remote came with it" + # Their first commit still holds the file they put in it, so nothing was rewritten quietly. + assert "NOTES.md" in read("show", "--stat", "--format=", f"{commits_before.split()[-1]}") + assert read("fsck", "--no-progress") == "" + # And the template arrived, so this is an acquisition and not a no-op that preserved + # the repository by doing nothing at all. + assert self._entries(target) == self.TEMPLATE_ENTRIES | {".git"} + + def test_the_preserving_recipe_carries_every_entry_beginning_with_a_dot(self, starter: Path, tmp_path: Path) -> None: + """`mv "$tmp"/*` drops these and exits 0, so the failure looks exactly like success. + + Read off the destination rather than reasoned about from the glob, which is the + whole point: a starter that arrives without its `.gitignore` commits `node_modules/` + into the baseline, and nothing in the run says so. + """ + target = tmp_path / "my-app" + self._make_repository(target, "main") + result = self._run(self.preserving_recipe, starter=starter, target=target) + assert result.returncode == 0, result.stderr + assert self.DOTTED_ENTRIES <= self._entries(target) + # Nested inside a dot-directory too, not just at the top level. + assert (target / ".github" / "workflows" / "ci.yml").is_file() + assert (target / ".claude" / "skills" / "bootstrap" / "SKILL.md").is_file() + # And the recipe never reaches for the glob that would have dropped them. + assert 'mv "$tmp"/*' not in self.preserving_recipe + + def test_the_preserving_recipe_refuses_a_directory_holding_git_and_anything_else(self, starter: Path, tmp_path: Path) -> None: + """Not the ruled case. The exception is one entry named `.git`, never `.git` and friends.""" + target = tmp_path / "my-app" + self._make_repository(target, "main") + (target / "my-file.txt").write_text("mine\n", encoding="utf-8") + result = self._run(self.preserving_recipe, starter=starter, target=target) + assert result.returncode != 0 + assert self._entries(target) == {".git", "my-file.txt"} + assert (target / "my-file.txt").read_text(encoding="utf-8") == "mine\n" + assert self._temporaries_beside(target) == [] + + def test_the_preserving_recipe_refuses_a_directory_holding_only_ignorable_cruft(self, starter: Path, tmp_path: Path) -> None: + """The names the earlier ruling deliberately declined go on refusing. + + `.DS_Store`, `.idea/`, `.vscode/` and `Thumbs.db` are not an exception waiting to be + granted: a list that grows by guesswork is how this rule drifts back into the agent + judging which of a user's files matter, which is what the refusal exists to forbid. + The recipe's own `ls -A` line is that rule in executable form, so it is read here. + """ + target = tmp_path / "my-app" + target.mkdir() + (target / ".idea").mkdir() + (target / ".vscode").mkdir() + (target / ".DS_Store").write_text("", encoding="utf-8") + (target / "Thumbs.db").write_text("", encoding="utf-8") + result = self._run(self.preserving_recipe, starter=starter, target=target) + assert result.returncode != 0 + assert self._entries(target) == {".idea", ".vscode", ".DS_Store", "Thumbs.db"} + assert self._temporaries_beside(target) == [] + + def test_no_delete_in_the_preserving_recipe_addresses_a_path_under_the_target(self, starter: Path, tmp_path: Path) -> None: + """The ordering claim, recorded rather than argued. + + Every argument every `rm` is given is logged by a shim on the `PATH`, and the run is + read back: the only paths a delete may be pointed at are the temporary clone's `.git` + and the temporary clone itself. This is what makes the recipe safe to aim at a + directory holding somebody's repository, and it is the assertion that would fail if + the discard were ever reordered to after the copy. + """ + real_rm = shutil.which("rm") + assert real_rm is not None, "these tests already require a POSIX userland" + log = tmp_path / "rm-targets.log" + shim_bin = tmp_path / "shim-bin" + shim_bin.mkdir() + shim = shim_bin / "rm" + shim.write_text( + f'#!/bin/sh\nfor a in "$@"; do case "$a" in -*) ;; *) echo "$a" >> "{log}";; esac; done\nexec {real_rm} "$@"\n', + encoding="utf-8", + ) + shim.chmod(0o755) + + target = tmp_path / "my-app" + self._make_repository(target, "main") + result = self._run(self.preserving_recipe, starter=starter, target=target, path_prefix=shim_bin) + assert result.returncode == 0, result.stderr + + targets = [line for line in log.read_text(encoding="utf-8").splitlines() if line] + assert targets, "the shim recorded nothing — the recipe no longer deletes, or the shim was bypassed" + under_the_users_directory = [line for line in targets if Path(line) == target or target in Path(line).parents] + assert under_the_users_directory == [], f"a delete was pointed inside the user's directory: {under_the_users_directory}" + assert all(".pipelex-starter-" in line for line in targets), f"a delete left the temporary path: {targets}" + + def test_the_preserving_recipe_removes_its_temporary_path_on_success_and_on_refusal(self, starter: Path, tmp_path: Path) -> None: + """A temporary directory left beside the user's project is litter they did not make, + and on the refusal paths it is litter with a whole starter inside it.""" + succeeding = tmp_path / "ok" / "my-app" + self._make_repository(succeeding, "main") + assert self._run(self.preserving_recipe, starter=starter, target=succeeding).returncode == 0 + assert self._temporaries_beside(succeeding) == [] + + refusing = tmp_path / "no" / "my-app" + self._make_repository(refusing, "main") + (refusing / "theirs.txt").write_text("mine\n", encoding="utf-8") + assert self._run(self.preserving_recipe, starter=starter, target=refusing).returncode != 0 + assert self._temporaries_beside(refusing) == [] + + # A clone that cannot run at all: the failure the `|| exit` chain was written for. + unreachable = tmp_path / "gone" / "my-app" + self._make_repository(unreachable, "main") + missing_remote = self.preserving_recipe.replace(STARTER_URL, f"file://{tmp_path / 'no-such-repository'}") + assert self._run(missing_remote, starter=starter, target=unreachable).returncode != 0 + assert self._temporaries_beside(unreachable) == [] + assert self._entries(unreachable) == {".git"} + + @pytest.mark.parametrize("spelling", [".", "./"]) + def test_the_preserving_recipe_serves_the_destination_spelled_here(self, starter: Path, tmp_path: Path, spelling: str) -> None: + """The destination is usually `.`, and the recipe has to survive being told so. + + `mkdir my-app && cd my-app && git init` is the "Where" rule's own account of how a user + reaches a directory holding nothing but `.git`, and they then ask for the project *here* — + so `` binds to `.`, not to a path with a parent to speak of. Computing the parent from + that spelling gives `.` again, which puts the temporary directory inside the destination; + the `ls -A` line then finds it beside `.git` and refuses, every time, on the one case the + ruling was written to serve. Every other recipe test binds `` to an absolute path, + which is exactly why this went unnoticed until round 2. + """ + target = tmp_path / "my-app" + self._make_repository(target, "main") + (target / "NOTES.md").write_text("theirs\n", encoding="utf-8") + subprocess.run(["git", "-C", str(target), "add", "NOTES.md"], check=True) + self._commit(target, "the user's own commit") + (target / "NOTES.md").unlink() + head = subprocess.run(["git", "-C", str(target), "rev-parse", "HEAD"], capture_output=True, text=True, check=True).stdout + + result = self._run(self.preserving_recipe, starter=starter, target=target, dir_literal=spelling, cwd=target) + + assert result.returncode == 0, result.stderr + assert self.TEMPLATE_ENTRIES <= self._entries(target) + assert self.DOTTED_ENTRIES <= self._entries(target) + # The temporary path was a sibling, and it was cleaned up. + assert self._temporaries_beside(target) == [] + assert [entry for entry in self._entries(target) if entry.startswith(".pipelex-starter-")] == [] + # And the user's repository is untouched: same commit, same branch. + assert subprocess.run(["git", "-C", str(target), "rev-parse", "HEAD"], capture_output=True, text=True, check=True).stdout == head + assert ( + subprocess.run(["git", "-C", str(target), "rev-parse", "--abbrev-ref", "HEAD"], capture_output=True, text=True, check=True).stdout.strip() + == "main" + ) + + def test_the_temporary_path_is_beside_the_target_and_collision_proof(self, starter: Path, tmp_path: Path) -> None: + """Beside, so the acquisition never crosses a filesystem or a small `/tmp`; named by + `mktemp`, so two runs in the same parent cannot land on each other.""" + recipe = self.preserving_recipe + assert 'tmp=$(mktemp -d "$(dirname "$dir")/.pipelex-starter-XXXXXX")' in recipe + target = tmp_path / "my-app" + self._make_repository(target, "main") + # The name is generated, so the same recipe run twice in one parent must not collide. + assert self._run(recipe, starter=starter, target=target).returncode == 0 + second = tmp_path / "other-app" + self._make_repository(second, "main") + assert self._run(recipe, starter=starter, target=second).returncode == 0 + assert self._temporaries_beside(target) == [] + + @pytest.mark.parametrize("target_name", ["prod", "codex", "mistral-vibe"]) + def test_the_recipes_executed_here_are_the_bytes_every_target_ships(self, target_name: str) -> None: + """This suite executes the template, so the renders must carry the same block. + + The acquisition recipes carry no Jinja, which is what makes reading the template + safe — but that is a claim about the renders, so it is read off them rather than + argued. `make agent-check` proves the committed trees are fresh; this proves the + freshness is of these lines, which are the ones a user installs and runs. + """ + config = load_target_config(REPO_ROOT / "targets", target_name) + installed = (resolve_output_dir(REPO_ROOT, config.source) / "skills" / "pipelex-scaffold" / "SKILL.md").read_text(encoding="utf-8") + template = SKILL_TEMPLATE.read_text(encoding="utf-8") + for marker in (self.DEFAULT_MARKER, self.PRESERVING_MARKER): + assert _recipe(installed, marker) == _recipe(template, marker), f"{target_name}: the shipped recipe is not the one executed here" diff --git a/wip/pipelex-integrate/plan.md b/wip/pipelex-integrate/plan.md index e848fb6..a1923d0 100644 --- a/wip/pipelex-integrate/plan.md +++ b/wip/pipelex-integrate/plan.md @@ -7,14 +7,14 @@ item: L-260830-344594 **Written 2026-08-30** as the execution tracker for [`design.md`](design.md); **amended 2026-09-06** to carry the campaign's second skill, `pipelex-scaffold`, whose design is [`scaffold-design.md`](scaffold-design.md). It schedules; it does not re-argue — when this file and a design disagree, the design wins unless the disagreement is logged under "Deviations" below. Section references (`§N`) are to `design.md`; `S§N` is `scaffold-design.md`. Ledger items `L-260830-344594` (integrate) and `L-260906-8ac105` (scaffold); the phases name the follow-up items they wait on or file. -**Status: active** since 2026-08-30, when the ten decision boxes of `design.md` were ratified as written (Phase 0). The 2026-09-06 amendments to `design.md` (boxes 3, 8 and 10 reworded, box 11 added) and `scaffold-design.md`'s boxes A–I were ratified in that session (Phase 0b). Which phases are still open is what the boxes below say, and Phase 1b's work is no longer in this branch at all — the 2026-09-13 entry under "Deviations from the design" carries why and where it went. +**Status: active** since 2026-08-30, when the ten decision boxes of `design.md` were ratified as written (Phase 0). The 2026-09-06 amendments to `design.md` (boxes 3, 8 and 10 reworded, box 11 added) and `scaffold-design.md`'s boxes A–I were ratified in that session (Phase 0b). Which phases are still open is what the boxes below say. Phase 1b left `feature/Codegen` when the scaffold half was cut out of it and came back on `feature/Scaffold-skill`, which is where it ships — the two 2026-09-13 entries under "Deviations from the design" carry the cut and the resumption. ## How to work a phase - `ledger claim L-260830-344594` before touching code; renew the claim once you are on the working branch. -- The working branch is `feature/Codegen`, in the worktree `_pipelex-plugins--codegen`. It carried both skills until the scaffold half was cut out of it — see the 2026-09-13 entry under "Deviations from the design" — so `pipelex-integrate` alone ships on it. The PR targets `dev` and its body carries `Closes L-260830-344594` and nothing else: `L-260906-8ac105` stays open for the branch that lands the scaffold, and must not be closed by this merge. A merged PR is landed with `/ledger-land`. +- The campaign has two working branches, one per skill, since the cut. `feature/Codegen` carried `pipelex-integrate` and landed on `dev` as `4de1c1f` (#18), closing `L-260830-344594`. `feature/Scaffold-skill`, in the worktree `_pipelex-plugins--scaffold-skill`, carries Phase 1b and closes `L-260906-8ac105`; it took `dev` as a merge rather than a rebase, so the review passes recorded against its SHAs stay reachable. Both target `dev`, and a merged PR is landed with `/ledger-land`. - **This checkout may be shared with other sessions.** Stage the files you touched explicitly (`git add `), never `git add -A`, so a phase never sweeps another session's work into its commit. Never run a formatter over files you did not author. -- Templates are the source of truth: edit `templates/skills/…/*.j2` and `skills/pipelex-integrate/references/*`, then `make build`; never edit `pipelex*/` outputs. Before pushing: `make agent-check` and `make agent-test`. The scaffold's own template and references are not in this branch — work them on the branch named in the 2026-09-13 deviation entry. +- Templates are the source of truth: edit `templates/skills/…/*.j2`, `skills/pipelex-integrate/references/*` and `skills/pipelex-scaffold/references/*`, then `make build`; never edit `pipelex*/` outputs. Before pushing: `make agent-check` and `make agent-test`. - `mthds_codegen` shipped in `@pipelex/mcp` 0.13.0, but the validate verdict's `main_pipe` signature is still unreleased there at the 2026-09-06 pause. Development and dogfood run against the local `../pipelex-mcp` checkout through the repo skill `/pipelex-mcp-source`; **switch back to `@latest` before any commit** and let that skill confirm no dev switch leaked into `targets/defaults.toml`. - Version discipline: everything accumulates under `[Unreleased]` in `CHANGELOG.md`; the release phase cuts the heading and bumps the version through `/release`. - At each checkpoint: tick the boxes, record the SHAs and versions outcomes landed in (never live git state), reconcile deviations into the later phases, and leave this file cold-start ready. @@ -96,7 +96,7 @@ Owner: `pipelex-plugins`. Everything in this phase renders into all three target ## Phase 1b — the `pipelex-scaffold` template and its references -> **Cut from `feature/Codegen` on 2026-09-13.** Every box below was ticked against work that is no longer in this branch: it was lifted out whole and preserved on `feature/Scaffold-skill` at `41c0b7ac8f348377eae7e57d5b50bc86e7bf31ef`, which is where the files, the tests and the second review round's fixes live. Read the boxes as the record of that ref, not as a description of this tree — nothing under `templates/skills/pipelex-scaffold/` or `skills/pipelex-scaffold/` exists here, and `make build` emits no scaffold skill. The 2026-09-13 entry under "Deviations from the design" carries why. +> **Cut from `feature/Codegen` on 2026-09-13 and resumed on `feature/Scaffold-skill` the same day.** The boxes below were ticked against work that was lifted out of `feature/Codegen` whole and preserved on `feature/Scaffold-skill` at `41c0b7ac8f348377eae7e57d5b50bc86e7bf31ef`. That branch took `dev` as a merge once `#18` had landed, so the boxes describe the tree it ships and the two 2026-09-13 entries under "Deviations from the design" carry the cut and what the resumption changed. Owner: `pipelex-plugins`. **Gate:** `scaffold-design.md`'s boxes ratified (Phase 0b). MCP-free: no `allowed-tools` MCP entries, no MCP-absent message, and the skill stays out of `MCP_SKILLS`; the template to model is `templates/skills/pipelex-synthetic-inputs/SKILL.md.j2` (MCP-free, references-bearing, a stop posture on a missing toolchain). @@ -109,6 +109,8 @@ Owner: `pipelex-plugins`. **Gate:** `scaffold-design.md`'s boxes ratified (Phase - [x] The report (S§5) with the session note as its own line. - [x] Mode (S§6) and a failure table condensed from S§7, including the "this is the template's own checkout" stop. - [x] `## Reference`: links to `references/starters.md` and `references/initializers.md`. +- [x] Branch A's acquisition into a directory whose only entry is `.git`, as `L-260913-f28d9d` ruled it on 2026-09-13: a `mktemp` path beside the target, the template's history discarded there before anything moves, the directory re-read immediately before the copy, `cp -R "$tmp"/. /` so the dotfiles come too, the temporary path removed on every exit, and no `git init` because the repository is the user's. Stated in the skill, in `references/starters.md` and in the failure table; executed against every directory shape by `TestScaffoldAcquisitionRecipes`. +- [x] The non-empty refusal as `L-260912-724b71` ruled it on 2026-09-13: a lone `.git` reads as empty, at both sites (the "Where" row and the failure-table stop row), written as that one directory entry by name — the cruft list was declined, so `.DS_Store`, `.idea/`, `.vscode/` and `Thumbs.db` still refuse and the test pins them refusing. Never *offering* to clear is untouched. **The references — `skills/pipelex-scaffold/references/`** @@ -309,6 +311,12 @@ Nothing met the bar, so nothing was fixed, and four findings were deferred with Coverage was short in the same place as round 1 and will stay short: Codex refused within seconds on "You've hit your usage limit … try again at Sep 19th, 2026 10:23 AM", so this branch has had no Codex pass at any round and cannot get one before the quota resets. cubic ran and returned an empty issue list, and said itself that a clean pass over a diff that is overwhelmingly prose is weak evidence. The official `code-review` lens produced the round's only findings, and its own fork reviewed the wrong repository again despite a brief carrying the worktree's absolute path — noted with the evidence on `L-260912-b16eb4`, the canonical item, of which round 1's `L-260913-b0caa5` is a duplicate. +**2026-09-13 — the scaffold resumed on its own branch, and the founder's ruling on the non-empty refusal went in.** `feature/Scaffold-skill` took `dev` as a **merge** and never a rebase, because a rebase rewrites every commit and the merge gate checks that a recorded review pass's SHA is still an ancestor — twice on this campaign that has been the difference between a landable branch and one whose review record evaporated. Every conflict sat in a file the removal commit `e449b91` had narrowed to what shipped without the scaffold (`CHANGELOG.md`, `CLAUDE.md`, `README.md`, `docs/build-targets.md`, `docs/decisions.md`), plus the files `#18` and `#20` advanced after the cut (the integrate template and its three rendered trees, its test, this tracker and `design.md`); the resolution takes `dev`'s later text as the base everywhere and restores the scaffold halves into it, `pipelex-integrate`'s two forward references included — the "no project at all" branch offers `/pipelex-scaffold` again rather than stopping, since the skill it names now ships. + +`L-260912-724b71` was ruled on 2026-09-13 and is implemented: **a directory holding nothing but `.git` reads as empty and is written into; everything else goes on refusing.** The cruft list was declined in the same ruling, so the exception is written as one directory entry named `.git` and never as a predicate over ignorable files — `.DS_Store`, `.idea/`, `.vscode/` and `Thumbs.db` still refuse, and the test pins them as refusing so that admitting one means rewording an asserted sentence. The rule that the skill never *offers* to clear anything is untouched: narrowing what counts as occupied is not permission to empty what is. The sites are the template's "Where" row and its failure-table stop row, with `scaffold-design.md` §2 and §7 amended and `docs/decisions.md` carrying the ruling. + +**What this branch deliberately did not do**, because the cut was for a decomposition problem and not a review problem: no second broad dogfood campaign. The earlier one produced fixes that became the next round's defect surface — round 1 returned thirty-four findings with fourteen inside the implementer's own fixes, and round 2 made the ratio worse. The only execution here was a targeted mutation battery on the new ruling test, restored and proven by sha256 rather than by `git status`. SC-9 and SC-10 stay unattempted and stay open for the reasons already recorded: SC-9 needs the founder's say-so to create a throwaway GitHub repository, and SC-10's empty-key branch is unreachable on a machine whose shell profile exports a key into every tool shell. + ## Where everything is - Brief: `wip/pipelex-integrate/brief.md`. Designs: `wip/pipelex-integrate/design.md` (integrate) and `wip/pipelex-integrate/scaffold-design.md` (scaffold). Upstream reading companion: `upstream-dependencies.md`. This tracker: `wip/pipelex-integrate/plan.md`. diff --git a/wip/pipelex-integrate/scaffold-design.md b/wip/pipelex-integrate/scaffold-design.md index f551398..3bb3c3f 100644 --- a/wip/pipelex-integrate/scaffold-design.md +++ b/wip/pipelex-integrate/scaffold-design.md @@ -26,7 +26,7 @@ The rule from `pipelex-integrate` applies: a cheap, reliable signal decides; an | --- | --- | --- | | **Language** | the user's word; the language the method's consumer is written in; a framework the user named | ask | | **Which branch** | a **named framework** the starters do not carry (FastAPI, Django, Express, Hono, Remix, a plain library, a Lambda) → the initializer; **"minimal"**, **"no demo code"**, **"just a project"** → the initializer; a **web app people use in a browser**, forms, an upload flow → the JS starter; a **CLI, script, batch job, worker or service** in Python → the Python starter | one question offering the matching starter first, saying what it brings (durable runs, forms or CLI modes, codegen wiring, CI, its own `release` skill) and what it costs (demos to keep as references or strip) | -| **Where** | the directory the user named; **"here"** when the working directory is empty; else a kebab-case directory named after the project | ask; never write into a directory that exists and is not empty | +| **Where** | the directory the user named; **"here"** when the working directory is empty; else a kebab-case directory named after the project | ask; never write into a directory that exists and is not empty — **except one whose only entry is `.git`, which reads as empty** (amended 2026-09-13, `L-260912-724b71`; see §7) | | **GitHub or local** | the user asked for a GitHub repository → `gh repo create --template`, after confirmation; otherwise a local clone with fresh history | local | A starter clone that is already in the working directory and has not been bootstrapped — `package.json` still says `pipelex-starter-js`, or `pyproject.toml` still says `piper` — is **branch A entered at step 3**: acquisition already happened, and the skill goes straight to running the clone's bootstrap. @@ -36,6 +36,7 @@ A starter clone that is already in the working directory and has not been bootst 1. **Prerequisites.** JavaScript: Node at or above the floor the starter's `package.json` `engines` names (22.12 at writing — the SDK is ESM-only and the starter's e2e specs `require()` it), and npm. Python: `uv` (the starter's Makefile installs and locks with it) and a Python inside the starter's `requires-python` range (3.11 to 3.14 at writing) that `uv python find` can see. Both: git. The GitHub branch also needs `gh` authenticated (`gh auth status`). A missing piece **stops** the skill with the exact thing missing and the starter README's own line about it; the skill never installs a toolchain. **Amended 2026-09-12 (Phase 3):** a runtime the machine already has and only the `PATH` is missing is not a missing piece — dogfooded on a `PATH` without `node`, the skill found the machine's `nvm`, activated it and carried on, which installs nothing and is the useful answer. It now checks `nvm`, `fnm`, `volta`, `asdf` and `mise`, uses what they already hold, says which one it used and that the user's own shell may not have it, and stops only when no runtime can be reached that way. 2. **Acquire.** - **Local, the default.** `git clone --depth 1 https://github.com/Pipelex/.git `; read the template's version from its `package.json` / `pyproject.toml` and its head SHA; then detach from the template — remove the clone's `.git`, `git init -b main` — so that `git status`, `git remote` and a future push belong to the user's project and not to the template. This is what GitHub's "Use this template" button produces: a copy with no history and no remote. The starters' READMEs say "don't clone it directly" to humans for exactly that reason, and the fresh history is how the skill honours it. + - **Local, into a directory that already holds a repository** (added 2026-09-13 on Louis's ruling of `L-260913-f28d9d`; see §11). `git clone` refuses a destination already holding a `.git`, so the one directory shape §2's "Where" row reads as empty is the one this default cannot serve. The skill clones into a `mktemp` path beside the directory, discards the template's `.git` there — before anything moves, so no deletion ever addresses a path under the user's directory — re-reads the directory (`ls -A` must be exactly `.git`, which makes a collision impossible and refuses anything else with nothing copied), copies the contents in with `cp -R "$tmp"/. /` so that the entries beginning with a dot come too, and removes the temporary path on every exit. Nothing is initialised: the repository is the user's and it stands, and the pristine commit lands on their branch. - **GitHub, on request.** `gh repo create / --template Pipelex/ --private --clone` (visibility is the user's call, asked, default private). Creating a repository on GitHub is an outward-facing action: the skill states the exact command and confirms before running it. GitHub writes the initial commit itself; the skill continues at step 3. - Both take the template's **default-branch head**, and the pristine commit below records the version and SHA it came from. Pinning a release tag is not offered unless the user asks; the starters cut releases, and a user who wants one names it. 3. **Commit the pristine template — exactly once.** `git add -A && git commit -m "Start from Pipelex/ ()"` from inside the directory. This is the one commit the skill makes, and it is load-bearing twice over: the Python starter's bootstrap renames the package directory with `git mv`, which refuses a path git does not track, and a committed baseline is what turns the bootstrap's edits into a diff the user can read before committing them. Nothing of the user's is in this commit — it is the template as it came. @@ -68,7 +69,7 @@ Automatic by default, with the plugin's usual rules: an explicit user signal win | Condition | The skill | | --- | --- | | A toolchain piece is missing (Node below the floor, no `uv`, no git) | STOP, name the exact missing piece and the starter README's line about it; never install a toolchain | -| The target directory exists and is not empty | STOP, ask for another; never delete, move or write into it, **and never offer to make room** — amended 2026-09-12 (Phase 3), after a run refused the directory and then offered to move the user's file aside and merge it back afterwards | +| The target directory exists and is not empty | STOP, ask for another; never delete, move or write into it, **and never offer to make room** — amended 2026-09-12 (Phase 3), after a run refused the directory and then offered to move the user's file aside and merge it back afterwards. **A directory holding nothing but `.git` reads as empty and is written into** — amended 2026-09-13 on Louis's ruling of `L-260912-724b71`; the exception is that one entry by name and not a class, so `.DS_Store`, `.idea/`, `.vscode/` and `Thumbs.db` go on refusing | | `git clone` fails (network, permissions) | report git's error verbatim; nothing to clean up beyond an empty directory | | `gh` is absent or not authenticated | fall back to the local clone and say the GitHub repository can be created later with `gh repo create --source .` | | The clone carries no `bootstrap` skill | follow the README's manual list, say the template changed | @@ -92,6 +93,10 @@ Family wiring, each one sentence: `pipelex-integrate`'s step 1 offers `/pipelex- | `L-260906-aa5083` | `pipelex-starter-python` | informational | the Python starter lacks `AGENTS.md` and `add-method`; nothing in this skill waits on it | | *to file at release* | both starters | docs | the READMEs' "Use this template" sections should name `/pipelex-scaffold` as the agent front door beside the button and `/bootstrap` — filed when the skill ships, so the pointer never precedes the thing it points at | +**Amended 2026-09-13 — a directory holding nothing but `.git` reads as empty.** The non-empty refusal (§2's "Where" row, §7's second row) treated any existing entry as occupancy. Louis ruled `L-260912-724b71` on 2026-09-13: a lone `.git` is not occupancy and the directory is written into. Three things carried it. `mkdir my-app && cd my-app && git init` is an ordinary way for a user to arrive at this skill; branch B itself runs `git init -b main` in the directory it is working in (§4 step 3), so the skill was refusing a state it produces one step later; and `references/initializers.md` already documents `uv init --package --no-workspace .` for exactly the empty-"here" case. **The cruft list was deliberately declined in the same ruling** — `.DS_Store`, `.idea/`, `.vscode/` and `Thumbs.db` keep refusing until a real report names one — because a list that grows by guesswork is how this rule drifts back into the agent judging which of a user's files matter, which is what the original refusal was right to forbid. So the exception is written as one directory entry named `.git` and never as a predicate over ignorable files. **What the ruling does not touch:** a directory read as empty is never *cleared*, so §7's "never offer to make room" stands unchanged — narrowing what counts as occupied is not permission to empty what is. + +**Amended 2026-09-13 — branch A acquires beside such a directory.** The amendment above is honoured by branch B, where `uv init` accepts a directory holding a `.git`, and cannot be honoured by branch A, whose first command on the chosen directory is `git clone` — which refuses any destination already holding a `.git`. The skill therefore accepted a directory it could not populate, and the motivating case the ruling itself cites dead-ended for a user who wanted the starter. Louis ruled `L-260913-f28d9d` on 2026-09-13: branch A clones to a temporary path beside the target, moves the template's files in, discards the temporary clone's own `.git`, and lets the user's repository stand (§3 step 2). **Scoping the allowance to branch B was rejected** — the exception exists because `mkdir my-app && cd my-app && git init` is an ordinary way to arrive, which is not specific to initializers, so that user has the same claim on the opinionated starter and would have met the refusal the exception was written to remove. This is the only option under which the ruling means one thing on both branches. **The end state is the one branch A already reaches by its other path** — the template in place, no template history, no Pipelex remote — so it is a different route to the same place, except that a repository the user already had survives instead of being replaced. The ruling carried a condition: the recipe is proven by being executed, not by being read, because it touches the one area of this skill that deletes and this campaign's history is that recipes composed inside review rounds became the next round's defects. It is executed against every directory shape by `TestScaffoldAcquisitionRecipes`. + ## Decision boxes for ratification | Box | Ruling | Ratified? | diff --git a/wip/pipelex-integrate/scaffold-review-deferrals.md b/wip/pipelex-integrate/scaffold-review-deferrals.md new file mode 100644 index 0000000..6785def --- /dev/null +++ b/wip/pipelex-integrate/scaffold-review-deferrals.md @@ -0,0 +1,34 @@ +--- +status: active +item: L-260906-8ac105 +--- + +# `pipelex-scaffold` — findings the review rounds confirmed and did not fix + +Round 1 of `/rev` on `pipelex-plugins#21` (branch `feature/Scaffold-skill`, 2026-09-13) ran cubic, the Codex review and adversarial passes, and the official `code-review`. What it fixed is in the pull request and the changelog. This file is the trace for what it confirmed and deliberately left, so none of it is a finding that merely evaporated. + +Everything below was read and verified in the tree — none of it rests on a reviewer's word alone. Each entry says why it was not fixed, which is always either "real but not important" or "the fix is a decision, not an edit". + +## Carried elsewhere + +- **A directory holding only `.git` is one branch A cannot clone into** — the sharpest finding of the round, raised by three reviewers. It needed a ruling because every fix changes behaviour the founder ruled on, so it became its own decision item, `L-260913-f28d9d`. Ruled on 2026-09-13 — branch A acquires beside such a directory and leaves the user's repository standing — and implemented on this branch, so it is no longer carried. +- **A branch-B project writes `.env` that nothing loads into the process** — rediscovered by the Codex review, already open as `L-260912-059765`. No new trace needed. + +## Deferred here + +- **The report says "this skill made exactly one commit" on paths where it made none.** The GitHub form (`gh repo create --template`) and a self-committing initializer (`create-next-app`) both produce the pristine commit themselves, and the skill correctly says to adopt it rather than force a second one. The unconditional report line at the template's "The report" section then misstates provenance. Real, and cosmetic in effect: the same step already tells the agent to name the commit and its message, so the user sees the truth beside the wrong sentence. Worth one clause next time this file is opened. +- **Step 1 checks a version floor only step 2 can read.** Branch A's prerequisites check Node against "the floor the starter's `package.json` `engines` field names", but step 1 runs before the clone exists, so the authoritative value is unreadable and the check necessarily runs against the hardcoded `22.12` the text itself hedges as "at writing". Harmless while the floor is stable; it becomes wrong silently when a starter raises it. +- **"It is the plugin's third MCP-free skill" is a hardcoded count**, in `CHANGELOG.md` and `docs/decisions.md`. The workspace guide forbids counts in docs because they go stale silently. It is accurate today and the phrasing has precedent already on `dev`, so changing it here would be a lone deviation; it wants doing across the repo at once or not at all. +- **Branch B's ownership sentence is broader than the steps beneath it.** "Nothing beyond what the initializer writes is authored by this skill" is glossed immediately with "no example code, no folder layout of its own, no opinion the framework did not ship", which scopes it to project shape — but steps 3 and 4 then author a `.gitignore` and the env pair. An agent following the explicit imperatives is not actually misled, which is why this is a wording imprecision rather than a defect. +- **The minimal TypeScript recipe still defaults to `--module nodenext`.** Round 1 annotated the default row so the caveat is visible where the command is chosen, rather than twelve lines below it, but it did not change the command. Making the bundler form the default is a recipe change that should be executed before it ships — this campaign's own history is that recipes composed inside a review round became the next round's defects. +- **`.env.example` ships an empty `PIPELEX_API_KEY=`, so the gated append can still leave two assignments.** Round 1 closed the defect that mattered — an append landing *after* a key the user had already filled, which every dotenv reader resolves to the later line. What remains is only the placeholder case, where the later line is the one the skill intends and the value is correct. A user editing the first line and seeing nothing change is the cost; rewriting in place instead of appending would need a `sed -i` whose BSD/GNU spelling differs, which is not worth trading a portability trap for a tidiness gain. + +## Deferred in round 2 + +Round 2 (2026-09-13, profile 4) read round 1's own fixes and the founder's ruling together. What it fixed is in the pull request and the changelog; these it confirmed and left, at a bar that admits confirmed defects and drops improvements. + +- **`` is a placeholder and the document never quotes it.** Raised by cubic as its highest finding, and the sharpest edge is real: `rm -rf /.git` in the default recipe would, under a `` holding a space, expand to `rm -rf my app/.git` and delete a sibling named `my`. Every shape was executed on this machine's BSD userland before deferring, and all of them fail closed — a space makes `$(dirname …)` emit two lines so `mktemp` fails; `ls -A my app` errors and the guard's string can never equal `.git`; a leading dash is eaten as `ls` options; and the destructive line is unreachable because `git clone my app` rejects the extra argument and `|| exit` fires first. The preserving recipe is quoted throughout as of this round, since resolving the destination gave it a variable to quote. What is left is a documentation-wide convention — quoting the placeholder in every recipe, or one sentence telling the agent to quote whatever it substitutes — which should be done across the file in one pass rather than at the one site a round happened to touch. +- **The recipes have more than one canonical copy.** cubic's observation that the skill body and `references/starters.md` both carry the acquisition and initializer commands, and that these have already drifted twice — round 1 found a guard pinned against the reference alone, round 2 found the pathspec and the double-clone on the other side. The diagnosis is right and the remedy is a structural change (one source the body, the reference and the tests all read), which is a decision rather than an edit and should not be made inside a review round. +- **The suite asserts prose sentences as well as behaviour.** cubic reads the exact-sentence assertions across the template and three renders as coupling that makes restructuring expensive. Declined on the merits rather than only on the bar: the skill *is* prose that instructs an agent, so a sentence going missing is the failure mode, and every half-application this branch has suffered was caught by exactly such an assertion. The executable recipe tests already carry the behavioural half. +- **The report is not required to say the commit landed on the user's existing branch.** Raised by `code-review`. The Mode section now confirms that commit before it is made, which puts the fact in front of the user at the moment it matters; saying it again afterwards is an improvement, and it applies identically to both branches, so it wants doing once for both. +- **A `` whose `.git` is a file — a linked worktree or a submodule working directory — is written into.** Weighed deliberately this round rather than merely noted, and left. Executed end to end: `ls -A` returns exactly `.git`, the guard passes, the copy lands, and the pristine commit goes onto that worktree's branch in the shared repository. It is left because it is still the user's own repository standing, because `git rev-parse --show-toplevel` prints `` itself for such a directory so branch B accepts the identical case already — and making branch A alone refuse would re-create the A/B disagreement `L-260913-f28d9d` was ruled to remove — and because the shape is barely reachable: a worktree or submodule directory normally holds its branch's files, so `ls -A` returns more than `.git` and the chain refuses before copying. Nothing is destroyed on the path that does reach the copy, and the commit is an ordinary one the user can reset.