Skip to content

About

Gated worker/judge loop for coding agents: sealed gates, script-scored rounds, keep only what improves

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

1 Commit

Folders and files

Repository files navigation

whileYouWait

tests version

whileYouWait lets a coding agent work on a goal in rounds while you do something else, and keeps only the rounds that measurably helped.

You write down what "done" means as a few commands that pass or fail. Those checks are sealed. One agent edits the code, a different one runs the checks, and the CLI decides the verdict. A round that breaks something, or touches the checks, is thrown away. You come back to a branch with only verified improvements on it.

The idea comes from Andrej Karpathy's AutoResearch loop: one surface to edit, a scorer the editor can't touch, keep if better, otherwise undo.

item greeter
  round 1  KEEP     score 0.67   greet-hi passes
  round 2  DISCARD  regression   baseline-suite broke
  round 3  DISCARD  tamper       .wyw/sealed/shout-up.mjs modified
  round 4  DONE     score 1.00   all item gates pass
branch wyw/greeter, merge it yourself

Requirements

  • Node 18 or later, and git. The project must be a git repo with a clean working tree.
  • Any coding agent that reads AGENTS.md and runs shell commands. Claude Code gets a thin plugin on top.

Install

Claude Code:

/plugin marketplace add walangstudio/marketplace
/plugin install whileYouWait@walangstudio

Restart Claude Code, then /whileYouWait:init.

Working from a checkout instead: claude --plugin-dir path/to/whileYouWait, or put the folder in CLAUDE_CODE_PLUGIN_DIRS in the env block of ~/.claude/settings.json.

Any other agent: point it at this repo's AGENTS.md and ask it to run init, then go.

Commands

Command Does
/whileYouWait:init Set up .wyw/ in the project, write the project-memory section to AGENTS.md, list existing tests for sealing
/whileYouWait:go [--rounds K] [--yes] [--tune] <goal> Draft the checks, show them to you, seal them, run up to K rounds
/whileYouWait:cycle Run one backlog item
/whileYouWait:draft-gates Only draft the checks for an item
/whileYouWait:seal Seal the drafted checks
/whileYouWait:tune Turn repeated failures into playbook habits, after you approve them

Commit what init lists before you run go. --yes skips the review of the drafted checks.

When it fits

  • Work you can score with a command: tests, a metric, a golden output.
  • The agent can run what it builds.
  • You have tokens to spare. Discarded rounds still cost tokens.

It doesn't fit one-off jobs, a whole app in one loop, or anything with no deterministic check. Do one feature, or a simple version of it, at a time.

How a round works

Step Who What
draft gate-author agent Writes the checks ("gates"). Each one must fail on the current code
seal CLI Moves them to .wyw/sealed/, hashes everything that scores, prints a seal digest
baseline CLI Runs the gates 3 times. Flaky ones are set aside; gates that already pass are rejected
build worker agent Edits code in a fresh git worktree
judge CLI + judge agent Checks the hashes, runs the gates 3 times (majority wins), decides
record CLI KEEP or DONE moves wyw/<item> forward. DISCARD deletes the worktree. ABORT (infra trouble) keeps it

Any regression discards. All item gates passing is DONE. Otherwise the score has to go up. The judge agent reports the verdict; it can't change it.

Gates

A gate is a command in .wyw/sealed/gates.json: {"id","kind","weight","cmd"}. Exit 0 passes. With "metric":{"min","max","threshold"} the last number it prints is the score. Kinds: rule, wiring, repro, baseline, regression.

Visual or subjective work needs a command that turns it into pass/fail or a number, like a screenshot diff.

What stops the worker cheating

  • Hashes. Gates, .wyw/score.mjs, .wyw/config.json and every closure path are hashed at seal time. A round that changes one, or adds a file to a sealed folder, is discarded as tamper. The check runs again after scoring, since the worker's code runs during it.
  • Values only the orchestrator holds. seal prints a digest and round start prints a round token. They live in the orchestrating agent's context, not on disk. The judge refuses (exit 5) if the state, ledger, manifest or item branch changed during the round, scoring included.
  • Existing tests are sealed by default. In a tested project, init lists them in closure and the first seal hashes them. Add test config there yourself.
  • Flaky isn't a loophole. A gate that starts flaking mid-item counts as failing and scores 0.
  • What's committed is what was tested. Ignored files are deleted before scoring, setup runs again so its output can't be hand-edited, and rewritten history is discarded.
  • A guard hook in Claude Code blocks direct edits to sealed files, worktrees included. It's a second layer. Shell commands can get around a hook; they can't get around the hashes.

Config

.wyw/config.json: score (command that prints gate JSON), setup (runs in every fresh worktree and again before judging, e.g. npm ci), runs (default 3), rounds (default 5), maxTotalRounds (hard cap, default 20), closure (extra protected paths).

Lockfiles aren't in closure by default, so the worker can add dependencies.

CLI

node bin/wyw.mjs <command>, from the project root:

Command Does
detect Print the project mode: greenfield, tested or untested
init Create .wyw/
seal Seal drafts, print sealDigest
item start <name> Start an item on branch wyw/<name>
baseline --seal-digest <d> Score the untouched code
round start --seal-digest <d> New worktree, print roundToken
round judge --round-token <t> Score the round and record the verdict
next [--rounds K] Say whether to stop
status Current state and the last 5 ledger rows
habits [--apply] Propose or apply playbook habits

--root <dir> works on any of them.

Known issues

  • It's not a sandbox. The worker runs as you, on your machine. The hashes and tokens catch tampering after the fact. They can't stop code that edits a gate and puts it back before the judge looks, or that plants a git-ignored file mid-scoring for a later gate to pick up.
  • Rounds run one at a time. A monorepo runs the full gate set every round.
  • .wyw/ is local state. It's kept out of git through .git/info/exclude.

Develop

npm test

How it's laid out:

  • bin/wyw.mjs: the CLI. Every verdict comes from here.
  • lib/core.mjs: sealing, scoring, the verdict rules, the tuner.
  • AGENTS.md and roles/: the host-neutral procedure and the three agent prompts.
  • templates/: what init copies into .wyw/.
  • .claude-plugin/, skills/, agents/, hooks/: the Claude Code adapter. Each file points back at AGENTS.md.

License

MIT, © walangstudio.

About

Gated worker/judge loop for coding agents: sealed gates, script-scored rounds, keep only what improves

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages