Repository navigation
Description
Epic. Owner request: Jerry, 2 Nov 2026, in the gitmoot-pipe seat session. Owner seat: gitmoot-pipe (merge rule self). Each step below is its own sub-issue and PR, merged after one independent exact-head review and green CI.
Why
lenscale.ai needs a scheduled production job with one AI-agent step: weekly "news-research", jerryfane/energy-finance-map#345. Gitmoot's gitmoot pipeline already does this, but the owner does not want Gitmoot (daemon, PR watcher, org, reviews, dashboard) as a production dependency. stagehand is the pipeline runner extracted into a small tool of its own, with omp as the harness for agent stages.
Owner decisions (2 Nov 2026)
| Topic | Decision |
|---|---|
| Approach | Build a standalone tool, not a timer and script first |
| Name and repo | stagehand, gitmoot/stagehand, public, Apache-2.0 |
| Model access | omp subscription through omp's own auth-gateway / auth-broker. No API key, no keyring |
| Quota | Shared with the fleet. No separate account |
| Cap | A usage cap (omp's list-price cost.total) is fine. It is not a dollar bill |
| Where it runs | hetzner2. Agent stages run in a dedicated sandboxd instance stagehand on hetzner2. Later this moves to a lenscale server, so hosts, ports and endpoints are settings, not hard-coded |
| Gitmoot convergence | Decided after v1 has run in production. The core keeps Store / Executor seams so Gitmoot could adopt it |
| Fleet rule | "Automatic agent steps may produce data, not code", added to the fleet guide. The agent cannot write files or run commands. The runner saves its answer. A fixed-rule script checks the answer and writes it. Every change is logged and can be undone. There are time and usage limits |
What v1 does
Spec (YAML). A subset of Gitmoot's pipeline spec, so specs stay recognisable, plus a few new keys:
name: news-research
description: Weekly research of rows marked "to research"
schedule: {interval: 168h, jitter: 30m}
env_file: /etc/stagehand/news-research.env # 0600 KEY=VALUE secrets
env: {EFM_LIMIT: "50"} # non-secret defaults
stages:
- id: export
cmd: python -m efm.research export --out "$STAGEHAND_WORK/rows.json"
cwd: /root/efm-daily
timeout: 10m
retry: 1
env_keys: [EFM_DB_URL, EFM_LIMIT] # deny by default
- id: research
needs: [export]
agent: omp
model: anthropic/claude-opus-5-5
effort: high
prompt_file: prompts/research.md
inputs: [rows.json] # copied from the run work dir into the VM
output: decisions.json # final answer, must be valid JSON
tools: [read, web_search]
max_cost: 5.0
timeout: 50m
- id: write
needs: [research]
cmd: python -m efm.research apply --in "$STAGEHAND_WORK/decisions.json"
cwd: /root/efm-daily
env_keys: [EFM_DB_URL]
- id: summary
needs: [write]
cmd: ./scripts/gram-summary.shShell stages
- Run
sh -c <cmd>incwd, with an exact environment:env_keysplusSTAGEHAND_*. The service environment is never inherited. - They run in their own process group, which is killed on timeout or cancel.
- The exit code decides the result.
- A stage may write
$STAGEHAND_RESULTas{"decision":"blocked","summary":"…","needs":["…"]}to park the run instead of failing. - stdout and stderr go to the stage log file.
Agent stages (agent: omp)
- Command: headless
omp -p --mode json --no-session --no-skills --no-rules --no-extensions --tools=<tools>, with--modeland--thinking, in a fresh sandboxd VM, as guest uid 1000. The agent has no write or exec tools. - Output: the runner writes
outputfrom the final assistant message and fails the stage if that message is not valid JSON. The agent never writes host files. - Caps:
- The time limit is the stage
timeout. omp also gets--max-timewith a margin. - The usage cap: the runner adds up
usage.cost.totalfrom eachmessage_endevent and kills the run once the total goes overmax_cost. - omp exits 0 even when a turn fails, so success is read from the event stream, not the exit code.
- The time limit is the stage
- Model calls never carry a real credential in the guest. The path is:
guest omp (baseUrl http://10.0.2.2:43293, apiKey = per-run random token)
-> hetzner2 127.0.0.1:43293 stagehand run gateway: listens only while an agent stage runs; checks + strips the per-run token
-> hetzner2 127.0.0.1:43294 reverse-SSH forward from the fleet host (stagehand-model-tunnel.service)
-> fleet host 127.0.0.1:43295 `stagehand gateway-proxy`: adds the omp auth-gateway bearer token
-> fleet host 127.0.0.1:4011 omp auth-gateway -> auth-broker (subscription)
- Item cap: the item cap per run (for example 50 rows) belongs in the consumer's export stage, not the runner.
Runs
- A blocked or failed stage parks the run, and nothing downstream starts.
resume <run> [--from <stage>]reruns that stage, the stages that depend on it, and any stage that has not succeeded.cancel <run>kills the stages that are running.- Only one run per pipeline can be active at a time.
- Each run stores a snapshot of its spec, so editing the spec never changes a run already in progress.
Recovery
- On start, the runner parks any stage still marked
runningfrom a crash with the reasoninterrupted. It does not retry it automatically. - It destroys any VM recorded for that stage.
Records (SQLite, pure Go modernc.org/sqlite)
- Per pipeline: spec, enabled,
next_due_at. - Per run: trigger, state, halt stage and reason, start, end, total cost.
- Per attempt: state, exit code, cost, tokens, log path, sandbox id, start, end, error.
Schedule: interval plus jitter. Missed intervals collapse into a single run.
CLI
add,list,show,runs,run [--wait],resume,cancel,enable,disable,remove,status --json.serve: the scheduler and executor loop, with an optional loopback-onlyGET /status.tick: one pass for timer mode.gateway-proxy: runs on the fleet host.- An exclusive lock makes sure only one process executes stages.
Status for admin.lenscale.ai: status --json and GET /status give, per pipeline:
- enabled;
- schedule;
next_due_at;- last run: id, state, start and end, halt stage and reason, cost.
Packaging: one static Go binary (CGO_ENABLED=0) with no GitHub, PR or org code. It runs as a systemd service.
Not in v1
- Pipeline chaining (
after:). - Service exposure and bearer-token run API.
- Receipts and proofs.
- Bundle publish and pull.
- Gitmoot's
implement,produce,orchestrate, PR gates and auto-merge. - Cron schedules.
sandboxd instance on hetzner2
The sandboxd seat drafted the plan and will build it. It is not created yet.
| Setting | Value |
|---|---|
| Instance | stagehand |
| Root | /var/lib/sandboxd-stagehand/fc |
| uid-base | 3010000 (Phobos uses 3000000–3001999) |
| Control API | 127.0.0.1:43292 |
| Guest host-port | 43293 |
| Size | max-vms 1, max-ttl 1h, 2 vCPU, 2048 MiB, 2048 MiB home disk, balloon on |
| Profile | strict (guest uid 1000, no root) |
| Image | built from the Phobos recipe's pinned inputs (omp 18.10.4, SHA-256 pinned), minus the Phobos-only parts |
The sandboxd seat confirmed there are no collisions:
- Host-wide names carry the instance name (
sbx_fc_stagehand,sbi-stagehand-<id>). - sandboxd refuses an instance that overlaps another one's root, UID range or volume dir.
Limits from phobos-dev:
- Do not restart
sandboxd-phobosor its tunnels. - Avoid Phobos's names and ports: 43290, 43291, 41359, 43221, 43222, 43341, 61324.
- Tell phobos-dev before touching the shared kernel, images, firewall or KVM.
- Schedule runs outside European daytime.
Known facts and risks
- omp/18.10.4
--mode jsonreportsusage{…,cost{total}}in each assistantmessage_end. Measured on 2 Nov: about $0.10 and about 12k tokens of fixed system-prompt overhead per call. - omp has no turn, token or cost cap of its own, and no sandbox. That is why the runner enforces the caps and sandboxd isolates the agent. omp's
readtool also fetches URLs. - The host omp auth-gateway must run a current omp (gitmoot#2475).
- Any root process on hetzner2 can use port 43294. That is accepted: hetzner2 root is fleet-controlled, and the guest cannot reach that port.
- The quota is shared with the fleet, and rate limits can stop a run. A failed or over-budget run writes nothing (#345 acceptance).
Plan (sub-issues)
- Runner core: spec, store, engine, shell stages, scheduler, CLI, status, CI.
- omp agent stage: omp runner, stream parsing, caps, output check, local executor.
- sandboxd executor.
- Model path: run gateway,
gateway-proxy, tunnel unit. - sandboxd instance and image on hetzner2 (built by the sandboxd seat).
- Deploy on hetzner2 and the fleet host, docs, end-to-end proof.
- lenscale adoption (lenscale seat, via enyo; tracked in jerryfane/energy-finance-map#345).
Analysis and evidence: /root/fleet-tools/state/gitmoot-pipe/analysis.md on the fleet host. Code ported from gitmoot 1b124f60, internal/pipeline.
Activity
stagehandand omp image on hetzner2 #7jerryfane commentedon Nov 2, 2026
Progress (2 Nov 23:5x): runner core in #10 (closes #3); omp agent stage in #11 (closes #4); sandboxd executor and model path in #12 (closes #5 and #6). The PRs are stacked; each is in its exact-head level-3 review. The sandboxd seat has been asked to build the instance (#7).
jerryfane commentedon Nov 3, 2026
Status, 3 Nov 03:10 CET: the runner is built, reviewed and deployed. #3–#8 are closed; #13 and #15 (found live) are fixed.
v0.1.0-dev-c36b550.canarypasses end to end (cost about 0.016 per run). Next run: 2026-11-10.Remaining: #9, lenscale's news-research pipeline, owned by enyo-intern (briefed in msg 386634).