Skip to content

feat(proof): EvalExecutorOffer contract + harvest 1x / deadline enforcement - #242

Merged
echobt merged 16 commits into
mainfrom
cursor/eval-executor-offer-d9e4
Sep 8, 2026
Merged

feat(proof): EvalExecutorOffer contract + harvest 1x / deadline enforcement#242
echobt merged 16 commits into
mainfrom
cursor/eval-executor-offer-d9e4

Conversation

@echobt

@echobt echobt commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Summary

Implements the 2026-09-08 executor lock: a generic EvalExecutorOffer (where the digest-pinned proof-eval image is rented) as a sibling of the RLM judge InferenceOffer (which stays judge-only), plus harvest enforcement of exactly 1x and the proof deadline. No challenge-specific strings or topic-id branches — pin ceilings, offer, 1x, deadline, tighten-only, fail-closed 503.

Isolation invariants (stated, never weakened): the control-plane host runs neither the eval image nor the RLM judge; the harvest is the only path from the control plane to a rented GPU; the offer names a remote machine class, never a host process or a specific machine. The rent plan carries topic_id only as scope — a hook for a topic-scoped attach — and interprets nothing about it.

Pin ceilings (config/proof-pin.toml, no secrets)

eval_executor_schema_version = 1
gpu_class = "1x"
max_proof_deadline_s_ceiling = 7200
allowed_lium_template_prefixes = ["proof-eval-"]  # optional extra constraint
eval_executor_commitment_alg = "sha256"

Pin validation locks schema / gpu_class / alg, allows the deadline ceiling to tighten never loosen, and rejects unusable prefixes. Older pins without the keys still boot (defaults are the locked values, empty allowlist).

Live offer (operator state, PROOF_EVAL_EXECUTOR_OFFER_FILE; never git)
EvalExecutorOffer { offer_id, lium_template_id, machine_shape, max_proof_deadline_s, eval_image_digest, config_commitment, status }machine_shape must equal pin gpu_class, deadline ≤ ceiling, digest must match the pin when non-empty, config_commitment = sha256(canonical{template, shape, deadline, digest}). lium_template_id is the digest-scoped template name (must carry the pinned digest's 12-hex prefix); the harvest resolves it only through the digest-bound resolver (reuse a listed template iff its image is eval_image@digest, else create one bound to it). A raw Lium template UUID is refused under any allowlist (it would be rented verbatim with no image check), and a pin with no digest binds no executor.

HTTP

  • GET /v1/statuseval_executor (public offer, all fields) + pin executor ceilings.
  • GET /v1/proof/executor → always 200: eval_executor, ready, reason, pin.
  • POST /v1/admin/proof/executor (operator bearer) → rotate/close live; pin-refused body is 400 and the previous offer stays. In-memory until restart (like submissions).
  • Lium path: missing / closed / shape ≠ 1xcan_score=false → submit 503, no row. Sim rents nothing and does not consult it. Scored rows stamp executor_offer_id + executor_commitment (the commitment of the configuration the run was actually held to).

Topic tighten-only: eval_executor.{require_offer_commitment, max_proof_deadline_s} (deadline ≤ pin ceiling, taken as a min against the offer). No per-topic machine_id (deny_unknown_fields → parse reject). The field is omitted from the signed payload when empty, so existing signed topics keep verifying.

Harvest runtime (proof-harvest, harvest-pod, prism-lium)

  • The plan is resolved once (LiveScorer::plan(), harvest-side PROOF_HARVEST_* overrides) before anything is staged or rented and travels back in EvalOutcome.executor; score() runs under that plan.
  • InstanceSpec: template_name = plan.template_id (never template_id), gpu_count = 1, exact_gpu_count = true. prism-lium::provision skips any offer whose rent_gpu_count ≠ 1 without POSTing a rent.
  • Deadline: the resolved deadline is the pod-side timeout — never clamped below it by the host fallback (PROOF_EVAL_TIMEOUT_SECS, default now = pin ceiling); the harvest also bounds its wait (deadline + 300 s) and always tears the pod down. EvalPod is split into stage + run(deadline) so staging never eats the deadline.
  • Deadline provenance: the run command prints EVAL_DEADLINE_HIT when the wrapper ended the run (exit 124, or 137 after the full budget). That → EvalError::ProofDeadlineExceeded { deadline_s, stdout_tail }503 with the tail. A bare exit=137 is reported as an external SIGKILL (e.g. OOM), not a deadline.
  • Overrides vs commitment: a topic that pins require_offer_commitment refuses any override that changes the template or deadline (OverrideBreaksCommitment). Every plan carries offer_commitment and config_commitment (what actually runs); the request stamps both (executor_offer_commitment / executor_commitment).
  • Env hot-swap without rebuild: PROOF_HARVEST_TEMPLATE_ID / PROOF_HARVEST_GPU_COUNT / PROOF_HARVEST_DEADLINE_SECS replace the offer's values; pin ceilings still bind (out-of-ceiling, unparseable, or raw-UUID → refuse, never clamp).

Ceremony: cargo run -p xtask -- proof-executor-offer --offer-id <slug> --max-proof-deadline-s <s> --out <path> builds a pin-validated offer with the commitment filled in. --out is resolved against the workspace: tracked paths (git ls-files) and untracked-but-trackable paths (git check-ignore fails) are refused; only gitignored trees (deploy/secrets/**) or paths outside the repo are written.

Review follow-ups (Greptile)

  • P1 Bind UUID Templates → d7ae7a0
  • P1 Commit Effective Overrides → f02cfc9
  • P1 Honor Proof Deadline → 2389281
  • P1 Protect Tracked Files → be2c39f
  • P2 Identify SIGKILL Source → 2389281

Modules touched

  • New: crates/proof-executor (offer type, commitment, validation, serves_topic, HarvestOverrides, ExecutorPlan), xtask/src/proof_executor_offer.rs.
  • crates/proof-task (pin keys + validation, TopicEvalExecutor, is_hex64/is_slug made pub), crates/prism-lium-types (InstanceSpec.exact_gpu_count), crates/prism-lium (provision abort), crates/harvest-pod (stage/run(deadline) split, effective_run_timeout_secs, deadline marker, hit_deadline / killed_externally), crates/proof-eval (LiveScorer::plan/score, scoring_readiness, eval_after_freeze, EvalOutcome.executor, new EvalError variants), crates/proof-harvest, crates/proof-http, crates/proof-store (row stamp), crates/proof-challenge, bins/proof-challenge.
  • Config/deploy/docs: config/proof-pin.toml, docker-compose.yml, deploy/env/proof-challenge.env.example, deploy/secrets/README.md, deploy/scripts/proof-operator-path.sh, docs/PROOF.md, docs/external-miner/proof.md, docs/COMPLETENESS.md, docs/OPERATOR_SECURITY.md, docs/runbooks/proof-submit-e2e.md, AGENTS.md.

Greptile

Every PR is reviewed by Greptile before merge. Config: .greptile/.

  • Greptile has reviewed this PR; findings are fixed or answered (4× P1 + 1× P2 addressed in the commits above; replies posted on each thread)
  • If the bot was silent, I commented @greptileai review

Test plan

  • cargo test --workspace (all green). Fail-closed units include: pin locks; offer validate/commitment/shape/deadline/allowlist; raw UUID refused under any allowlist; unpinned digest binds nothing; plan + env override abort; commitment-pinned topic refuses config-changing overrides; resolved-plan commitment stamped on request/row; provision exact_gpu_count never POSTs a 2x rent (wiremock); harvest closed/8x/topic-mismatch/raw-UUID refuse before boot; 7200 s deadline honored above a 5400 s fallback; wrapper-caused kills marked and a real-shell early SIGKILL not; deadline → 503 + tail; hung pod torn down; HTTP status/executor/admin/submit 503 + no rows; binary boot with missing/bad offer file; xtask refuses README/Cargo.toml/new root files and leaves README untouched. No live Lium rent in CI.
  • cargo fmt --all -- --check
  • cargo clippy --workspace --all-targets -- -D warnings, cargo deny check, xtask loc-cap (proof-task 1488/1500, prism-lium 1495/1500), consensus-lint, spec-check, design-check, external-docs-check
  • Branch diff scanned for challenge-specific strings (benchmark / model / topic names): none; the only topic ids are pre-existing generic test fixtures.
  • deploy/scripts/assert-compose-matrix.sh needs docker compose (not available in the agent VM); the compose change is one env key in the existing proof-challenge block

Risk

  • Live Lium hosts stay 503 until an open 1x EvalExecutorOffer naming a digest-scoped template is on the host (file or admin POST) — intended fail-closed; sim/local-e2e unaffected.
  • EvalPod trait split (stage + run) and LiveScorer (plan + score(&ExecutorPlan)) are internal (no external implementers).
  • HarvestLimits.gpu_count removed (width is the plan's exact 1x). PROOF_EVAL_TIMEOUT_SECS default 5400 → 7200 (fallback only).
  • Topics that tighten eval_executor add a signed field; topics that don't are byte-identical under the signature. No emission, seal, or domain-tag changes. No secrets in git.

Naming

I did not rename BASE_* environment variables, deployed host paths
(/opt/base, /run/base, …), GHCR baseintelligence/base package names, or
base-*-v1 cryptographic domain tags, unless this PR's purpose is a coordinated
cutover documented in docs/NAMING.md.

Open in Web Open in Cursor 

cursoragent and others added 11 commits September 8, 2026 11:23
…torOffer crate

Co-authored-by: Mathis <echobt@users.noreply.github.com>
…ed width

Co-authored-by: Mathis <echobt@users.noreply.github.com>
…es deadline exits

Co-authored-by: Mathis <echobt@users.noreply.github.com>
… scorer

Co-authored-by: Mathis <echobt@users.noreply.github.com>
…d the run to the proof deadline

Co-authored-by: Mathis <echobt@users.noreply.github.com>
…ated status/submit

Co-authored-by: Mathis <echobt@users.noreply.github.com>
…he executor slot

Co-authored-by: Mathis <echobt@users.noreply.github.com>
…offer

Co-authored-by: Mathis <echobt@users.noreply.github.com>
…d harvest 1x enforcement

Co-authored-by: Mathis <echobt@users.noreply.github.com>
Co-authored-by: Mathis <echobt@users.noreply.github.com>
…n the rent plan

Co-authored-by: Mathis <echobt@users.noreply.github.com>
@greptile-apps

greptile-apps Bot commented Sep 8, 2026

Copy link
Copy Markdown

Greptile Summary

This update completes the proof-executor safety hardening: executor templates are digest-bound, commitment-pinned topics cannot be changed by harvest overrides, approved deadlines are honored by pod execution, external SIGKILL failures are diagnosed separately from deadline expiry, and offer-generation output cannot overwrite repository-managed files.

Confidence Score: 5/5

Safe to merge.

No outstanding findings remain. The five prior threads were resolved by cursor[bot] with current-code fixes: raw Lium template UUIDs are rejected before rental; commitment-pinned topics reject template or deadline overrides and executed configuration commitments are recorded; resolved deadlines are used directly for pod execution; bare external SIGKILL results are not classified as deadline exhaustion; and tracked or trackable workspace output paths are refused.

Files Needing Attention: None.

Reviews (3): Last reviewed commit: "docs(proof): executor binding, commitmen..." | Re-trigger Greptile

Comment thread crates/proof-executor/src/lib.rs Outdated
Comment thread crates/proof-executor/src/plan.rs
Comment thread crates/harvest-pod/src/lib.rs
Comment thread crates/harvest-pod/src/lib.rs Outdated
Comment thread xtask/src/proof_executor_offer.rs Outdated
@echobt

echobt commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

@greptileai review please — EvalExecutorOffer contract PR, CI green.

cursoragent and others added 5 commits September 8, 2026 12:57
…enance from the wrapper

Greptile P1 'Honor Proof Deadline': effective_run_timeout_secs no longer
clamps a resolved deadline to the host fallback, so an approved 7200 s proof
gets 7200 s on a host whose fallback is 5400 s. The fallback (default now the
pin ceiling) applies only when no deadline was resolved.

Greptile P2 'Identify SIGKILL Source': run_cmd records whether the timeout
wrapper ended the run (exit 124, or 137 after the full budget) with an
EVAL_DEADLINE_HIT line; hit_deadline keys on that, and a bare exit=137
(external SIGKILL such as OOM) is reported by killed_externally instead of
being called a deadline.

Co-authored-by: Mathis <echobt@users.noreply.github.com>
…kspace path

Greptile P1 'Protect Tracked Files': the output path is resolved against the
workspace root; inside it, git ls-files (tracked) and git check-ignore
(would be committed) decide, so README.md, Cargo.toml, or a new offer.json at
the root are refused while gitignored deploy/secrets/** and paths outside the
repo are allowed. Git unavailable → refuse. Tests cover tracked, trackable,
ignored, and outside paths and prove run() leaves README.md untouched.

Co-authored-by: Mathis <echobt@users.noreply.github.com>
…fuse raw Lium UUIDs

Greptile P1 'Bind UUID Templates': an empty allowlist no longer admits a raw
Lium template UUID, which the provider would rent verbatim before the
digest-bound resolver could check its image. lium_template_id must now be a
digest-scoped template name that carries the pinned digest's 12-hex prefix
(the resolver reuses a listed template only when its image is
eval_image@digest, else creates one bound to it); a UUID is RawTemplateId
under any allowlist, and a pin without a digest binds no executor
(UnpinnedDigest). The former test expecting a UUID under an empty allowlist
now expects Err.

Co-authored-by: Mathis <echobt@users.noreply.github.com>
…ics; commit the resolved plan

Greptile P1 'Commit Effective Overrides': a PROOF_HARVEST_* override that
changes the template or deadline is refused (OverrideBreaksCommitment) when
the topic pins require_offer_commitment — the topic approved the offer's
configuration, not the operator's. For every run the plan now carries both
the offer's commitment and the commitment of the configuration that actually
runs (template, 1x, effective deadline, digest); the harvest request stamps
both (executor_offer_commitment / executor_commitment) and the scored row's
executor_commitment is the executed one, threaded back through
LiveScorer::plan() -> EvalOutcome.executor.

Also in this change set: LiveScorer::score takes the resolved ExecutorPlan
(single resolution, harvest-side overrides), the harvest always resolves the
template by digest-bound name (template_id stays None), and a bare exit=137
is reported as an external SIGKILL rather than the proof deadline.

Co-authored-by: Mathis <echobt@users.noreply.github.com>
…eview

Co-authored-by: Mathis <echobt@users.noreply.github.com>
@echobt

echobt commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

@greptileai please re-review after P1 fixes (UUID bind, commitment overrides, deadline timeout, xtask tracked paths).

@echobt
echobt merged commit 0d41824 into main Sep 8, 2026
5 checks passed
@echobt
echobt deleted the cursor/eval-executor-offer-d9e4 branch September 8, 2026 13:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants