feat(proof): EvalExecutorOffer contract + harvest 1x / deadline enforcement - #242
Conversation
…torOffer crate Co-authored-by: Mathis <echobt@users.noreply.github.com>
…ed width Co-authored-by: Mathis <echobt@users.noreply.github.com>
…es deadline exits Co-authored-by: Mathis <echobt@users.noreply.github.com>
… scorer Co-authored-by: Mathis <echobt@users.noreply.github.com>
…d the run to the proof deadline Co-authored-by: Mathis <echobt@users.noreply.github.com>
…ated status/submit Co-authored-by: Mathis <echobt@users.noreply.github.com>
…he executor slot Co-authored-by: Mathis <echobt@users.noreply.github.com>
…offer Co-authored-by: Mathis <echobt@users.noreply.github.com>
…d harvest 1x enforcement Co-authored-by: Mathis <echobt@users.noreply.github.com>
Co-authored-by: Mathis <echobt@users.noreply.github.com>
…n the rent plan Co-authored-by: Mathis <echobt@users.noreply.github.com>
Greptile SummaryThis update completes the proof-executor safety hardening: executor templates are digest-bound, commitment-pinned topics cannot be changed by harvest overrides, approved deadlines are honored by pod execution, external SIGKILL failures are diagnosed separately from deadline expiry, and offer-generation output cannot overwrite repository-managed files. Confidence Score: 5/5Safe to merge. No outstanding findings remain. The five prior threads were resolved by cursor[bot] with current-code fixes: raw Lium template UUIDs are rejected before rental; commitment-pinned topics reject template or deadline overrides and executed configuration commitments are recorded; resolved deadlines are used directly for pod execution; bare external SIGKILL results are not classified as deadline exhaustion; and tracked or trackable workspace output paths are refused. Files Needing Attention: None. Reviews (3): Last reviewed commit: "docs(proof): executor binding, commitmen..." | Re-trigger Greptile |
|
@greptileai review please — EvalExecutorOffer contract PR, CI green. |
…enance from the wrapper Greptile P1 'Honor Proof Deadline': effective_run_timeout_secs no longer clamps a resolved deadline to the host fallback, so an approved 7200 s proof gets 7200 s on a host whose fallback is 5400 s. The fallback (default now the pin ceiling) applies only when no deadline was resolved. Greptile P2 'Identify SIGKILL Source': run_cmd records whether the timeout wrapper ended the run (exit 124, or 137 after the full budget) with an EVAL_DEADLINE_HIT line; hit_deadline keys on that, and a bare exit=137 (external SIGKILL such as OOM) is reported by killed_externally instead of being called a deadline. Co-authored-by: Mathis <echobt@users.noreply.github.com>
…kspace path Greptile P1 'Protect Tracked Files': the output path is resolved against the workspace root; inside it, git ls-files (tracked) and git check-ignore (would be committed) decide, so README.md, Cargo.toml, or a new offer.json at the root are refused while gitignored deploy/secrets/** and paths outside the repo are allowed. Git unavailable → refuse. Tests cover tracked, trackable, ignored, and outside paths and prove run() leaves README.md untouched. Co-authored-by: Mathis <echobt@users.noreply.github.com>
…fuse raw Lium UUIDs Greptile P1 'Bind UUID Templates': an empty allowlist no longer admits a raw Lium template UUID, which the provider would rent verbatim before the digest-bound resolver could check its image. lium_template_id must now be a digest-scoped template name that carries the pinned digest's 12-hex prefix (the resolver reuses a listed template only when its image is eval_image@digest, else creates one bound to it); a UUID is RawTemplateId under any allowlist, and a pin without a digest binds no executor (UnpinnedDigest). The former test expecting a UUID under an empty allowlist now expects Err. Co-authored-by: Mathis <echobt@users.noreply.github.com>
…ics; commit the resolved plan Greptile P1 'Commit Effective Overrides': a PROOF_HARVEST_* override that changes the template or deadline is refused (OverrideBreaksCommitment) when the topic pins require_offer_commitment — the topic approved the offer's configuration, not the operator's. For every run the plan now carries both the offer's commitment and the commitment of the configuration that actually runs (template, 1x, effective deadline, digest); the harvest request stamps both (executor_offer_commitment / executor_commitment) and the scored row's executor_commitment is the executed one, threaded back through LiveScorer::plan() -> EvalOutcome.executor. Also in this change set: LiveScorer::score takes the resolved ExecutorPlan (single resolution, harvest-side overrides), the harvest always resolves the template by digest-bound name (template_id stays None), and a bare exit=137 is reported as an external SIGKILL rather than the proof deadline. Co-authored-by: Mathis <echobt@users.noreply.github.com>
…eview Co-authored-by: Mathis <echobt@users.noreply.github.com>
|
@greptileai please re-review after P1 fixes (UUID bind, commitment overrides, deadline timeout, xtask tracked paths). |
Summary
Implements the 2026-09-08 executor lock: a generic
EvalExecutorOffer(where the digest-pinnedproof-evalimage is rented) as a sibling of the RLM judgeInferenceOffer(which stays judge-only), plus harvest enforcement of exactly1xand the proof deadline. No challenge-specific strings or topic-id branches — pin ceilings, offer,1x, deadline, tighten-only, fail-closed 503.Isolation invariants (stated, never weakened): the control-plane host runs neither the eval image nor the RLM judge; the harvest is the only path from the control plane to a rented GPU; the offer names a remote machine class, never a host process or a specific machine. The rent plan carries
topic_idonly as scope — a hook for a topic-scoped attach — and interprets nothing about it.Pin ceilings (
config/proof-pin.toml, no secrets)Pin validation locks schema /
gpu_class/ alg, allows the deadline ceiling to tighten never loosen, and rejects unusable prefixes. Older pins without the keys still boot (defaults are the locked values, empty allowlist).Live offer (operator state,
PROOF_EVAL_EXECUTOR_OFFER_FILE; never git)EvalExecutorOffer { offer_id, lium_template_id, machine_shape, max_proof_deadline_s, eval_image_digest, config_commitment, status }—machine_shapemust equal pingpu_class, deadline ≤ ceiling, digest must match the pin when non-empty,config_commitment = sha256(canonical{template, shape, deadline, digest}).lium_template_idis the digest-scoped template name (must carry the pinned digest's 12-hex prefix); the harvest resolves it only through the digest-bound resolver (reuse a listed template iff its image iseval_image@digest, else create one bound to it). A raw Lium template UUID is refused under any allowlist (it would be rented verbatim with no image check), and a pin with no digest binds no executor.HTTP
GET /v1/status→eval_executor(public offer, all fields) + pinexecutorceilings.GET /v1/proof/executor→ always 200:eval_executor,ready,reason,pin.POST /v1/admin/proof/executor(operator bearer) → rotate/close live; pin-refused body is 400 and the previous offer stays. In-memory until restart (like submissions).1x→can_score=false→ submit 503, no row. Sim rents nothing and does not consult it. Scored rows stampexecutor_offer_id+executor_commitment(the commitment of the configuration the run was actually held to).Topic tighten-only:
eval_executor.{require_offer_commitment, max_proof_deadline_s}(deadline ≤ pin ceiling, taken as a min against the offer). No per-topicmachine_id(deny_unknown_fields→ parse reject). The field is omitted from the signed payload when empty, so existing signed topics keep verifying.Harvest runtime (
proof-harvest,harvest-pod,prism-lium)LiveScorer::plan(), harvest-sidePROOF_HARVEST_*overrides) before anything is staged or rented and travels back inEvalOutcome.executor;score()runs under that plan.InstanceSpec:template_name = plan.template_id(nevertemplate_id),gpu_count = 1,exact_gpu_count = true.prism-lium::provisionskips any offer whoserent_gpu_count ≠ 1without POSTing a rent.timeout— never clamped below it by the host fallback (PROOF_EVAL_TIMEOUT_SECS, default now = pin ceiling); the harvest also bounds its wait (deadline + 300 s) and always tears the pod down.EvalPodis split intostage+run(deadline)so staging never eats the deadline.EVAL_DEADLINE_HITwhen the wrapper ended the run (exit 124, or 137 after the full budget). That →EvalError::ProofDeadlineExceeded { deadline_s, stdout_tail }→ 503 with the tail. A bareexit=137is reported as an external SIGKILL (e.g. OOM), not a deadline.require_offer_commitmentrefuses any override that changes the template or deadline (OverrideBreaksCommitment). Every plan carriesoffer_commitmentandconfig_commitment(what actually runs); the request stamps both (executor_offer_commitment/executor_commitment).PROOF_HARVEST_TEMPLATE_ID/PROOF_HARVEST_GPU_COUNT/PROOF_HARVEST_DEADLINE_SECSreplace the offer's values; pin ceilings still bind (out-of-ceiling, unparseable, or raw-UUID → refuse, never clamp).Ceremony:
cargo run -p xtask -- proof-executor-offer --offer-id <slug> --max-proof-deadline-s <s> --out <path>builds a pin-validated offer with the commitment filled in.--outis resolved against the workspace: tracked paths (git ls-files) and untracked-but-trackable paths (git check-ignorefails) are refused; only gitignored trees (deploy/secrets/**) or paths outside the repo are written.Review follow-ups (Greptile)
Modules touched
crates/proof-executor(offer type, commitment, validation,serves_topic,HarvestOverrides,ExecutorPlan),xtask/src/proof_executor_offer.rs.crates/proof-task(pin keys + validation,TopicEvalExecutor,is_hex64/is_slugmade pub),crates/prism-lium-types(InstanceSpec.exact_gpu_count),crates/prism-lium(provision abort),crates/harvest-pod(stage/run(deadline)split,effective_run_timeout_secs, deadline marker,hit_deadline/killed_externally),crates/proof-eval(LiveScorer::plan/score,scoring_readiness,eval_after_freeze,EvalOutcome.executor, newEvalErrorvariants),crates/proof-harvest,crates/proof-http,crates/proof-store(row stamp),crates/proof-challenge,bins/proof-challenge.config/proof-pin.toml,docker-compose.yml,deploy/env/proof-challenge.env.example,deploy/secrets/README.md,deploy/scripts/proof-operator-path.sh,docs/PROOF.md,docs/external-miner/proof.md,docs/COMPLETENESS.md,docs/OPERATOR_SECURITY.md,docs/runbooks/proof-submit-e2e.md,AGENTS.md.Greptile
Every PR is reviewed by Greptile before merge. Config:
.greptile/.@greptileai reviewTest plan
cargo test --workspace(all green). Fail-closed units include: pin locks; offer validate/commitment/shape/deadline/allowlist; raw UUID refused under any allowlist; unpinned digest binds nothing; plan + env override abort; commitment-pinned topic refuses config-changing overrides; resolved-plan commitment stamped on request/row; provisionexact_gpu_countnever POSTs a 2x rent (wiremock); harvest closed/8x/topic-mismatch/raw-UUID refuse before boot; 7200 s deadline honored above a 5400 s fallback; wrapper-caused kills marked and a real-shell early SIGKILL not; deadline → 503 + tail; hung pod torn down; HTTP status/executor/admin/submit 503 + no rows; binary boot with missing/bad offer file; xtask refuses README/Cargo.toml/new root files and leaves README untouched. No live Lium rent in CI.cargo fmt --all -- --checkcargo clippy --workspace --all-targets -- -D warnings,cargo deny check,xtask loc-cap(proof-task 1488/1500, prism-lium 1495/1500),consensus-lint,spec-check,design-check,external-docs-checkdeploy/scripts/assert-compose-matrix.shneedsdocker compose(not available in the agent VM); the compose change is one env key in the existingproof-challengeblockRisk
1xEvalExecutorOffernaming a digest-scoped template is on the host (file or admin POST) — intended fail-closed; sim/local-e2e unaffected.EvalPodtrait split (stage+run) andLiveScorer(plan+score(&ExecutorPlan)) are internal (no external implementers).HarvestLimits.gpu_countremoved (width is the plan's exact1x).PROOF_EVAL_TIMEOUT_SECSdefault 5400 → 7200 (fallback only).eval_executoradd a signed field; topics that don't are byte-identical under the signature. No emission, seal, or domain-tag changes. No secrets in git.Naming
I did not rename
BASE_*environment variables, deployed host paths(
/opt/base,/run/base, …), GHCRbaseintelligence/basepackage names, orbase-*-v1cryptographic domain tags, unless this PR's purpose is a coordinatedcutover documented in
docs/NAMING.md.