Repository navigation
Release gate: v0.123-dev → v0.123 (waits for cross-repo E2E) - #1670
MuncleUscles wants to merge 86 commits into
Conversation
* fix: surface stuck-head alert context * fix: avoid finalization claim deadlock * ci: narrow release workflow secrets * ci: keep codecov upload non-blocking * test: cover recovery storm health summary * test: cover no-progress health scan * test: cover no-progress health fallbacks * test: cover detailed consensus health endpoint
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the
|
…1669) * fix(fees): user-facing money correctness + canonical RPC alignment Money correctness (designer rulings — all user-visible): - appeal bonds now settle: bond × 3/2 credited to the appellant on successful appeal, explicit forfeiture on failure, 1:1 return on cancel; payouts credited to the recorded appellant (not the tx sender), idempotent across re-settles. Previously every bond was silently destroyed. - zero-value submitAppeal no longer bypasses min-bond validation (gasless mode unaffected) - unused pre-funded rotations refunded: rounds bill min(funded, actual LEADER_ROTATION events) + 1 leader slots instead of always rotations[i]+1 - LEADER_TIMEOUT rounds charge the leader allocation at 50% (extensible per-outcome multiplier table) - round indexing no longer counts LEADER_ROTATION events as consensus rounds (shared helper backend/consensus/history.py) — fixes settlement ladders and appeal-bond quotes when appealRounds > 0 Canonical RPC alignment (cross-backend receipt schema, superset rule): - gen_getTransactionStatus returns {status, statusCode} with canonical v0.6 ITransactions ordinals (0-14, incl. VALIDATORS_TIMEOUT=12, LEADER_TIMEOUT=13) - transaction payloads expose top-level txExecutionResult(Name) and a canonical fees object (deposit/userValue/distribution/locked/consumed as decimal strings; null when gasless) CI: GenVM precompile cache key now uses the actual GENVM_TAG (env or .env), failing loud when absent — the Dockerfile grep had been yielding an empty key since GENVM_TAG lost its default. * fix: keep transaction value an int; soften GenVM cache-key guard; black - top-level 'value' returns to plain int per the numeric-columns contract (tests/db-sqlalchemy/test_numeric_types.py) — the blanket _json_safe_numbers stringification from the fee-accounting merge broke it for values > 2^53; the breakage predates this PR but was masked by CI path filters until now. Big-int-safe consumers read the canonical decimal-string fees object instead. - GENVM_TAG cache-key step: warn + 'untagged' sentinel instead of hard-fail — the load-test and integration jobs run prebuilt images (--no-build) and legitimately have no tag; hard-failing broke them while the warning keeps the degradation visible. - black: 2 files reformatted (pre-commit gate). * style: restore calldata.py to pinned black 24.4.2 formatting Local black 25.1.0 had reformatted it differently from CI's pre-commit pin; the file was never part of this PR's functional changes.
* fix(ci): default GENVM_TAG for standalone PR builds docker-compose hard-requires GENVM_TAG since 963f935, but no PR workflow sets it and .env.example ships it empty, so any run that reaches the buildx bake step fails with 'GENVM_TAG is required' (first surfaced on #1672). Default to the GENVM_TAG repo variable, falling back to the current GenVM release, exported both to .env and the job env so bake and compose interpolation agree. * fix(ci): pin default GENVM_TAG to v0.3.0-rc3 (v0.6 fee host protocol) The Load Tests state-integrity step timed out with the Counter deploy stuck in PROPOSING (run 27292725606). Stack logs show every GenVM run failing instantly: genvm manager /genvm/run failed: 500 {'error': 'unknown variant `execution_budget_per_round`, expected one of: Internal, External'} The studio v0.6 fee code sends message_fee_allocation nodes in the GenVM v0.3.0-rc3 schema (MessageFeeAllocationNode: message_type/ parent_index + flat MessageFeeParams). GenVM v0.3.0-rc4 replaced that with the tagged Internal/External MessageAllocationNodeParams enum, so the rc4 manager rejects the payload, the leader burns its full 600s slot, and deploys end in LEADER_TIMEOUT (or sit in PROPOSING until the test gives up). Default the CI GenVM tag to v0.3.0-rc3 until the host protocol is migrated to the rc4 schema. vars.GENVM_TAG still takes precedence.
…ocol adaptation (#1674) * fix(ci): default GENVM_TAG for standalone PR builds docker-compose hard-requires GENVM_TAG since 963f935, but no PR workflow sets it and .env.example ships it empty, so any run that reaches the buildx bake step fails with 'GENVM_TAG is required' (first surfaced on #1672). Default to the GENVM_TAG repo variable, falling back to the current GenVM release, exported both to .env and the job env so bake and compose interpolation agree. * fix(ci): pin default GENVM_TAG to v0.3.0-rc3 (v0.6 fee host protocol) The Load Tests state-integrity step timed out with the Counter deploy stuck in PROPOSING (run 27292725606). Stack logs show every GenVM run failing instantly: genvm manager /genvm/run failed: 500 {'error': 'unknown variant `execution_budget_per_round`, expected one of: Internal, External'} The studio v0.6 fee code sends message_fee_allocation nodes in the GenVM v0.3.0-rc3 schema (MessageFeeAllocationNode: message_type/ parent_index + flat MessageFeeParams). GenVM v0.3.0-rc4 replaced that with the tagged Internal/External MessageAllocationNodeParams enum, so the rc4 manager rejects the payload, the leader burns its full 600s slot, and deploys end in LEADER_TIMEOUT (or sit in PROPOSING until the test gives up). Default the CI GenVM tag to v0.3.0-rc3 until the host protocol is migrated to the rc4 schema. vars.GENVM_TAG still takes precedence. * feat(docker): add GENVM_REF source-build path for GenVM images GENVM_TAG (release tarball) and GENVM_REF (git ref/SHA, built from source) are now mutually exclusive build args; exactly one must be set. The source path adds a nixos/nix builder stage that replicates genvm's own release recipe (.github/workflows/release.yaml + support/ci/pipelines/build.sh): patch flake-config.json with the resolved SHA, prefetch pinned deps, nix build manager-<arch>-linux / executor-<arch>-linux / runners-all, and merge the outputs into the same /genvm layout the tarballs produce. Prebuilt runner tars are preloaded from genvm's public GCS cache (fixed-output derivations) so runners-all rarely compiles anything. The tarball path is unchanged; post-install runs with --default-download false only for source builds (no release to download the executor from). * feat(ci): support vars.GENVM_REF — resolve to SHA, scope buildx cache by it When the GENVM_REF repo variable is set (e.g. 'main'), the workflows resolve it to a commit SHA via the GitHub API, pass GENVM_REF=<sha> as the build arg instead of GENVM_TAG, key the genvm-precompile cache on ref-<sha>, and move the buildx gha cache into a scope=genvm-<sha> namespace so image layers invalidate exactly when genvm moves. Behavior with the variable unset is unchanged (GENVM_TAG path). * feat(genvm): adapt host protocol to GenVM main (CON-549 fee schema) Studio spoke the rc3-era GenVM host protocol; genvm main (rc4+, #310) restructured the message-fee surface and the manager rejects our /genvm/run payload with 500 "unknown variant 'execution_budget_per_round', expected one of: Internal, External". Request (message_fee_allocation): flat parent-pointer nodes with message_type InternalAccepted/InternalFinalized/External and a 4-field fee_params dict become a nested tree ('children') with 'on' accepted|finalized and an externally tagged params enum — {'Internal': {7 fields incl. max_price_gen_per_time_unit, storage_fee_max_gas_price, receipt_fee_max_gas_price}} or {'External': {gas_limit, max_gas_price}}. The three price caps are sourced from Studio's fees_distribution (they are not part of the canonical per-node 5-field feeParams ABI). Non-root allocations are now sent as children of their roots so GenVM can emit correct subtrees (genvm executor/crates/common/src/domain/fees.rs, modules/implementation/src/manager/run.rs, tests/runner/origin/fees.py). Response (emissions): PostMessage/DeployContract/EthSend now carry fee_params as a struct and an ABI-encoded allocation 'subtree' (8-field internal params ABI). These are translated back at the GenVM boundary to Studio's canonical surfaces: 5-field internal / 2-field external feeParams bytes and the allocationSubtree node list. Studio's canonical fee ABI and fees.consumed reporting are unchanged. backend/node/genvm/origin/* is re-synced with genvm main's tests/runner/origin (new fees.py vendored module). * fix(fees): restore Mode-1 child bucket seeding; skip phase check for external allocations Two regressions from b88e059 ('enforce studio fee execution floor', June 4) that broke TestMessageFeeModes (added with the original v0.6 fee accounting, #1649): - create_child_fee_accounting zeroed the child's message-fee bucket when the message carried no allocation subtree, refunding the declared-budget remainder instead. Mode-1 children have no subtree by definition but must still receive the remainder as their own message bucket (FeeBudgetTransferred custody semantics). The leaf-refund expectation test from b88e059 is updated to the seeding semantics. - fill_message_fee_payload_from_allocation phase-checked (onAcceptance) external allocations; external messages have no accepted/finalized lifecycle (GenVM's matches_external ignores 'on' for the same reason). * fix(docker): disable nix syscall filtering in the genvm builder stage nix's seccomp BPF filter fails to load under qemu/Rosetta emulation ('unable to load seccomp BPF program: Invalid argument'), which blocks cross-platform local source builds (e.g. linux/amd64 on Apple Silicon). Harmless on native hosts — the nixos/nix image already runs with the build sandbox disabled. * feat(docker): GENVM_BUILD_CORES knob for the genvm source build Caps nix 'cores' for the genvm builder stage (default 0 = all). A full-parallel rust build can OOM memory-constrained or emulated builders (observed: cargo -j10 under Rosetta in a 16GB VM kills the nix daemon). * feat(ci): bind GenVM via checked-in third_party/genvm/version, env override Replaces the repo-variable mechanism: the GenVM binding is now a versioned file (same convention as genlayer-node), flipped to the release tag at release cut. Resolution order in CI: 1. GENVM_TAG / GENVM_REF already in the environment (e2e matrix / release train) wins; both set is a hard error. 2. Otherwise third_party/genvm/version: ^v[0-9] = release tag (tarball path), anything else = git ref (source build, resolved to a SHA for exact cache invalidation). Missing/empty file is a hard error rather than a guess. The file says 'main' on v0.123-dev, so PR CI builds genvm from source and validates the binding with no out-of-band setup. * fix(docker): derive genvm executor version from the bound ref's manifest genvm's patch-manifest requires the executor version to extend a version the committed manifest knows (base-match or previous-patch lookup); v0.0.0-dev<sha> matched nothing and failed the manager/executor builds ('Could not find any previous version for tag'). Derive the highest vX.Y.Z from modules/install/data/manifest.yaml of the ref being built and tag the source build vX.Y.Z-dev<sha> (verified against genvm's support/scripts/patch-and-check-manifest.py). * fix(docker): preserve exec bits when merging genvm source-build outputs cp --no-preserve=mode stripped the nix-store exec bits, so the base stage failed with '/genvm/bin/post-install.py: Permission denied' (exit 126) after an otherwise successful source build on CI. Preserve modes, then chmod -R u+w so post-install can write venvs/cache dirs. Also correct the builder comment: arm64 builds natively (the flake target follows TARGETARCH; runners are universal wasm from the GCS cache either way). * fix(docker): native arm64 genvm source builds + binding parse hardening - Work around genvm upstream bug: support/rust.nix patchelf's its rust toolchain with a hardcoded x86-64 ELF loader, breaking native aarch64-linux builds ('.cargo-wrapped: cannot execute'). On arm64 the builder rewrites it to ld-linux-aarch64.so.1, guarded to skip cleanly once fixed upstream and to fail loudly if the line multiplies. - nixos/nix ships no sed: install gnused alongside python3. - Harden the workflow binding parse: CRLF/whitespace trim + optional quoted values in third_party/genvm/version. Validated locally: full backend image builds natively on Apple Silicon (genvm stage 7m20s, GENVM_BUILD_CORES=6, no OOM); executor reports genvm v0.3.0-dev<sha>-aarch64-linux-release. (Implementation reviewed via codex.) * fix(docker): skip lief bin-patch for source-built genvm post-install's lief RPATH rewrite SIGABRTs on the zig/musl-linked amd64 executor built from source ('Can't read got entries' / 'Node not found. Can't resize the section'). Source-built outputs already carry $ORIGIN-relative rpaths from genvm's own nix patch-rpath, so the lief pass is redundant on this path; bin-check still verifies the binaries execute. Tarball path unchanged. * fix(genvm): retry transient 'modules not running' 500 from the manager Studio stops+restarts the genvm llm/web modules on every validator change; a /genvm/run landing in that window gets 500 'modules are required but not running' from the manager. The check exists in rc3 identically, but genvm main's module restart window is wide enough that mock-validator integration tests (which reconfigure validators right before each deploy) hit it near-deterministically — txs park in PROPOSING until timeout (run 27311560490). Route only this transient 500 (plus the sibling 'not all are running') into the existing GENVM_MANAGER_RUN_RETRIES retry loop via a dedicated GenVMManagerRetryableError; every other non-200 keeps failing fast (e.g. protocol errors must not be retried). Retry stats record the reason. (Implemented via codex; +2 parametrized regression tests.) * fix(genvm): match GenVM main's 32-byte U256 fuel framing in host protocol GenVM main widened fuel values to U256 in the host wire protocol; base_host.py still used rc3's 8-byte framing in two places: - REMAINING_FUEL_AS_GEN reply: we sent 8 bytes, the executor reads 32 (executor/src/host/mod.rs remaining_fuel_as_gen at v0.3.0-devef04f7f). The executor blocked on the missing 24 bytes while our host loop waited for the next method - a silent read-read deadlock. It sits in the exec_prompt path (wasi/genlayer_sdk.rs fetches remaining fuel right before the LLM module call), so every nondet execution parked in PROPOSING until LEADER_TIMEOUT while deterministic contracts were unaffected (runs 27311560490, 27314907925). - CONSUME_FUEL request: executor writes a 32-byte U256, we read 8, desyncing the stream whenever it is invoked. Locally wizard_of_coin went from a 10-minute hang to passing in ~11s. * fix(consensus): don't kill workers on transient NO_PROVIDER_FOR_PROMPT leader errors The llm module reports NO_PROVIDER_FOR_PROMPT as fatal, but in Studio it is transient/environmental: every validator change stops+restarts the llm module, so a leader run claimed inside that window (or on a stack without a real API key) sees a provider table lacking its node address. The fatal-leader circuit breaker stopped the worker in that case; with mock-validator tests reconfiguring validators before every test, all workers died one by one and the queue stalled silently (health: worker_task_died). The tx already has a bounded retry budget (recovery cycles escalate to CANCELED), so keep the worker alive and back off briefly (GENVM_TRANSIENT_FATAL_BACKOFF_S, default 3s) before resetting so retries don't burn through their cycles inside the same restart window. * test: modernize integration contracts to GenVM v0.3 SDK The stack now source-builds GenVM from main, where the floating runner aliases py-genlayer:test / :latest / py-genlayer-multi:test resolve to the genvm-main runner whose SDK no longer star-exports gl - every fixture still on an alias died at import with NameError: name 'gl' is not defined (and the upgrade suite cascaded into Contract-not-found). - Pin all alias fixtures to the v0.3.0-rc3 runner already used by every green contract (py-genlayer:1jb45aa8..., py-genlayer-multi:06zyvrl...): 8 icontracts fixtures, 5 inline upgrade-test fixtures, and multi_file_contract/runner.json. Bodies were already valid v0.3 SDK. - log_indexer: repin py-lib-genlayer-embeddings from the vanished 09h0i209... hash (invalid_contract) to 0bmbm3cy..., whose VecDB takes an explicit Distance type parameter - annotate with EuclideanDistanceSquared. - Allow overriding the integration RPC endpoint via TEST_JSONRPC_URL (request helper, conftest rate-limit probe, upgrade tests, utf8 roundtrip test). The previous hardcoded localhost:4000 made the suite untestable against a non-default stack; CI behavior is unchanged as the variable is unset there. Contract repins implemented via codex; verified against a local stack built from this branch (mock validators): 29/31 previously-failing tests pass; genvm_smoke needs a real LLM key (CI has one) and log_indexer is slow-but-valid locally (schema verified via RPC). * style: black-format worker.py backoff hunk * fix(genvm): discard storage changes reported by readonly executions GenVM main reports result_storage_changes for readonly (view) executions too - e.g. genlayer-embeddings VecDB lazily initializes its cover-tree storage on first access, which can happen inside a view knn call. provide_result applied those changes into the readonly state proxy and tripped 'assert not self.readonly' in storage_write, surfacing as '-32603 Internal error' on every gen_call to such a contract (test_log_indexer). View-execution storage changes are ephemeral VM-side effects: skip applying them. * test: upgrade integration contracts to genvm main SDK Supersedes the rc3 repin (098bbb8): instead of pinning fixtures backwards, every integration contract now uses GenVM main's SDK idiom, pinned to main's concrete runner hashes (no test/latest aliases): - py-genlayer:1zr6nqk597d97kg0dyxg0shhrykx5v02zjgnyrajapy4wlqvfvwh - py-genlayer-multi:0jw6688wgnbqghr2wwg2mqkf2ryb5if7w6bp5zcwbq6zlks5ybg4 - py-lib-genlayer-embeddings:0wcvi35grdr47ynkckzriz5sjn5080w2njk7v2cqx3xpp6p1989y Idiom changes (mirroring genvm's own tests, e.g. tests/cases/semi-stable/llm/woc.py): - import genlayer as gl + from genlayer import * (gl is no longer star-exported); class X(gl.contract.Contract) - gl.contract.get_at / gl.contract.deploy (were gl.get_contract_at / gl.deploy_contract); raise gl.vm.UserError (gl.Rollback is gone) - eq_principle.prompt_comparative principle arg is positional-only - @Allow from genlayer.storage (was @allow_storage) - exec_prompt(..., response_format='text') - embeddings VecDB takes an explicit Distance type param Backward-compat canary: examples/contracts/user_storage.py deliberately stays on the rc3 runner (py-genlayer:1jb45aa8..., old idiom) so test_user_storage keeps proving old-runner contracts execute on the genvm-main executor. (The executor parses the whole leading comment block as version+runner JSON, so the canary note sits after a blank line.) gltest 0.24.0 only discovers contracts with the legacy gl.Contract base; tests/integration/gltest_compat.py (activated in conftest) extends the AST matcher to accept both bases until the genlayer-test pin moves past 0.29.2 (upstream main already matches gl.contract.Contract). log_indexer specifics: the main embeddings runner pairs with the main py-genlayer (the old 09h0... hash no longer exists); similarity values recalibrated for its EuclideanDistanceSquared metric; update_log now locates entries by plain iteration because cover-tree knn from a write method currently trips GenVM main's deterministic-mode float trap (wasm_trap DeterministicMode) - views still exercise knn. Bulk rewrite implemented via codex; verified against a local stack built from this branch: previously-failing set + upgrade suite + canary all green locally except genvm_smoke (needs a real LLM key; CI has one). * style: appease autoflake on star-imports; skip genvm-lint for main-runner contracts - autoflake (pre-commit) removes 'from genlayer import *' where no star-exported name is used - accepted for the 7 contracts that only use the gl namespace. - Released genvm-linter (<=0.11.0) cannot load the genvm-main SDK layout ('No module named genlayer.py'); the lint workflow now skips contracts pinned to the main runner hash until a compatible linter ships (genvm-linter dxp-694), and keeps linting rc3-era contracts (canary, load/direct fixtures). * test(unit): cover host fuel-width framing, transient-fatal backoff, readonly storage discard Targeted unit tests for the Sonar new-code coverage gate (PR #1674): - test_host_loop_fuel_widths: drives host_loop over a socketpair and pins the GenVM main wire framing - CONSUME_FUEL reads a 32-byte LE U256 (value > 2**64) and REMAINING_FUEL_AS_GEN replies OK + exactly 32 LE bytes. Regression tests for the read-read deadlock fixed in dfdee12. - test_worker_transient_fatal: NO_PROVIDER_FOR_PROMPT fatal leader errors reset the tx, back off (3s default), and keep the worker alive; other fatal causes still stop the worker; validator-side errors are left to consensus. - test_provide_result_readonly: provide_result discards result storage changes for readonly (view) executions and applies them for writable ones (49db980). Also exclude examples/contracts/** from Sonar coverage: they are GenVM intelligent-contract fixtures executed inside the GenVM, never under the backend test runner (18 of the 38 'uncovered new lines' were contract fixture lines). Full tests/unit suite in-container: 903 passed, 1 skipped.
…d shape) (#1672) The canonical fees.consumed object now reports how many protocol time units the execution actually used, per round and in total: - leaderTimeunitsUsed / validatorTimeunitsUsed: decimal-string totals across all rounds (validator total includes the leader's validator-function execution) - perRound: [{round, consensusRound, leaderTimeunits, validatorTimeunits, maxValidatorTimeunits}] - maxValidatorTimeunits is the single largest validator execution that round, the number to compare against the per-validator validatorTimeunitsAllocation Unit mapping (documented at backend/consensus/history.py): 1 time unit == 1 second of GenVM wall-clock runtime (consensus spec glossary; Studio's DEFAULT_INITIAL_TIME_UNITS_ALLOCATION = 10*60). Consumption is derived from each receipt's measured processing_time (ms), rounded UP per execution - never raw seconds. Computation is a pure helper (time_unit_consumption) over the persisted consensus_history round entries: leader-rotation events fold into the next completed round, with a fallback to consensus_data when no history rounds exist. Backward compatible: new keys only; gasless transactions still return fees: null. This is the reference shape for the node to mirror 1:1; consumer: gltest --fee-profile (genlayer-testing-suite #94) fee-profile suggestions for leader/validator time-unit allocations.
…1677) ad77f6b changed the response to {status, statusCode} — a breaking change for deployed apps that parse the string (Rally polls it on every transaction). The string shape is a compatibility contract; the dict moves to a new additive method, gen_getTransactionStatusDetails. No in-tree consumer used the dict.
* test(ci): gasless-mode integration coverage Rally's hosted studio runs with all three price envs at zero (GENLAYER_STUDIO_GEN_PER_TIME_UNIT / STORAGE_UNIT_PRICE / RECEIPT_GAS_PRICE). Only unit tests covered that configuration; no integration/CI job ever ran a zero-price stack, so fee-path changes could silently break the hosted deployment. - tests/integration/test_gasless_mode.py (@pytest.mark.gasless): pins sim_getFeeConfig enabled=false + zero policy prices, fee-less deploy/write/read via the SDK, canonical receipt fees: null, the sim_estimateTransactionFees surface (the RPC genlayer-js estimateTransactionFees hits), and the tolerant behavior that explicit fee params on a gasless stack are accepted (required deposit is 0), not rejected. - pytest.ini: register the gasless marker and exclude it from the default sweep (same convention as error_handling). - backend_integration_tests_pr.yml: new test-gasless job reusing the bake/compose pipeline with the three price envs forced to 0, a readiness assert on sim_getFeeConfig, and a serial -m gasless run. Buildx cache is read-only in this job so it rides the main test job's cache without racing its writes. Verified locally against a zero-price compose stack (arm64, GENVM_REF source build): 2 passed. * test(ci): appease Sonar hotspots in the gasless job - SHA-pin docker/setup-buildx-action and docker/bake-action (S7637) - drop the explicit http:// scheme from the local readiness curl (S5332); curl defaults to http and the endpoint is runner-local only * refactor(fees): reduce canonical fee helper complexity * test(ci): stabilize gasless and oracle CI
Prevent Kubernetes liveness from restarting workers before the configured transaction timeout can own stuck transaction recovery. Validated by CI, load test, and focused worker health unit tests.
The RPC process holds one validators session for its whole lifetime, and
every gen_call, sim_call and sim_estimateTransactionFees reads the registry
through it (Manager.snapshot). When the Postgres connection dropped
mid-query, nothing rolled that session back, so every later execution
failed with PendingRollbackError ("Can't reconnect until invalid
transaction is rolled back") until the pod restarted. Plain RPC reads use
per-request sessions and kept working, which is the pattern in #1790.
Registry and interceptor methods now roll the session back on any
exception before re-raising. Success paths are unchanged.
The new db-sqlalchemy test terminates the session's backend with
pg_terminate_backend; without the fix the next read raises the exact
error from the report.
Fixes #1790
Lands four reviewed fork PRs together after a full E2E run on this exact head: - #1765: redact validator private keys from call receipts and explorer payloads - #1715: bind postgres, redis and ollama host ports to loopback - #1773: normalize addresses in raw-SQL balance operations - #1789: normalize the address in sim_fundAccount Co-authored-by: Kanan <93033289+kriss39@users.noreply.github.com> Co-authored-by: Starling <82058629+Starling-spell@users.noreply.github.com> Co-authored-by: Alicepoltora <Alicepoltora@users.noreply.github.com>
The response-format block lives in a plain (non f-) string, so the
doubled braces reached the LLM literally. A real model follows the
template and answers `{{...}}`, which json.loads rejects. This made
test_llm_erc20 fail whenever the mock did not apply and the call fell
through to the real provider (CI runs on #1784, 2026-09-10).
Co-authored-by: Claude <noreply@anthropic.com>
…ns (#1784) * fix(genvm): forward flat fee allocation inputs 🐛 Re-vendor the fee allocation types and replace recursive children with checked direct-child budgets and host-encoded FlatArrays subtrees. Preserve committed fee bytes, ordering, roots and original allowances. Pin GenVM v0.6.0-rc7, the release cut from the fee-allocation branch. * test(genvm): pin the reroute test to the rc9 executor line ✅ GenVM v0.6.0-rc7 ships executor v0.3.0-rc9; the missing v0.3.0-rc7 directory left the pinned deploy stuck in PROPOSING * fix(fees): reserve parent appeal capacity in Studio allocations Price each on-acceptance internal allocation using its emitting parent’s prepaid appeal horizon and multiply the complete child requirement, including direct child budgets. Carry the root appeal count from the transaction or child fee distribution. Fund every discovered acceptance emission opportunity while keeping an inferred per-message declared budget to one child primary plus its direct subtree. Cover the 360/1440 boundary, nested parents, finalization, discovery, and child accounting. --------- Co-authored-by: kp2pml30 <kp2pml30@gmail.com> Co-authored-by: Daniel Rojas <danielrojasc888@gmail.com>
* fix(consensus): finalize transaction-local accepted state Persist each successful acceptance's post-state alongside its rollback snapshot, then reload only that post-state during finalization. Verify receipt hashes and preserve agreed decisions when legacy state cannot be recovered safely, rather than promoting a younger accepted head or canceling the older transaction. Cover overlap, deployments, failed and successful appeals, recomputation, empty state, execution errors, and legacy rows through the real PostgreSQL worker path. No schema migration is required. Rollout must drain or explicitly recover pre-fix in-flight decisions that lack verifiable state. Validation: 1690 unit tests passed (repository-recommended RPC manager exclusion); 244 database tests passed, 1 existing xfail; changed-file pre-commit checks passed. Full stack E2E not run. Local branch only pending release-plan confirmation. * fix(consensus): skip backed-off finalization heads without reordering contracts A finalization head in retry backoff was claimed and released every poll, so the oldest stuck head starved every other contract's finalization. claim_next_finalization now takes the worker's backed-off hashes and skips contracts whose head (or an older non-terminal tx) is backing off, while the existing ordering guard still blocks younger same-contract txs. Retry tracking is cleared once a finalization succeeds. * fix(consensus): recover legacy finalization state from successor pre-states Rows accepted before transaction-local finalization state was persisted can only finalize when the moving contract head still hashes to their receipt. Once a successor is accepted the head moves and the row is stuck. A successor's saved pre-state is exactly the predecessor's post-state when nothing intervened, so after the existing sources fail the resolver now tries the transaction's own pre-state (unchanged-state writes) and up to three successor pre-states, accepting only one whose hash matches the receipt's contract_state_hash. Anything else still raises FinalizationStateError without canceling. Adds a read-only finalization_legacy_report CLI that classifies pending legacy rows with the same resolver before deploy. * fix(consensus): finalize decided predecessors before a code upgrade applies A type 3 upgrade used to swap the code slot while earlier decided transactions on the same contract were still inside their appeal window. Those rows then finalized, or were re-executed by an appeal, against code they were never decided on, and their paid appeals stayed open across the code change. The upgrade now closes the contract's earlier lifecycle before touching code: - It defers while any other row on the contract holds a lease or is mid-consensus, so it never races live work (UPGRADE_DEFER_SECONDS). - Every ACCEPTED+SUCCESS predecessor must resolve a verified finalization state up front. If one cannot, the upgrade is canceled with predecessor_finalization_state and nothing is changed. - Each decided predecessor (ACCEPTED, UNDETERMINED, *_TIMEOUT) is force-finalized through the normal path: acceptance delivery repair, messages, fee settlement and value refunds. A pending paid appeal is refunded first via abort_pending_appeal_admission, which only aborts an admission with no recorded outcome. - Progress (closed rows, refunds) is persisted on the upgrade row in the same commit as each refund, and that write also renews the lease. A retry skips rows that are already final and can never refund twice. - The code is applied last, in one commit with the FINALIZED status, after rechecking ownership and that no predecessor reopened. - Never-started PENDING rows stay queued and run on the new code. Validation and contract errors cancel the upgrade as before. Transient failures go through the generic retry path, and rows that are already finalized stay final. Co-Authored-By: Claude <noreply@anthropic.com> * fix(consensus): upgrades permanently close the appeal and rewind lifecycle Force-finalizing predecessors only holds if nothing can reopen them afterwards. Before this change, an appeal could still be admitted while the upgrade was queued. A successful appeal's rewind also reset every newer row on the contract to PENDING, including FINALIZED and CANCELED rows and the upgrade itself. That replayed an applied upgrade and resurrected terminal transactions. - Appeal admission is rejected with CanNotAppeal when a PENDING or FINALIZED upgrade is queued after the decision. The check runs in the RPC precheck and again under the row lock in admit_transaction_appeal. - Rewinds only reset rows that can still be replayed (get_rewindable_newer_transactions). Terminal rows and upgrades are never reset, and rollback_transactions guards this again in memory. Upgrades applied before this change could overtake decided rows that are still open. When such an older row finalizes, or a validator appeal on it restores its pre-state, the code of the latest FINALIZED upgrade after it is kept (with_later_upgrade_code). Without this, the old code would come back. The legacy finalization report now also lists these rows as "interleaved upgrade (<hash>)" so operators can see them before deploying. Co-Authored-By: Claude <noreply@anthropic.com> * fix(consensus): keep untyped legacy rows rewindable `type != UPGRADE_CONTRACT` evaluates to NULL for rows with no type, so the rewind query silently dropped them. Treat a NULL type as rewindable. Validation: unit 1731 passed (known env-only test_studio_fees failure), PostgreSQL suite 268 passed, 1 xfailed. Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Daniel Rojas <danielrojasc888@gmail.com> Co-authored-by: Claude <noreply@anthropic.com>
An unexpected exception in a JSON-RPC handler was returned to the client as str(exc). For SQLAlchemy errors that text carries the SQL statement and all bound values, so a failed INSERT into validators could return a validator private key. sim_createValidator was also the only validator mutator without the hosted-mode guard. - Gate create_validator with check_forbidden_method_in_hosted_studio, like the other validator mutators; the hosted UI already hides it. - Set hide_parameters=True on the RPC and consensus worker engines so no exception text (responses, logs, persisted receipt errors) includes bound values. - Return "Internal database error" for SQLAlchemy errors in the RPC catch-all; other unexpected errors keep their message. Tests cover each part and fail when the corresponding change is reverted.
…1802) * fix(genvm): route legacy-runner contracts to the retained v0.2 executor The manager picks an executor line by major version only, so every major-0 contract that is not explicitly `# v0.2.x` resolves to the newest line (v0.3). That line ships only its own runners and rejects older ones as `invalid_contract runner malformed`. Contracts with a `# v0.1.x` header, no version line, a Seq header, or a zip runner.json that depend on v0.2 runners could therefore not be deployed on v0.123, and factories could not deploy such children (child deploys carry no executor pin). Also select the retained line when the code's runner spec (parsed like GenVM: zip runner.json, else the leading comment block after an optional version line) declares a runner that appears in an installed `$GENVMROOT/executor/v0.2.*/data/all.json` and in no newer line. Contracts that existed before the multi-line upgrade are already pinned by migration c1d2e3f4a5b6, so no data migration is added. Validation: on studio-next (v0.123.0-rc.8), simulated gen_call deploys of Rally's XID (`# v0.1.0`), CampaignIC (unversioned Seq) and CampaignFactory (zip, py-genlayer-multi) fail with "runner malformed" without a selector and load the runner (constructor TypeError for missing args) with `re:^v0\.2\.`. Registry path checked in the rc.8 jsonrpc image. * test(genvm): keep fake runner ids out of the image pin assertion The backend image greps py-genlayer:<hash> from backend/examples/tests and requires each pin to be installed, so the literal latest/unknownhash ids failed the CI image build. * fix(genvm): limit runner-based v0.2 routing to children of v0.2 contracts Studio's policy is that new deployments run on the newest GenVM line, while contracts already on the retained v0.2 line keep working, including the contracts they deploy (factory children, shards). The previous commit routed any new deployment declaring a v0.2-only runner to v0.2, which also admitted new top-level legacy deployments. Restore #1762's header-only rule for top-level deployments, simulated deployments and schema reads, and move runner detection to a separate legacy_executor_selector_for_child_code. At child insertion, consensus pins a deploy child to re:^v0\.2\. through its sim_config when the emitting contract's stored selector is a v0.2 pin (migration backfill, explicit pin or an earlier derived pin) and the child's code is v0.2-only. The child's snapshot factory and accepted effect already honor a sim_config pin, so the selector is persisted on the new contract. An inherited explicit pin still cascades unchanged, and non-deploy children are untouched. Validation: backend unit suite 1773 passed, 7 skipped on a clean checkout; black clean. * fix(genvm): keep inferred v0.2 pins local and runner sniffing non-fatal Address Codex review on #1802. An inferred child pin was stored in the child's sim_config like an explicit override, so it cascaded to every descendant: a v0.2-derived child deploying current-line code forced that grandchild onto v0.2. Mark inferred pins with genvm_executor_selector_inferred, strip them before cascading, and recompute per deployment. Explicit pins keep cascading as before; to_dict only emits the flag when set. Runner sniffing now runs during consensus message emission, so it must never raise. Treat any unreadable archive (encrypted, unsupported compression) and over-deep JSON as "no declared runners", and walk the parsed spec iteratively. Invalid code still reaches GenVM's invalid-contract path. Validation: new tests fail on the previous head (6) and pass here; backend unit suite 1777 passed, 7 skipped on a clean checkout.
The v0.2 executor emits internal messages with messageFee equal to the minimum primary fee while still carrying the allocation subtree. Studio derives the child's message budget as declared - primary, so it got 0, never matched the subtree sum, and create_child_fee_accounting raised MessageAllocationsNotEqualBudget. A single-node tree left the child with no message budget instead, so its own deploys failed with OOM fees. Either way a v0.2 contract could not fund grandchildren, which breaks Rally create_campaign (factory -> CampaignIC -> shards). When the declared budget is exactly the primary fee and the matched allocation pins children, declare primary + children, as the v0.3 executor already does. Other declared budgets and grant-derived trees are left unchanged. Validated on a local rc.9 stack with this module mounted: the Rally factory pinned to v0.2 now creates the CampaignIC and its shard, and a v0.3 Top -> Mid -> Leaf nested deploy is unchanged.
…pted txs (#1807) AcceptedState committed the contract's accepted state in pre_effects (register_contract / update_contract_state commit the session) and only then dispatched accepted-phase messages. When dispatch raised, the worker rolled back only the dispatch writes and retried; the stuck-tx recovery reset the tx to PENDING and re-executed it on top of its own committed post-state. With a Rally factory this registered campaign X, then the retry hit the contract's "Id already exists" guard and the tx finalized with no child and messageFeesConsumed 0; a contract without such a guard would apply its writes twice. The deterministic trigger was create_child_fee_accounting rejecting a child (MessageAllocationsNotEqualBudget, MessageDeclaredBudgetInsufficient, BudgetTooLow, PhaseTimeoutOutOfBounds, ...), which _attach_child_fee_accounting wrapped into a RuntimeError for the whole phase. Allocation-match errors (MessageNoMatchingAllocation, MessageEmissionPhaseMismatch) and reveal-time budget errors already raise in _sync_reveal_message_fee_accounting, before anything in AcceptedState commits, so they never hit this path. Two changes: - Child creation failures are contained per child, mirroring ConsensusMain._processInternalMessages, which wraps each child creation in try/catch and emits InternalMessageSkipped instead of reverting the phase. The child is marked, _author_message_phase_locally gives it the zero tx id like any other rejected child, and the existing _refund_skipped_internal_message path returns its budget/value and records skippedReason. The parent is ACCEPTED and finalizes normally. This covers the finalized phase too, since it shares _get_messages_data. - decide_accepted moves the contract write effects from pre_effects to the head of post_effects, after dispatch. The contract write is now the commit that publishes the children together with the post-state, so any other dispatch failure (DB, invariant, helper) leaves the contract head untouched and the existing retry re-executes from the true pre-state. Transient errors keep their retry/cancel behaviour. Rejected alternatives: a fee preflight before pre_effects (duplicates _get_messages_data and still needs a terminal outcome; containment makes it unnecessary); resuming dispatch instead of re-executing (needs new "acceptance committed" state plumbing through claim and recovery); dispatching before all pre_effects (children would commit before the parent's contract state, visible to cross-contract reads); deferring all session commits for the whole state (monkeypatching commit, stale ORM objects after raw SQL updates). Tests: - test_studio_fees: rejected child fee accounting marks the child and the local authority assigns the zero tx id (raised RuntimeError before). - db-sqlalchemy: a failing dispatch leaves the accepted head at the pre-state (was committed before); a rejected child fee accounting accepts the parent with no child, message_fee_consumed 0 and a recorded skip reason (raised before). - decide_accepted / legacy executor tests updated for the new effect placement, plus an ordering test. Unit suite (clean copy outside the main checkout to avoid its .env): 1779 passed, 7 skipped. DB suite against a throwaway Postgres: 270 passed, 1 xfailed.
…1810) - Re-vendor origin/base_host.py from genvm-manager: run_genvm takes an optional network and adds it to the run request only when set; the base64 decode for the http shim stays a local patch - Studio passes its own GENVM_NETWORK constant on every run - Every run, schema runs included, now carries a timestamp, the current time when the transaction has none: the manager picks the executor line by the forks active at it, and the 2024 fallback predates the studio v0.2 fork - Pin genvm to v0.6.0-rc11, the first release that gates executor lines and runners by network forks
…) (#1818) * fix(rpc): add opt-in NewTransaction receipt log for pre-v2 SDKs Since #1748 receipts only carry CreatedTransaction. genlayer-js before v2 only parses NewTransaction, so it throws "Transaction not processed by consensus" for a transaction Studio did process, and the caller never gets the txId. GENLAYER_STUDIO_LEGACY_NEW_TRANSACTION_RECEIPT_LOG (default off) adds the legacy log after CreatedTransaction, so a Studio can be upgraded before its clients migrate. These SDKs also need gasless mode, since they send no fees. The default keeps the receipt faithful to CreationPhase. Validated with an in-place upgrade from v0.121.25 on rc.10 images with fees off, using genlayer-js 0.20.3 and 0.18.9. * fix(rpc): run sync RPC handlers and snapshot loads off the event loop Forward-port the RPC threadpool part of #1766 (2004fc4) to v0.123: sync JSON-RPC handlers run in the threadpool, async endpoints load contract snapshots and ConsensusData queries in workers, and worker notifications are published on the app loop via call_soon_threadsafe. The validator registry read handlers stay on the loop because they use the validators manager's shared long-lived session. * fix(genvm): run v0.2 line selectors no earlier than the studio v0.2 fork A sim_config.genvm_datetime before 2026-01-01 made the manager drop the v0.2 line (studio network fork in genvm-chainforks.json, v0.6.0-rc11), so every run of a pinned contract or a '# v0.2.x' deploy was refused before start: gen_call errors, and consensus retried until cancel while counting GenVM failures toward the worker health threshold. Clamp such runs to the fork activation time. * fix(genvm): give each executor version its own precompile cache dir /genvm-cache is a node-wide hostPath shared by every Studio release on the node, and GenVM keys precompiled modules only by runner hash. v0.121 (v0.2.16) and v0.123 (v0.2.17, v0.3.x) wrote into the same pc/ tree, so v0.2 calls could load modules built by another executor and time out. Point each executor's cache_dir at /genvm-cache/<executor-version>/, keep the precompile marker inside each per-version dir, and leave the legacy shared pc/ tree untouched for older Studios. Bump the CI precompile cache key so old-layout caches are not restored. * fix(migrations): lift inherited statement_timeout for evm envelope and queue order backfills c1d2e3f4a5b6 sets SET LOCAL statement_timeout = '30s' and env.py runs the whole chain in one transaction, so e4f6a8c0d2b4 and f5a7c9e1b3d5 inherited the limit and were cancelled on prod-sized data, rolling back the upgrade. Reset statement_timeout to 0 at the start of both upgrades; lock_timeout and the single-transaction behaviour are unchanged. * fix(rpc): commit sim_fundAccount's tx together with its credit fund_account committed the PENDING faucet SEND (value_credited=false) before credit_tx_value_once ran. A consensus worker could claim it in that gap; execute_transfer's unlocked value_credited read then credited the recipient a second time once the endpoint committed (balance 2000 for a 1000 fund, seen in the rc11 bundle's integration run). The race predates the threadpool change, which can widen the window. Insert with commit=False so the row only becomes visible already credited. The worker-side guard in execute_transfer is still a plain read; making it a compare-and-set is left for a follow-up. * ci(genvm): key the precompile cache on host CPU features Precompiled GenVM modules carry the CPU features of the host that built them. The cache key only had OS and arch, so a cache saved on an AVX-512 runner was restored on one without avx512bitalg. GenVM then rejected every module ("compilation setting has_avx512bitalg is enabled, but not available on the host"), recompiled in memory on each run, and the first legacy v0.2 schema call hit GENVM_TIMEOUT at 30s (rc11 bundle run 42737821857, suite 52m vs 33m). The arch-only entrypoint marker still said "already precompiled", so nothing re-precompiled. Add a hash of the /proc/cpuinfo flags to the key so each runner CPU type gets its own cache entry.
… line (#1822) Existing projects (Collective Memory, argue.fun, Rally) keep deploying unchanged code after the v0.123 upgrade. That code declares runners that only the retained v0.2 executor ships (e.g. py-genlayer:1jb45..., py-genlayer:1j12s...), and without a `# v0.2.x` header the manager sent it to the newest line, which ends the deploy with `runner malformed`. Top-level deploys, simulated deploys (fee estimates) and schema reads now select the retained line from the code's runner spec as well as from the header. This reverses the narrowing in #1802, which limited runner inference to children of v0.2 emitters; new deploys of code the newest line can run still get the newest line. The registry parser also reads the v0.3 `{latest, versions}` shape, so the legacy-only set is computed against the real v0.3.0-rc11 registry instead of treating every v0.3 entry as empty. Validated against the real v0.2.17 and v0.3.0-rc11 registries: argue DebateResolution/BridgeSender, Rally XID/Referral/Campaign/Shard/Factory, all 85 Rally runner.json files and CM ContentValidator select v0.2; Studio examples and integration contracts keep the newest line. Not covered: closing v0.2 deploys later needs a per-Studio flag gating this inference, rejecting explicit v0.2 selectors and dropping v0.2 pins on cascade. Runner aliases and unknown hashes still go to the newest line.
|
/run-e2e all |
|
E2E status was updated. Follow the current E2E and merge checks on this PR. Detailed diagnostics are available internally. |
…s ✨ (#1821) - Re-vendor the GenVM host with async GhostFactory address prediction and the corrected node preset, keeping base64 result decoding - Share 1 CREATE2 formula between host prediction and Studio placement; remove the legacy derivation and per-line address selection - Always send the configured create2 inputs, to every host and nested connection - Keep empty-calldata receive, revert unsupported factory calls, and forbid other targets - Pin GenVM v0.6.0-rc13, whose manager requires create2 and runs executor v0.4 on the studio network - Move the salted-deploy test contracts to the v0.4 runners and `Salted`: the v0.3 SDK predicts the legacy child address - Cover prediction, collisions, unconditional request inputs, and nested-run isolation
- py-genlayer: babwp9t3... -> kzybg95x... - py-genlayer-multi: 2f6ynxqm... -> ys9w7shp... - v0.1 and v0.2 pins in the legacy executor selection tests are left as they are
… SDK returns (#1827) * fix(genvm): keep salted children of v0.2 parents at the address their SDK returns #1821 placed every salted child at the GhostFactory CREATE2 address. The v0.2 SDK is immutable, ignores the host's create2 parameters and returns keccak(0x01 || parent || salt || chain id)[:20] to the contract, so a v0.2 factory (Rally's CampaignFactory and CampaignIC) stored an address where Studio no longer deployed anything. A salted child whose emitter runs on the retained v0.2 line is placed at that address again, as on v0.121 and rc.12. Every other line keeps CREATE2. The emitter's line comes from the snapshot pin, then the contract row (for snapshots saved before pins existed), then the deployment's own selection. * chore: drop wheel files committed by mistake * test(genvm): a pinned snapshot decides the line without reading the row
|
/run-e2e all |
|
E2E status was updated. Follow the current E2E and merge checks on this PR. Detailed diagnostics are available internally. |
…t trees (#1828) genvm-manager v0.6-dev (26b2ff4d) dropped lief from post-install. When the ELF interpreter baked into bin/genvm-modules is absent on the host, it now shells out to patchelf and fails if it is not on PATH. The E2E pipeline builds GenVM on the runner and injects it with GENVM_SOURCE_MODE=prebuilt, so the interpreter path points at the runner's build directory and the Studio image build fails at post-install. Older post-install versions bundle lief and ignore the extra package.
|
/run-e2e all |
|
E2E status was updated. Follow the current E2E and merge checks on this PR. Detailed diagnostics are available internally. |
|
/run-e2e all |
|
E2E status was updated. Follow the current E2E and merge checks on this PR. Detailed diagnostics are available internally. |
|
/merge-train |
|
The merge request cannot proceed. Check reviews, required CI and the selected landing cut. Related changes may require action by their owners; internal details are available to the team. |
Standing release-gate PR:
v0.123-dev→v0.123This is a long-lived gate PR, not a normal feature PR. Do not merge ad-hoc.
Purpose
v0.123-devis the active integration branch for the v0.6 fee work. Individual PRs merge intov0.123-devand only need to pass normal repo CI — they are not blocked on the full cross-repo release train being green.v0.123-devis promoted tov0.123. It waits for cross-repo E2E (the release train) to go green before it is merged.v0.123-dev.How to use
v0.123".(Mirrors genlayerlabs/genlayer-consensus#1070.)