Repository navigation
Build the guest binaries from nvx-rs - #561
Open
Pedro Henrique Penna (ppenna) wants to merge 6 commits into
Open
Pedro Henrique Penna (ppenna) wants to merge 6 commits into
Pedro Henrique Penna (ppenna) wants to merge 6 commits into
Conversation
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
The privileged runtime and packaging migration needs human review and exact-head CI confirmation, with a cross-platform provenance issue still unresolved.
1 open finding
What changed in this PR
Completes the guest-build migration for #358, using nvx-rs binaries across Alpine, Ubuntu, and Azure Linux while preserving existing executable paths.
Changes:
- Integrates pinned Rust guest builds and static-executable validation.
- Extends artifact provenance, release metadata, caches, and setup for Rust inputs.
- Replaces C implementations and updates tests and documentation.
| File | Description |
|---|---|
| THIRD_PARTY_NOTICES.md | Adds Rust runtime and dependency notices. |
| SOURCE-MANIFEST.json | Records Rust toolchain and guest sources. |
| scripts/test_tool_versions.py | Checks musl target setup. |
| scripts/setup/setup-linux-runner.sh | Explicitly selects the target toolchain. |
| scripts/setup/setup-linux-mshv.sh | Installs and verifies the musl target. |
| scripts/nvx.py | Exposes guest-build, test, and ELF commands. |
| scripts/nvx_tools/ubuntu.py | Includes Rust customization inputs. |
| scripts/nvx_tools/release.py | Validates Rust source metadata. |
| scripts/nvx_tools/elf.py | Checks static ELF linkage. |
| scripts/nvx_tools/ci.py | Tracks Rust guest build dependencies. |
| scripts/nvx_tools/build.py | Installs Rust binaries and records provenance. |
| scripts/nvx_tools/benchmark.py | Records Rust device-I/O provenance. |
| scripts/nvx_tools/azurelinux.py | Includes Rust build inputs. |
| scripts/nvx_tools/archive.py | Excludes generated source-archive content. |
| README.md | Describes shared Rust guest binaries. |
| nvx-rs/src/protocol/outer.rs | Adds guest records and flow-control codecs. |
| nvx-rs/src/protocol/mod.rs | Exports shared guest protocol APIs. |
| nvx-rs/src/protocol/maps.rs | Adds byte-oriented guest mapping decoding. |
| nvx-rs/src/protocol/features.rs | Defines agent feature sets. |
| nvx-rs/src/protocol/boot.rs | Parses mapping-count boot parameters. |
| nvx-rs/src/net/cidr.rs | Documents existing internals. |
| nvx-rs/src/lib.rs | Exposes the Linux guest module. |
| nvx-rs/src/host/session.rs | Documents existing session internals. |
| nvx-rs/src/host/process.rs | Documents existing process internals. |
| nvx-rs/src/host/platform/windows.rs | Documents existing Windows primitives. |
| nvx-rs/src/host/platform/mod.rs | Documents platform modules. |
| nvx-rs/src/host/platform/linux.rs | Documents existing Linux primitives. |
| nvx-rs/src/host/instance.rs | Documents existing instance internals. |
| nvx-rs/src/host/filesystem.rs | Documents existing filesystem internals. |
| nvx-rs/src/host/artifacts.rs | Documents existing artifact helpers. |
| nvx-rs/src/guest/time/mod.rs | Adds time-tool dispatch. |
| nvx-rs/src/guest/time/events.rs | Formats bounded time-ABI events. |
| nvx-rs/src/guest/time_probe/mod.rs | Adds diagnostic command dispatch. |
| nvx-rs/src/guest/time_probe/format.rs | Preserves libc numeric formatting. |
| nvx-rs/src/guest/sys/x86.rs | Wraps x86 instructions. |
| nvx-rs/src/guest/sys/terminal.rs | Wraps terminal configuration and draining. |
| nvx-rs/src/guest/sys/stdio.rs | Wraps libc formatting and scanning. |
| nvx-rs/src/guest/sys/scheduler.rs | Wraps affinity and scheduling operations. |
| nvx-rs/src/guest/sys/port.rs | Provides time and shutdown port access. |
| nvx-rs/src/guest/sys/poll.rs | Wraps descriptor polling. |
| nvx-rs/src/guest/sys/msr.rs | Provides owned MSR-device access. |
| nvx-rs/src/guest/sys/mapping.rs | Owns mappings and device writes. |
| nvx-rs/src/guest/sys/kernel.rs | Wraps kernel-log and auxiliary-vector queries. |
| nvx-rs/src/guest/sys/ioctl.rs | Wraps console and entropy ioctls. |
| nvx-rs/src/guest/sys/filesystem.rs | Wraps filesystem and mount operations. |
| nvx-rs/src/guest/sys/errno.rs | Provides typed errno and libc diagnostics. |
| nvx-rs/src/guest/parse.rs | Preserves libc integer parsing. |
| nvx-rs/src/guest/mod.rs | Organizes guest tools and unsafe boundary. |
| nvx-rs/src/guest/ingress/test_mode.rs | Processes ingress snapshot fixtures. |
| nvx-rs/src/guest/ingress/mod.rs | Adds ingress entry and protocol definitions. |
| nvx-rs/src/guest/ingress/encode.rs | Encodes bounded listener announcements. |
| nvx-rs/src/guest/agent/tty.rs | Handles agent terminal setup. |
| nvx-rs/src/guest/agent/diagnostics.rs | Provides agent diagnostics and I/O helpers. |
| nvx-rs/src/error.rs | Documents existing error fields. |
| nvx-rs/src/bin/nvx-time.rs | Adds time-tool entry point. |
| nvx-rs/src/bin/nvx-time-probe.rs | Adds time-probe entry point. |
| nvx-rs/src/bin/nvx-reseed.rs | Adds reseed entry point. |
| nvx-rs/src/bin/nvx-port-io.rs | Adds port-I/O entry point. |
| nvx-rs/src/bin/nvx-mmio-write.rs | Adds MMIO-write entry point. |
| nvx-rs/src/bin/nvx-managed-agent.rs | Initializes the managed agent. |
| nvx-rs/src/bin/nvx-ingress.rs | Adds ingress entry point. |
| nvx-rs/src/bin/nvx-exit.rs | Selects freestanding or hosted exit runtime. |
| nvx-rs/src/bin/nvx-device-io.rs | Selects device-I/O runtime. |
| nvx-rs/src/bin/nvx-console-pending.rs | Adds console-query entry point. |
| nvx-rs/freestanding/src/runtime/syscall.rs | Implements raw Linux syscall boundary. |
| nvx-rs/freestanding/src/runtime/mod.rs | Organizes freestanding runtime modules. |
| nvx-rs/freestanding/src/runtime/hosted.rs | Provides hosted execution adapters. |
| nvx-rs/freestanding/src/runtime/entry.rs | Provides freestanding startup and buffers. |
| nvx-rs/freestanding/src/runtime/args.rs | Wraps initial-stack arguments. |
| nvx-rs/freestanding/src/lib.rs | Defines the no-std runtime crate. |
| nvx-rs/freestanding/src/exit.rs | Implements VM exit-code handling. |
| nvx-rs/freestanding/Cargo.toml | Configures the freestanding crate. |
| nvx-rs/Cargo.toml | Adds guest binaries, dependencies, and profile. |
| nvx-rs/Cargo.lock | Locks guest dependencies. |
| nvx-rs/build.rs | Documents existing bundle-build internals. |
| guest/common/nvx-reseed.c | Removes the C reseed implementation. |
| guest/common/nvx-mmio-write.c | Removes the C MMIO implementation. |
| guest/common/nvx-exit.c | Removes the C exit implementation. |
| guest/common/nvx-console-pending.c | Removes the C console implementation. |
| doc/usage.md | Documents new CLI commands. |
| doc/setup.md | Documents Rust guest prerequisites. |
| doc/project-structure.md | Describes the Rust guest layout. |
| doc/guests/ubuntu-guest.md | Documents Rust binary installation. |
| doc/guests/azurelinux-guest.md | Documents Rust binaries and provenance. |
| doc/contribute.md | Updates guest testing guidance. |
| doc/ci.md | Describes Rust checks and cache inputs. |
| doc/build.md | Documents Rust builds and packaging. |
| doc/benchmarks.md | Documents Rust benchmark provenance. |
| .github/workflows/copilot-setup-steps.yml | Triggers setup validation on Rust changes. |
| .github/copilot-instructions.md | Updates setup documentation. |
| .github/actions/validate-nvx/action.yml | Replaces retired C-driven test suites. |
| .github/actions/check-nvx-rs/action.yml | Adds workspace and musl validation. |
| .github/actions/build-guest-artifacts/action.yml | Updates guest cache identities. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Pedro Henrique Penna (ppenna)
force-pushed
the
agents/fix-issue-358-build-switch
branch
from
October 15, 2026 08:31
4185f68 to
b5ff179
Compare
Pedro Henrique Penna (ppenna)
force-pushed
the
agents/fix-issue-358-build-switch
branch
from
October 15, 2026 08:41
b5ff179 to
e24b046
Compare
Pedro Henrique Penna (ppenna)
force-pushed
the
agents/fix-issue-358-build-switch
branch
from
October 15, 2026 08:54
e24b046 to
cc31fe2
Compare
Pedro Henrique Penna (ppenna)
force-pushed
the
agents/fix-issue-358-build-switch
branch
from
October 16, 2026 20:58
cc31fe2 to
9a90173
Compare
Pedro Henrique Penna (ppenna)
force-pushed
the
agents/fix-issue-358-build-switch
branch
from
October 16, 2026 21:22
9a90173 to
196c29f
Compare
Pedro Henrique Penna (ppenna)
force-pushed
the
agents/fix-issue-358-build-switch
branch
from
October 16, 2026 21:45
196c29f to
4acf331
Compare
Replace native C helper compilation with the shared Rust guest-binary builder introduced by #528. Build one checked set of all nine executables per initramfs, install it at the established /sbin/nvx-* paths, and record the aggregate source and individual binary digests. Remove the obsolete C compiler flags, helper inventories, and build functions. Keep C test-only flags local until the C-driven suites are deleted later in this stack. Part of #358. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Build one checked set of Rust guest executables in the digest-pinned Rust 1.95.0 image and reuse its exact bytes in Alpine, Ubuntu, and Azure Linux through a guest-only base, so the kernel and Linux source targets never compile them. Keep the toolchain available only for Ubuntu's two fresh determinism attempts, and remove the guest C compilers from the base. Bind reusable binary sets to normalized Rust-source, toolchain, target, profile, and executable digests, and refuse the output of an earlier build that a rebuild would upgrade in place. Record all nine helpers in the Azure Linux package manifest. Part of #358. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Cover the nvx-rs guest inputs in every image identity, package the crate without its target directory, and declare the exact Rust toolchain and musl target in SOURCE-MANIFEST.json. Verify those pins and inputs from nvx.py verify and release packaging. Make device-I/O benchmark provenance validate the shared Rust source digest, and bind the Azure Linux cache input to the newly tracked files. Part of #358. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Hash the complete nvx-rs tree except target, the exact toolchain pin, and the shared build modules into every guest cache, and advance the guest cache schemas. Because hashFiles hashes contents only, also append the path-aware Rust source digest that the guest package manifests record, which the new guest-binary-source-identity command prints, so a module move that keeps every file's bytes still rebuilds the images. Treat nvx-rs changes as OpenVMM VMM-test inputs so they rebuild and boot the affected images. Keep Copilot setup and both Linux setup paths on the pinned musl target, and revalidate Copilot setup for every nvx-rs change. Part of #358. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Remove the nine remaining guest C implementations and the C-compiled Python suites whose behavior the nvx-rs unit, guest-binary, loopback, and real-VM tests now cover. Drop the deleted modules from both CLI validation legs. Add a repository regression that rejects C, header, or assembler sources outside the OpenVMM submodule. Part of #358. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Document the one pinned build shared by native builds, Docker, and CI, the nine binaries shipped by every guest, their provenance, cache inputs, test ownership, and release source. Now that the Rust definitions are authoritative, replace the obsolete source comparisons with literal wire values and remove C-era wording. Extend the third-party notices for the Rust standard libraries, compiler_builtins and its compiler-rt-derived routines, libc, nix, getrandom, and their linked transitive crates, and musl's complete guest-binary scope. Closes #358. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pedro Henrique Penna (ppenna)
force-pushed
the
agents/fix-issue-358-build-switch
branch
from
October 16, 2026 22:19
4acf331 to
f1b8428
Compare
This was referenced Oct 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #358 (PR 5 of 5).
#528, #531, #533, and #550 have merged, so the six commits here are the whole change. They are rebased onto
devate67a06f40, which includes #583.Summary
nvx.py build-guest-binariesroutine from Port the small guest tools to Rust #528./sbin/nvx-*paths, through a guest-only Docker base that the kernel targets do not inherit.SOURCE-MANIFEST.jsonto the complete Rust source/toolchain identity. The guest cache keys end with its path-aware digest, whichnvx.py guest-binary-source-identityprints.openvmm/, and update the current documentation.libc,nix,getrandom,bitflags,cfg-if, andmemoffset) inTHIRD_PARTY_NOTICES.md.The issue originally named six programs, one of which,
nvx-port-io, #574 has since retired. Every remaining compiledguest/commonprogram switches together: the other five plusnvx-ingress,nvx-time,nvx-time-probe, andnvx-exit, nine in all.Rebase onto the merged #550
The rebase adapted the stack to what landed underneath it:
nvx-port-io, so the build, Docker stage, manifests, notices, and documentation cover nine binaries.cargo buildfor every binary and has nonvx-rs/freestandingcrate. The source identity coversnvx-rs/src, the crate manifest, lockfile, and build script,rust-toolchain.toml, and the two build modules, and the documentation no longer describes a separate freestanding link.GuestRustBuildConstants.BINARIES_VARIABLE(NVX_GUEST_BINARIES) rather than a second constant.build-guest-binariesnow upgrades an earlier build's output in place, soinspect_guest_binariesrefuses an output that lacks an added executable or still holds the retirednvx-port-io; a new test covers both cases.dev's.dockerignoreinput and kernel patch identity alongside this PR's Rust inputs and v2 schemas, and validate-nvx keepstest_markdown_links.py.THIRD_PARTY_NOTICES.mdaddsgetrandom0.4.3, which theguestfeature links.Latest rebase and review follow-ups
devate67a06f40: openvmm: answer guest UDP to host loopback from the gateway #583, which moves OpenVMM fromc6d97c529to5ab8a8462and extends the microVM tests, plus a performance-baseline update. The rebase applied without conflicts.APP_STOP,APP_CANCEL, andAPP_STOPPED, as nvx-rs: replace the guest C programs with Rust binaries #358 asks.guest-baseDocker stage carries the Rust sources and binaries, so the kernel, debug kernel, and Linux source targets no longer compile them.Acceptance criteria
openvmm/.git ls-filesreports zero.c,.h,.S, or.sfiles outside the submodule, andBuildConstantsTests.test_no_c_or_assembler_sources_remain_outside_openvmmenforces it.9a901738b, the Docker build installed one digest-checked set from the pinned builder; all three package manifests record the same source digest and the same nine binary digests, and none listsnvx-port-io.verify-guest-determinismpasses.cc31fe2e0, theubuntu-determinismDocker target rebuilt both attempts from fresh Cargo targets and passed. The exact-head CI run rebuilt this head's Ubuntu artifacts without a cache hit and passed it again.9a901738b, each of prometheus32 (KVM), prometheus30 (MSHV), and prometheus28 (WHP) passed 25 microVM scenario runs across Alpine, Ubuntu, and Azure Linux, including time-ABI conformance with the exhaustive check, snapshot and restore, managed exec and lifecycle, sandbox lifecycle, and ingress, plus all 18test-aci-edge-sandboxescases.cc31fe2e0, before the rebase; see below.4acf331ad, before the Docker stage split and the baseline-only rebase.Size comparison
Measured at
cc31fe2e0, with ten binaries before the rebase, and not re-measured since. Fresh Docker artifacts use the same guest inputs. Positive compressed deltas are under 0.9%; every unpacked initramfs is 228,352 bytes smaller.Cold boot
Measured at
cc31fe2e0, before the rebase, and not re-measured since. prometheus32 used one exact kernel and OpenVMM binary, zero warmups, and 20 retained runs per build. Runs were interleaved in pairs with the first build alternating.dev2735f2ace)cc31fe2e0)For context, #550 measured
nvx-time's own baremetal phase p50s (µs elapsed):Boot is about 0.3 ms slower; restore is mixed.
Validation
The hardware rows are from
9a901738b, before the latest rebase (OpenVMM5ab8a8462) and the review follow-ups. Those follow-ups touch only Rust tests and a module comment, the guest cache keys and the command that prints their Rust identity, the Docker stage layout, and documentation; the exact-head CI run covers the current head on KVM, MSHV, and WHP. At9a901738b, every host built OpenVMM from the pinnedc6d97c529, and prometheus32 built all guests with Docker from a fresh BuildKit cache; the other hosts checked the copied artifacts by SHA-256.f1b84285c: Ruff, Linux and Windows Pyright, 1110 validate-nvx tests (34 expected skips) including the Markdown link tests, the link checker, 70 adversarial tests (6 expected skips), and host-profile tests. With the same Rust sources:nvx-rsandaci_edge_sandboxesformatting, Clippy in both feature modes, tests, documentation, agent bundling, and the 1.89 MSRV checks.aci_edge_sandboxes), documentation, and 1.89 MSRV checks;nvx-rsbundling;check-guest-binaries,build-guest-binaries, andtest-guest-binaries; 1011 validate-nvx tests and Specula. Two fresh native guest-binary builds were byte-identical; their bytes differ from the Docker builder's, with the same source digest, so the guests' identical bytes come from the one Docker builder.The previous head's validation, including the Docker Desktop byte comparison, WSL checks, and device-I/O smoke, is in this description's edit history.
Merge order
#528, #531, #533, and #550 have merged; this PR completes the stack.