Skip to content

kore-host: host programs read and write Kore volumes with Kore's own engine - #31

Merged
micro-perceptron merged 4 commits into
alphafrom
ft/amazing-cray-qf1tpm
Nov 6, 2026
Merged

kore-host: host programs read and write Kore volumes with Kore's own engine#31
micro-perceptron merged 4 commits into
alphafrom
ft/amazing-cray-qf1tpm

Conversation

@SnowCheetos

@SnowCheetos SnowCheetos commented Nov 6, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Host programs (vKore first) need to read and write Kore's DagFS volumes: a machine's disk image, or a Kore disk attached to Linux. This gives them Kore's own procedure and engine instead of a second implementation of the format.

Changes

  • crates/host (kore-host) is the host-side facade. kore_host::volume re-exports the exported kore-vol library (format, put, rm, read and its Mount, the bounds, FORMAT_VERSION, VolError::is_in_use) and adds no behaviour of its own. It is #[forbid(unsafe_code)]. AGENTS.md records its rule: no second locate, recover or commit path, no host-side format writer, no parsing of volume bytes outside the engine. Behaviour changes go to Kore's tools/kore-vol and arrive through tools/sync-native.py.
  • native/ is re-exported byte for byte at Kore main 7074bda28c416ffe34562857ee43cbc883482782. That commit carries MicroPerceptron/kore#1475 (merged by rebase): tools/kore-vol as a library whose mount releases every engine handle it opens. tools/kore-vol is a new owner root. Following its path dependencies adds the filesystem service closure it drives (kore-svc-dagfs, kore-svc-block, kore-svc-device, kore-svc-mm, kore-svc-swap, kore-nvme, kore-svc-extract-worker-c), for 26 packages and 382 exact files. Publishing that closure was an explicit decision: the alternative is a second DagFS implementation. Eleven of main's commits since v0.40.0 (e5e0b96a) reach the closure; docs/provenance.md lists them. The DagFS on-disk format is unchanged (version 5).
  • Advisories. kore-wasi's Wasmtime requirements move to 48.0.8. The root Cargo.lock takes Wasmtime/Cranelift/Pulley 48.0.8 and zstd 0.13.4, zstd-safe 7.4.0 and zstd-sys 2.1.2. That clears nine RUSTSEC vulnerabilities and ten unsoundness notices, which fail the audit on every branch cut from alpha. Nothing else in either lock moves.
  • Makefile runs the exported kore-vol suites in test, formats kore-host, and keeps it out of Miri. README.md, docs/integration.md and docs/provenance.md describe the facade and the export.

Commits:

  • 74524f9: the facade, and the export with its volume closure.
  • 83d8347: re-export at MicroPerceptron/kore#1475's ab6e7b15, where a mount releases every engine handle on every path. Before it, a long-lived reader could exhaust its session's 32 handles.
  • 7033ba6: Wasmtime 48.0.8 and the zstd family, past the advisories.
  • 3aac3b5: re-export at Kore main 7074bda2, once MicroPerceptron/kore#1475 landed.

Verification

CI is green at 3aac3b5: host (including zig fmt, test-foreign and bindings-check), the three targets, Miri, Advisories and Lockfile currency.

docs/provenance.md records the local set. Local runs, with the pinned nightly:

  • 3aac3b5:
    • Tests: the workspace (every feature except test-foreign), the native kore-rt/kore-dagfs suites, the exported kore-vol suites (1 unit, 9 CLI, 6 library) and its loop-device suite as root (4), kore-host's volume tests, and the tool tests.
    • Checks: the no-default-feature check, Clippy and documentation with warnings denied, cargo fmt --check, the three Kore user-target checks for kore-std/kore-ffi/kore-entry, check-alloc-symbols, check-entry, Miri, the owner graph, the native drift check and the local-composition fixture.
    • Bindings: cbindgen regenerates kore.h byte-identical, and the C consumers compile for all three ISAs.
  • 7033ba6: cargo audit --deny warnings exits 0 on both lockfiles; on the previous root lock it reports the 19 findings. Both lockfiles resolve --locked, and kore-wasi's tests pass.

The export has run end to end in vKore at 3aac3b5 (MicroPerceptron/vkore#53, under QEMU 8.2.2 TCG):

  • A production ISO built from Kore 7074bda2, and a production v0.40.0 ISO, each booted to ready. vKore read off the volume a screenshot the guest saved.
  • A disk v0.40.0 formatted was booted by the 7074bda2 ISO, and its volume read back with both builds' captures.
  • A GPT disk on a loop device took an install through its Kore user partition. It was read back through both the whole-disk and partition nodes, with nothing outside the partition touched.

Not verified

Physical hardware.

Notes

🤖 Generated with Claude Code

https://claude.ai/code/session_01ACHC7gzaAcosxJUPe73SWh

claude added 2 commits November 2, 2026 11:18
@claude
host: kore-host, and the native export at Kore v0.40.0 with its volum…
74524f9 
…e closure

Host programs that keep Kore machines (vKore first) need to read and write a
machine's DagFS volume, which no host kernel mounts. They now get it from
libkore, through the filesystem row's own engine rather than a second
implementation of the format:

- native: re-exported at Kore 1b6cb322 (main e5e0b96a, Cargo 0.40.0, plus
  the kore-vol library split). `tools/kore-vol` is a new owner root; the
  exporter's closure adds kore-svc-dagfs, -block, -device, -mm, -swap,
  kore-nvme and kore-svc-extract-worker-c: 26 packages, 381 exact files.
- crates/host: `kore-host`, the facade a host program depends on.
  `kore_host::volume` re-exports the procedure (format, put, rm, read with
  list/read_to/walk, FORMAT_VERSION, bounds) and adds no behavior.
- Makefile: `make test` runs the exported kore-vol suites against this
  distribution's lock; fmt covers kore-host; Miri excludes it (real image
  files under POSIX record locks).
- README, integration, AGENTS and provenance describe the host surface,
  the closure it exports, and the offline/one-owner-per-disk rules.

Verified locally: workspace tests (all features but test-foreign), native
kore-rt/kore-dagfs and kore-vol suites, tool tests, no-default check,
clippy -D warnings, docs -D warnings, Kore user-target checks on x86_64,
aarch64 and riscv64gc, check-entry, Miri, owners, native drift and
check-local composition. Not run locally (Zig 0.16.0 could not execute
here): zig fmt, test-foreign, bindings-check, kore-lua's Kore-target build.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ACHC7gzaAcosxJUPe73SWh
@claude
native: re-export at Kore ab6e7b15, where a mount releases every handle
83d8347 
Kore `ab6e7b15` makes `kore-vol`'s mount close every engine handle it opens
on every path. Before it, reading a missing file kept its parent directory
open, reading a directory kept itself open, a copy its sink stopped kept the
file open, and a failed listing or walk step kept its directory: a
long-lived reader such as a volume browser ran out of its session's 32
handles. The export moves two files in `tools/kore-vol`, nothing else.

Rerun for this export: the native drift check, the tool tests, the exported
`kore-vol` suites (9 CLI, 6 library, 1 unit) and `kore-host`'s tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ACHC7gzaAcosxJUPe73SWh
Copilot AI balanced review requested due to automatic review settings November 6, 2026 00:00
@SnowCheetos SnowCheetos mentioned this pull request Nov 6, 2026
Merged
Copilot AI reviewed Nov 6, 2026

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing. Review the Copilot code review billing settings.

@SnowCheetos Claude

SnowCheetos commented Nov 6, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Two checks were red at 83d8347. Neither failure came from this PR.

Advisories: fixed in this PR by 7033ba6, and green at 3aac3b5. A correction to my first version of this comment, which named only zstd-safe and zstd-sys. The audit's full list for the root Cargo.lock, which alpha shares:

  • wasmtime 48.0.6: RUSTSEC-2026-0378, -0379, -0422, -0423 and -0424.
  • zstd 0.13.3: -0391.
  • zstd-safe 7.3.0: -0398 and -0403, plus ten unsoundness notices (-0395 through -0406, except those two).
  • zstd-sys 2.1.0: -0410.

native/Cargo.lock is clean. Updating the lockfile alone was not enough, as I had first suggested. kore-wasi's Wasmtime requirements had to move to 48.0.8, so that no consumer resolves a vulnerable release. The lock moves the Wasmtime/Cranelift/Pulley family and the zstd family (32 packages) and nothing else. zstd comes only from the host test engine's default cache feature; nothing built for a Kore target links it. The scheduled audit on alpha stays red until this lands there.

copilot-pull-request-reviewer never reviewed 83d8347: its session ended with HTTP 402, "You have exceeded your monthly quota". That is the organization's Copilot review quota. The same error stopped the reviewer on MicroPerceptron/kore#1475 and MicroPerceptron/vkore#53. A re-run won't pass until the quota resets or is raised.

Generated by Claude Code

claude added 2 commits November 6, 2026 06:40
@claude
wasi: wasmtime 48.0.8 and zstd 0.13.4, past October's RUSTSEC advisories
7033ba6 
## Why

The Advisories workflow (`cargo audit --deny warnings`) refuses the root
`Cargo.lock` on every branch cut from `alpha`, including
#31:

- wasmtime 48.0.6: RUSTSEC-2026-0378 (a panic lowering a host-created
  stream into a guest that expects another payload type), -0379 (aarch64
  Cranelift jumps to the wrong target on a large `br_table`), -0422 (typed
  functions with `i31` or `nullref` arguments may panic), -0423
  (cross-engine type confusion with host-created struct types) and -0424
  (unchecked memory use moving borrowed resources between components).
  48.0.8 fixes all five.
- zstd 0.13.3: RUSTSEC-2026-0391 (an abandoned stream's data leaks into the
  next stream on a reused context). zstd-safe 7.3.0: -0398 and -0403, and
  ten unsoundness notices, -0395 through -0406 but those two (use after
  free, double free, out-of-bounds reads, uninitialized memory). zstd-sys
  2.1.0: -0410. zstd 0.13.4, zstd-safe 7.4.0 and zstd-sys 2.1.2 fix them all.

zstd reaches the lock only through the host test engine: kore-wasi's
Wasmtime dev-dependency keeps Wasmtime's default features, whose `cache`
brings `wasmtime-internal-cache` and with it zstd. kore-wasi's own
Wasmtime (`runtime`, `component-model`, `async`) does not, and nothing
built for a Kore target links it.

## What

- `crates/wasi/Cargo.toml`: kore-wasi's requirements on `wasmtime` (the
  target dependency and the host dev-dependency) and on `wasmtime-wasi-io`
  move to `48.0.8`, so no consumer resolves a vulnerable release through
  this crate.
- `Cargo.lock` moves those two families and nothing else: 32 packages,
  versions and checksums only.
  - `wasmtime`, `wasmtime-environ`, `wasmtime-internal-*`,
    `wasmtime-wasi-io` and `pulley-*` 48.0.8; `cranelift-*` 0.135.8.
  - `zstd` 0.13.4, `zstd-safe` 7.4.0, `zstd-sys` 2.1.2+zstd.1.5.7.
- `native/Cargo.lock` holds neither family and is unchanged.

## Verification

- `cargo audit --deny warnings` (advisory database fetched 2026-11-06,
  1409 advisories) exits 0 on `Cargo.lock` and on `native/Cargo.lock`. On
  the previous `Cargo.lock` it reports the nine vulnerabilities and ten
  unsound notices above.
- Lockfile currency: `cargo metadata --locked` resolves both manifests.
- `cargo test --locked -p kore-wasi --all-features`: 10 passed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ACHC7gzaAcosxJUPe73SWh
@claude
native: re-export at Kore main 7074bda2, where MicroPerceptron/kore#1…
3aac3b5 
…475 landed

## Why

The previous export named Kore `ab6e7b15`, a commit on
MicroPerceptron/kore#1475's branch. That pull request is merged, by rebase,
and its branch is gone: its four `tools/kore-vol` commits are on main as
`e39d7308`, `99ef4b86`, `3445d543` and `9d616ce0`. The export moves to main so
that what libkore publishes, and what its consumers pin, is a Kore that main
carries.

## What

- `native/` is re-exported byte for byte at
  `7074bda28c416ffe34562857ee43cbc883482782` with `tools/sync-native.py`.
  `tools/kore-vol` is byte-identical to `ab6e7b15`'s.
- Eleven of the 52 commits main took since `e5e0b96a` reach the closure: 35
  files change and `kore-window` gains `desktop.rs` (26 packages, 382 files).
  DagFS gets a cwd anchor and `MatchView`, paths where `.` adds no step and
  `resolve` folds `..`, and dead views reaped by asking the kernel. Also: a
  launch context may fill the payload it borrows (`PROCESS_IO_ABI` stays 13),
  a mapped view's registry entry is retired once, a DMA intake tells a CPU-view
  refusal from a device-publication one, System Settings in
  `kore-window`/`kore-app`, and two reseals of the services' boot digests.
- No exported manifest changed, so `native/Cargo.lock` did not. The DagFS
  on-disk format stays at version 5.
- `docs/provenance.md` records the export, what reaches the closure and what
  was run.

## Verification

Local, with the pinned nightly, all exit 0:
- `cargo test` for the workspace (every feature but `test-foreign`), the
  native `kore-rt`/`kore-dagfs` suites, the exported `kore-vol` suites (1 unit,
  9 CLI, 6 library) and its device suite as root over loop devices (4),
  `kore-host`'s volume tests (2), and the tool tests.
- The no-default-feature check; Clippy and documentation with warnings denied;
  `cargo fmt --check`.
- The three Kore user-target checks for `kore-std`/`kore-ffi`/`kore-entry`,
  `check-alloc-symbols`, `check-entry`, and Miri.
- The owner graph, the native drift check and the local-composition fixture.
- cbindgen 0.29.4 regenerates `extern/c/include/kore.h` byte-identical, and the
  C consumers compile for all three ISAs.

Not run: `zig fmt`, `test-foreign`, the Zig half of `bindings-check` and
`kore-lua`'s Kore-target build. This environment cannot execute a Zig
toolchain; CI's host job runs them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ACHC7gzaAcosxJUPe73SWh
SnowCheetos pushed a commit to MicroPerceptron/vkore that referenced this pull request Nov 6, 2026
@claude
Pin libkore 3aac3b5: Kore main 7074bda2's engine, verified on its ISO
d8a8848 
## Why

MicroPerceptron/kore#1475 is merged, by rebase: the Kore commits vKore named
(`ab6e7b15` for the engine, `547af218` for the power button) are `9d616ce0` and
`7074bda2` on Kore's main, and MicroPerceptron/libkore#31 now exports its
native tree at `7074bda2` (libkore `3aac3b5`). vKore's one compatibility
identity moves with it, and its docs name commits Kore's main carries.

## What

- `Cargo.toml`: `kore-host` at libkore
  `3aac3b5ce4870421581ffa9b8abc0837c63d2c41`. `Cargo.lock` moves the 16 libkore
  packages and no other package; Cargo re-resolves six Windows-only edges onto
  the `windows-sys` 0.59 already in the lock.
- `docs/compatibility.md`: the identity (libkore `3aac3b5`, Kore `7074bda2`,
  and the `7074bda2` ISO's SHA-256 beside v0.40.0's); the PM1 fix by its main
  commit; what a stop means for a capture.
- A capture and a graceful stop. Kore writes a capture seconds after
  PrintScreen (seven to ten under TCG here), and its shutdown does not wait
  for one still being written. v0.40.0 never hears the power button, so a stop
  gave the capture `--stop-after` plus the 10 s grace; a Kore that hears it
  powers off 0.35 s after the press, and `--stop-after 5` lost the capture.
  The README's round trip now waits 15 s, the compatibility and desktop docs
  say why, and so does `--screenshot`'s long help.
- README: what is verified, against both ISOs. Block devices leave the list of
  what is not: a GPT disk on a loop device has been used through vKore.

## Verification

Linux x86_64 container, QEMU 8.2.2 TCG, with this pin:
- `cargo fmt --check`, `cargo clippy --workspace --all-targets -D warnings`,
  `cargo test --workspace` (117 passed).
- The opt-in boot test (`vkore-vm --test host`) on the `7074bda2` ISO and on
  the v0.40.0 ISO: 3 passed each.
- The `7074bda2` ISO (SHA-256 `b085e919…`), a fresh machine: ready (19/19
  services, 45/45 apps, homes 1/1), PrintScreen, a graceful stop 0.35 s after
  the press. With `--stop-after` 5 or 7 the capture was lost; with 10 or 30 it
  was saved. `shots --get` copies it out with its record (build
  `7074bda28c41`), byte-identical to `volume cat`.
- The v0.40.0 ISO, a fresh machine: ready (19/19, 44/44, 1/1), PrintScreen,
  the stop forced after the 10 s grace, the capture copied out.
- A disk v0.40.0 formatted (and later Kore wrote), booted by the `7074bda2`
  ISO: vKore flags the changed ISO, Kore reaches ready and seeds its new app
  (45/45), and the volume holds both builds' captures.
- A GPT disk on a loop device: a device format refused, an install through the
  partition node, exports through the partition and whole-disk nodes identical
  to the source, nothing outside the user partition changed.
- The desktop under Xvfb: the disk opened on its screenshots, a capture
  previewed with its record and saved byte-identical, and the machine started
  headless, followed to ready and stopped gracefully in 0.3 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ACHC7gzaAcosxJUPe73SWh
@micro-perceptron
micro-perceptron merged commit 1b467cc into alpha Nov 6, 2026
7 checks passed
@micro-perceptron
micro-perceptron deleted the ft/amazing-cray-qf1tpm branch November 6, 2026 19:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@micro-perceptron micro-perceptron micro-perceptron approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@SnowCheetos @micro-perceptron @claude