Runtime-only scope and design for Weave binding extensions

CLAUDE.md

@@ -31,15 +31,17 @@
 
 # Project
 
-Shuttle is a WebAssembly runtime library (Java binding + native Rust:
-Wasmtime + Weave) for Minecraft computer mods. It is not a mod itself.
-OpenComputers will use it through an addon; a new Retro Computing mod
-will include it by default. Current status and open decisions:
+Shuttle is only a persistable Wasm runtime: Wasmtime integrated with Weave,
+plus its Java binding, like JNLua is for Lua in OpenComputers. It is not a
+mod. OpenComputers will use it through an addon; a new Retro Computing mod
+(RC) will include it by default. Anything at the level of "a computer"
+(firmware ABI, devices, ticks, power, BIOS/OS boot, machine save format)
+belongs to those mods, not to Shuttle. Current status and open decisions:
 `native/HANDOFF.md`. Overview and docs: `README.md`, `docs/`.
 
 | Path | What |
 | --- | --- |
-| `native/` | Rust workspace: `shuttle-core`, `shuttle-runner`, C test guests |
+| `native/` | Computer prototype (`shuttle-core`, `shuttle-runner`, C guests); belongs to RC, will move there |
 | `shuttle-java/` | `jp.akkiserver.shuttle`: Wasmtime + Weave JNI binding |
 | `shuttle-java-example/` | Weave example |
 | `shuttle-wit-parser/`, `shuttle-wit-maven-plugin/` | WIT tooling |

README.md

@@ -2,68 +2,59 @@
 
 # Shuttle
 
-> **Status:** early prototype. The native core runs headless only; no mod uses Shuttle yet.
+> **Status:** early prototype. No mod uses Shuttle yet.
 
-Shuttle is a WebAssembly runtime library for Minecraft computer mods. It is not a mod itself: computer mods use it to run the programs on their in-game computers. Two consumers are planned: OpenComputers (OC), through a separate addon mod, and a new Retro Computing mod, which will include Shuttle by default. Neither integration exists yet.
+Shuttle is a persistable WebAssembly runtime for Java: [Wasmtime](https://github.com/bytecodealliance/wasmtime) integrated with [Weave](https://github.com/akkiserver-dev/weave), plus its Java binding. It plays the role for Wasm that JNLua plays for Lua in OpenComputers: it runs guest code and can save it mid-execution, and nothing more. It is not a mod itself.
 
-On behalf of the host mod, Shuttle runs a stack of Wasm guests:
-
-```
-firmware          implemented by Shuttle's native core
-  └─ BIOS            player-made Wasm (for example stored on an EEPROM item)
-       └─ OS kernel  player-made Wasm, started by the BIOS with `exec`
-            └─ child programs  started by the kernel with `co_create`
-```
-
-The firmware talks to guests only through the **firmware ABI** (`fw.*`, `co.*`, `sys.call`). Device calls look like OpenComputers method calls, either synchronous (`dev_call`) or asynchronous (`dev_submit` + a completion event). The ABI is specified in a separate draft document (draft 0.1, not published in this repository); [`native/README.md`](native/README.md) lists which parts of it are implemented.
+Computer mods build on it. Two consumers are planned: OpenComputers (OC), through a separate addon mod, and a new Retro Computing mod (RC), which will include Shuttle by default. Everything at the level of "a computer" (firmware, BIOS/OS boot, devices, ticks, the save format of a whole machine) belongs to those mods, not to Shuttle.
 
 ## How it works
 
-- **Runtime:** [Wasmtime](https://github.com/bytecodealliance/wasmtime), made checkpointable with [Weave](https://github.com/akkiserver-dev/weave) (branch `shuttle-host-hooks`). Weave transforms a Wasm module so that a running program can be unwound into its linear memory and rebuilt later. This lets a host mod save a computer with the world and continue it mid-execution after a reload.
-- **Native core (Rust, `native/`):** Wasmtime, Weave, the firmware ABI, the `co_*` machinery, memory limits and snapshot capture. Today it is driven by the headless `shuttle-runner`.
-- **Java binding (`shuttle-java`):** Java API over Wasmtime and Weave through JNI, for the host mod to call. A high-level Machine API on top of the native core is planned but not written yet.
+- **Persistence:** Weave transforms a Wasm module so that a running program can be unwound into its linear memory and rebuilt later. A host mod can therefore save a running guest with the world and continue it after a reload.
+- **Java binding (`shuttle-java`):** Wasmtime and Weave through JNI: load a module, run it for a bounded number of polls, define host functions, take and restore snapshots. Planned extensions (yielding imports, memory limits, nested instances) are described in [docs/design/runtime-extensions.md](docs/design/runtime-extensions.md).
 - **Targets:** Windows, macOS and Linux, each on x86_64 and arm64.
 
 ## Repository layout
 
 | Path | What |
 | --- | --- |
-| [`native/`](native) | Shuttle native core (Rust): `shuttle-core`, the headless `shuttle-runner`, and C test guests in `native/guests`. See [`native/README.md`](native/README.md). |
 | [`shuttle-java/`](shuttle-java) | Java bindings for Wasmtime plus the Weave (checkpointable Wasm) JNI layer. See [docs/java-binding.md](docs/java-binding.md). |
 | [`shuttle-java-example/`](shuttle-java-example) | Runnable Weave example: run a guest a few "ticks" at a time, snapshot it, restore it and continue. |
 | [`shuttle-wit-parser/`](shuttle-wit-parser), [`shuttle-wit-maven-plugin/`](shuttle-wit-maven-plugin) | Build-time WIT tooling: generate Java component-context interfaces from WIT files. See [docs/wit-codegen.md](docs/wit-codegen.md). |
+| [`native/`](native) | A prototype of a whole computer (firmware ABI, BIOS → OS → child programs) on Wasmtime + Weave, in Rust. It belongs to the RC mod and will move there; it is kept here for now as a reference. See [`native/README.md`](native/README.md). |
 | [`docs/`](docs) | Documentation. Start at [docs/README.md](docs/README.md). |
 
 The Java modules form one Maven reactor: groupId `jp.akkiserver`, parent artifactId `shuttle`, one artifact per directory (`shuttle-java`, `shuttle-java-example`, `shuttle-wit-parser`, `shuttle-wit-maven-plugin`). The version is set in the root [`pom.xml`](pom.xml).
 
 ## Quick start
 
-Native core (requires Rust, plus clang and wasm-ld for the C guests):
+Java bindings (requires JDK 21, Maven and Rust; Maven picks the native build for your OS and CPU automatically):
 
-```powershell
-.\native\scripts\build.ps1          # Windows: build guests + workspace, then verify
-```
 ```sh
-./native/scripts/build.sh           # Linux / WSL2
+mvn -pl shuttle-java,shuttle-java-example -am install
+mvn -pl shuttle-java-example exec:java
 ```
 
-`verify` boots BIOS → OS → child, then repeats the run with capture → restore into a fresh machine at every tick boundary and checks that the logs are identical.
+The second command runs the Weave example. Platform profiles, CI and known problems are covered in [docs/building.md](docs/building.md).
 
-Java bindings (requires JDK 21, Maven and Rust; Maven picks the native build for your OS and CPU automatically):
+The `native/` prototype (requires Rust, plus clang and wasm-ld for the C guests):
 
+```powershell
+.\native\scripts\build.ps1          # Windows: build guests + workspace, then verify
+```
 ```sh
-mvn -pl shuttle-java,shuttle-java-example -am install
-mvn -pl shuttle-java-example exec:java
+./native/scripts/build.sh           # Linux / WSL2
 ```
 
-The second command runs the Weave example. Platform profiles, CI and known problems are covered in [docs/building.md](docs/building.md).
+`verify` boots BIOS → OS → child, then repeats the run with capture → restore into a fresh machine at every tick boundary and checks that the logs are identical.
 
 ## Documentation
 
-- [docs/building.md](docs/building.md): building the native core and the Maven modules, CI, requirements, known issues
+- [docs/building.md](docs/building.md): building the Maven modules and the native/ prototype, CI, requirements, known issues
 - [docs/java-binding.md](docs/java-binding.md): the `shuttle-java` API (Wasmtime and Weave) and its architecture
+- [docs/design/runtime-extensions.md](docs/design/runtime-extensions.md): planned runtime extensions (design)
 - [docs/wit-codegen.md](docs/wit-codegen.md): generating Java component contexts from WIT
-- [native/README.md](native/README.md): the native core prototype, its results and its ABI coverage
+- [native/README.md](native/README.md): the computer prototype (moving to the RC mod), its results and its ABI coverage
 
 ## Credits and license
 

docs/README.md

@@ -2,9 +2,10 @@
 
 | Document | Contents |
 | --- | --- |
-| [building.md](building.md) | Requirements; building the native core (`native/scripts/build.ps1`, `build.sh`, what `verify` checks); building the Maven modules and their profiles; CI workflows; known issues. |
+| [building.md](building.md) | Requirements; building the Maven modules and their platform profiles; building the `native/` prototype (`native/scripts/build.ps1`, `build.sh`, what `verify` checks); CI; known issues. |
 | [java-binding.md](java-binding.md) | The `shuttle-java` module: Wasmtime API (engine, module, store, linker, instance, memory, functions), value types, Component Model support, the Weave JNI layer for checkpointable guests, and the architecture (Java API / JNI layer in Rust / Wasmtime). |
 | [wit-codegen.md](wit-codegen.md) | `shuttle-wit-parser` and `shuttle-wit-maven-plugin`: generating Java component-context interfaces from WIT files. |
-| [../native/README.md](../native/README.md) | The native core prototype (`shuttle-core`, `shuttle-runner`, C guests): layout, build, test results, and which parts of the firmware ABI draft are implemented. |
+| [design/runtime-extensions.md](design/runtime-extensions.md) | Design (not implemented): yielding imports, memory limits and nested-instance primitives for the Weave binding. |
+| [../native/README.md](../native/README.md) | The computer prototype (`shuttle-core`, `shuttle-runner`, C guests), which will move to the Retro Computing mod: layout, build, test results, firmware ABI coverage. |
 
 Back to the [project README](../README.md).

docs/design/runtime-extensions.md

@@ -0,0 +1,171 @@
+# Design: runtime extensions for the Weave binding
+
+Status: **design, not implemented** (2026-10-15). API names marked *(proposed)* need
+the owner's approval before implementation; names marked *(approved)* were approved
+on 2026-10-17.
+
+## Scope
+
+Shuttle provides a Weave-integrated, persistable Wasmtime runtime and its Java
+binding, in the same role that JNLua plays for OpenComputers. Everything at the level
+of "a computer" (firmware ABI, devices, ticks, power, BIOS/OS boot, the SHTL save
+format) belongs to the host mod; the prototype in `native/shuttle-core` is going to
+move to the Retro Computing mod.
+
+Two features of that prototype are generic runtime features and belong in Shuttle's
+Java binding (`shuttle-java`, package `jp.akkiserver.shuttle`):
+
+1. **Weave extensions**: yielding imports, unwinding from inside a host call, and
+   per-instance memory limits.
+2. **Nested instances**: the primitives that a host needs to run one guest from
+   inside another guest's host call, with a shared execution budget (what
+   `native/shuttle-core` uses for its `co_*` imports).
+
+The existing Weave layer (`WeaveModule`, `WeaveInstance`, `WeaveCaller`,
+`shuttle-java/src/main/rust/wasmtime-binding/src/weave.rs`) is extended; nothing is
+removed.
+
+## 1. Weave version
+
+`shuttle-java` pins Weave `0f26893`, which lacks all three extensions. They exist in
+`677cce6` (branch `shuttle-host-hooks` of `akkiserver-dev/weave`), which `native/`
+already uses:
+
+| Weave change | API |
+| --- | --- |
+| `0286544` yielding imports, per-instance memory limits | `TransformOptions::yielding_imports`, `weave_wasmtime::request_unwind`, `WeaveInstance::set_memory_limit`, `Ctx::set_memory_limit` |
+| `677cce6` change the limit from inside a host call | `Ctx::set_memory_limit` usable in host functions |
+
+The Weave-side names in the table are kept as they are (approved). `Cargo.toml` pins
+Weave by commit (`rev`), so the branch name `shuttle-host-hooks` does not matter to
+Shuttle and stays.
+
+Changes:
+
+- `wasmtime-binding/Cargo.toml`: all four Weave crates to rev
+  `677cce64cf9728485849af21f7bf36f801175d4f`; wasmtime to 46.0.3 (same as `native/`).
+- `weave.rs` (`module_create`): the `TransformOptions` literal gains
+  `..Default::default()`, otherwise it stops compiling once the new field exists.
+
+## 2. Yielding imports
+
+**Problem.** `WeaveCaller.requestUnwind()` sets the poller to `UnwindNext`: the host
+call returns normally, and the guest stops at its next instrumented poll. A host
+function cannot stop the guest *at the call* and produce its result later (for
+example, a disk read that completes on the server thread).
+
+**Weave's mechanism.** A module transformed with
+`TransformOptions::yielding_imports = [(module, name), ...]` treats calls to those
+imports as unwind points. Inside such a host call,
+`weave_wasmtime::request_unwind(caller)` makes the guest unwind right at the call
+site. On `resume`, the guest calls the **same import again** with the same
+arguments, and that second call's return value is used.
+
+**Java API** *(approved)*:
+
+```java
+WeaveModule module = WeaveModule.load(engine, wasm,
+        WeaveModule.Options.builder()
+                .pollPeriod(64)
+                .stackPages(4)
+                .yieldingImport("dev", "read")      // may be repeated
+                .build());
+
+builder.define("dev", "read", "ii>i", (caller, args) -> {
+    Result r = pending.get(key(args));
+    if (r == null) {                    // first call: start the work, stop the guest
+        submitToServerThread(args);
+        caller.suspend();
+        return new long[] {0};          // ignored: the guest unwinds
+    }
+    return new long[] {r.value()};      // re-call after resume: the real result
+});
+```
+
+- `WeaveModule.Options` (builder) with `pollPeriod`, `stackPages`, `yieldingImport(module,
+  name)`. The existing `load(engine, wasm)` and `load(engine, wasm, pollPeriod,
+  stackPages)` stay and map to it.
+- `WeaveCaller.suspend()` calls `request_unwind`. `call`/`resume` then return
+  `isUnwound()`.
+- Calling `suspend()` from an import that is not declared as yielding throws. Weave
+  does not check this itself: `request_unwind` only refuses calls during module
+  initialization, and otherwise just sets the unwind state. The binding therefore
+  records, per defined host function, whether its import is in the module's
+  yielding list, and `suspend()` checks that flag.
+- Because the import is called again after `resume`, and possibly after a
+  `snapshot` + `restore` in a later session, the host keeps the request state on its
+  side and makes the second call idempotent.
+- `requestUnwind()` keeps its current meaning (stop at the next poll).
+
+## 3. Memory limits
+
+Without a limit, a guest can grow its memory until the host runs out.
+
+**Java API** *(approved)*:
+
+| Method | Maps to | Notes |
+| --- | --- | --- |
+| `WeaveInstance.setMemoryLimit(long bytes)` | `WeaveInstance::set_memory_limit(Some(n))`, or `None` for a negative value | any negative value (conventionally `-1`) means unlimited; there is no separate `clear` method |
+| `WeaveInstance.memorySize()` | `mem_view().size(0)` | current size of memory 0, for accounting |
+| `WeaveCaller.setMemoryLimit(long bytes)` | `Ctx::set_memory_limit` (same rule) | inside a host call, e.g. to shrink a parent's limit after creating a child |
+
+- A `memory.grow` past the limit returns -1 to the guest; it does not trap.
+- The limit is not part of a Weave snapshot. After `restore`, the host sets it again.
+
+## 4. Nested instances
+
+`native/shuttle-core` runs a child program from inside the kernel's `co_resume` host
+call (`native/crates/shuttle-core/src/imports.rs`, `instantiate_child` and
+`co_resume`). The runtime part of that is small:
+
+1. **Re-entrancy.** A host function may `call` or `resume` another `WeaveInstance`
+   on the same thread. Each instance has its own Store, so this already works in
+   Rust. The Java binding has to allow it (no global lock, the outer `WeaveCaller`
+   stays valid) and a test has to cover it.
+2. **Shared execution budget.** The child's polls come out of the parent's
+   remaining budget. The host needs:
+   - `WeaveCaller.pollBudget()` / `WeaveCaller.setPollBudget(long)` *(approved)*: the
+     parent's `Poller::UnwindAfter(n)`, read and written inside the host call.
+   - `WeaveInstance.pollBudget()` *(approved)*: after `resume`, the child's leftover
+     budget (`take_poller`), so the host can compute how many polls it used.
+   - The existing `unwindAfter(n)` keeps its name.
+
+   The host gives the child `min(requested, parent remaining)` with
+   `unwindAfter(...)`, resumes it, and subtracts the polls the child used from the
+   parent:
+
+   ```java
+   long remaining = caller.pollBudget();
+   long given = Math.min(requested, remaining);
+   child.unwindAfter(given);
+   WeaveResult r = child.resume();
+   long used = given - child.pollBudget();
+   caller.setPollBudget(remaining - used);
+   ```
+
+3. **Snapshots** stay per instance: `snapshot()` on the parent and on each child, plus
+   `WeaveModule.hash()` to identify the module. Putting them into one save file is
+   the host's job.
+
+Not part of Shuttle: the `co_*` import names and their ABI, child handles, the
+call/yield record format, and the policy for how much memory a kernel may use.
+
+## Out of scope for now
+
+- Worker threads and stack sizes (`max_wasm_stack`), epoch-based time limits.
+- A versioned envelope around snapshots (today the bytes are gzip of Weave's encoding).
+- Moving `native/` to the Retro Computing mod repository.
+
+## Tests for the implementation
+
+Extend `shuttle-java/src/test/java/jp/akkiserver/shuttle/WeaveTest.java`:
+
+- Yielding import: `suspend()` stops the guest at the call; after `resume()` the
+  import is called again and its value is used; the same across `snapshot()` +
+  `restore()`; `suspend()` from a non-yielding import throws.
+- Memory limit: `memory.grow` past the limit returns -1 and the guest keeps running;
+  the limit set from inside a host call takes effect; `memorySize()` matches.
+- Nested instances: a host function resumes a child; the child's polls are taken
+  from the parent's budget; parent and child snapshots restore and continue.
+
+Run `mvn test` on Windows and on WSL2.

docs/java-binding.md

@@ -281,7 +281,9 @@ Rules that follow from the API:
 
 ### Relation to `native/`
 
-The Weave layer in `shuttle-java` and the native core in `native/` are separate. `shuttle-java` pins Weave commit `0f26893`, which does not have the `shuttle-host-hooks` additions (yielding imports, memory limits). `native/` pins `677cce6` on branch `shuttle-host-hooks`. How the Java layer will connect to `shuttle-core` (a Machine API over JNI) has not been designed yet.
+`native/` is a prototype of a whole computer (firmware ABI, devices, BIOS → OS → child programs) built directly on Weave in Rust. That level belongs to the host mod, so `native/` will move to the Retro Computing mod and is not connected to `shuttle-java`.
+
+`shuttle-java` pins Weave commit `0f26893`, which does not have the `shuttle-host-hooks` additions (yielding imports, memory limits) that `native/` uses (`677cce6`). Bringing those, plus the primitives for nested instances, into this binding is designed in [design/runtime-extensions.md](design/runtime-extensions.md).
 
 ## Architecture