diff --git a/CLAUDE.md b/CLAUDE.md index 4021ab4..7ccfb0b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -31,15 +31,17 @@ # Project -Shuttle is a WebAssembly runtime library (Java binding + native Rust: -Wasmtime + Weave) for Minecraft computer mods. It is not a mod itself. -OpenComputers will use it through an addon; a new Retro Computing mod -will include it by default. Current status and open decisions: +Shuttle is only a persistable Wasm runtime: Wasmtime integrated with Weave, +plus its Java binding, like JNLua is for Lua in OpenComputers. It is not a +mod. OpenComputers will use it through an addon; a new Retro Computing mod +(RC) will include it by default. Anything at the level of "a computer" +(firmware ABI, devices, ticks, power, BIOS/OS boot, machine save format) +belongs to those mods, not to Shuttle. Current status and open decisions: `native/HANDOFF.md`. Overview and docs: `README.md`, `docs/`. | Path | What | | --- | --- | -| `native/` | Rust workspace: `shuttle-core`, `shuttle-runner`, C test guests | +| `native/` | Computer prototype (`shuttle-core`, `shuttle-runner`, C guests); belongs to RC, will move there | | `shuttle-java/` | `jp.akkiserver.shuttle`: Wasmtime + Weave JNI binding | | `shuttle-java-example/` | Weave example | | `shuttle-wit-parser/`, `shuttle-wit-maven-plugin/` | WIT tooling | diff --git a/README.md b/README.md index 44503a3..4204d4b 100644 --- a/README.md +++ b/README.md @@ -2,68 +2,59 @@ # Shuttle -> **Status:** early prototype. The native core runs headless only; no mod uses Shuttle yet. +> **Status:** early prototype. No mod uses Shuttle yet. -Shuttle is a WebAssembly runtime library for Minecraft computer mods. It is not a mod itself: computer mods use it to run the programs on their in-game computers. Two consumers are planned: OpenComputers (OC), through a separate addon mod, and a new Retro Computing mod, which will include Shuttle by default. Neither integration exists yet. +Shuttle is a persistable WebAssembly runtime for Java: [Wasmtime](https://github.com/bytecodealliance/wasmtime) integrated with [Weave](https://github.com/akkiserver-dev/weave), plus its Java binding. It plays the role for Wasm that JNLua plays for Lua in OpenComputers: it runs guest code and can save it mid-execution, and nothing more. It is not a mod itself. -On behalf of the host mod, Shuttle runs a stack of Wasm guests: - -``` -firmware implemented by Shuttle's native core - └─ BIOS player-made Wasm (for example stored on an EEPROM item) - └─ OS kernel player-made Wasm, started by the BIOS with `exec` - └─ child programs started by the kernel with `co_create` -``` - -The firmware talks to guests only through the **firmware ABI** (`fw.*`, `co.*`, `sys.call`). Device calls look like OpenComputers method calls, either synchronous (`dev_call`) or asynchronous (`dev_submit` + a completion event). The ABI is specified in a separate draft document (draft 0.1, not published in this repository); [`native/README.md`](native/README.md) lists which parts of it are implemented. +Computer mods build on it. Two consumers are planned: OpenComputers (OC), through a separate addon mod, and a new Retro Computing mod (RC), which will include Shuttle by default. Everything at the level of "a computer" (firmware, BIOS/OS boot, devices, ticks, the save format of a whole machine) belongs to those mods, not to Shuttle. ## How it works -- **Runtime:** [Wasmtime](https://github.com/bytecodealliance/wasmtime), made checkpointable with [Weave](https://github.com/akkiserver-dev/weave) (branch `shuttle-host-hooks`). Weave transforms a Wasm module so that a running program can be unwound into its linear memory and rebuilt later. This lets a host mod save a computer with the world and continue it mid-execution after a reload. -- **Native core (Rust, `native/`):** Wasmtime, Weave, the firmware ABI, the `co_*` machinery, memory limits and snapshot capture. Today it is driven by the headless `shuttle-runner`. -- **Java binding (`shuttle-java`):** Java API over Wasmtime and Weave through JNI, for the host mod to call. A high-level Machine API on top of the native core is planned but not written yet. +- **Persistence:** Weave transforms a Wasm module so that a running program can be unwound into its linear memory and rebuilt later. A host mod can therefore save a running guest with the world and continue it after a reload. +- **Java binding (`shuttle-java`):** Wasmtime and Weave through JNI: load a module, run it for a bounded number of polls, define host functions, take and restore snapshots. Planned extensions (yielding imports, memory limits, nested instances) are described in [docs/design/runtime-extensions.md](docs/design/runtime-extensions.md). - **Targets:** Windows, macOS and Linux, each on x86_64 and arm64. ## Repository layout | Path | What | | --- | --- | -| [`native/`](native) | Shuttle native core (Rust): `shuttle-core`, the headless `shuttle-runner`, and C test guests in `native/guests`. See [`native/README.md`](native/README.md). | | [`shuttle-java/`](shuttle-java) | Java bindings for Wasmtime plus the Weave (checkpointable Wasm) JNI layer. See [docs/java-binding.md](docs/java-binding.md). | | [`shuttle-java-example/`](shuttle-java-example) | Runnable Weave example: run a guest a few "ticks" at a time, snapshot it, restore it and continue. | | [`shuttle-wit-parser/`](shuttle-wit-parser), [`shuttle-wit-maven-plugin/`](shuttle-wit-maven-plugin) | Build-time WIT tooling: generate Java component-context interfaces from WIT files. See [docs/wit-codegen.md](docs/wit-codegen.md). | +| [`native/`](native) | A prototype of a whole computer (firmware ABI, BIOS → OS → child programs) on Wasmtime + Weave, in Rust. It belongs to the RC mod and will move there; it is kept here for now as a reference. See [`native/README.md`](native/README.md). | | [`docs/`](docs) | Documentation. Start at [docs/README.md](docs/README.md). | The Java modules form one Maven reactor: groupId `jp.akkiserver`, parent artifactId `shuttle`, one artifact per directory (`shuttle-java`, `shuttle-java-example`, `shuttle-wit-parser`, `shuttle-wit-maven-plugin`). The version is set in the root [`pom.xml`](pom.xml). ## Quick start -Native core (requires Rust, plus clang and wasm-ld for the C guests): +Java bindings (requires JDK 21, Maven and Rust; Maven picks the native build for your OS and CPU automatically): -```powershell -.\native\scripts\build.ps1 # Windows: build guests + workspace, then verify -``` ```sh -./native/scripts/build.sh # Linux / WSL2 +mvn -pl shuttle-java,shuttle-java-example -am install +mvn -pl shuttle-java-example exec:java ``` -`verify` boots BIOS → OS → child, then repeats the run with capture → restore into a fresh machine at every tick boundary and checks that the logs are identical. +The second command runs the Weave example. Platform profiles, CI and known problems are covered in [docs/building.md](docs/building.md). -Java bindings (requires JDK 21, Maven and Rust; Maven picks the native build for your OS and CPU automatically): +The `native/` prototype (requires Rust, plus clang and wasm-ld for the C guests): +```powershell +.\native\scripts\build.ps1 # Windows: build guests + workspace, then verify +``` ```sh -mvn -pl shuttle-java,shuttle-java-example -am install -mvn -pl shuttle-java-example exec:java +./native/scripts/build.sh # Linux / WSL2 ``` -The second command runs the Weave example. Platform profiles, CI and known problems are covered in [docs/building.md](docs/building.md). +`verify` boots BIOS → OS → child, then repeats the run with capture → restore into a fresh machine at every tick boundary and checks that the logs are identical. ## Documentation -- [docs/building.md](docs/building.md): building the native core and the Maven modules, CI, requirements, known issues +- [docs/building.md](docs/building.md): building the Maven modules and the native/ prototype, CI, requirements, known issues - [docs/java-binding.md](docs/java-binding.md): the `shuttle-java` API (Wasmtime and Weave) and its architecture +- [docs/design/runtime-extensions.md](docs/design/runtime-extensions.md): planned runtime extensions (design) - [docs/wit-codegen.md](docs/wit-codegen.md): generating Java component contexts from WIT -- [native/README.md](native/README.md): the native core prototype, its results and its ABI coverage +- [native/README.md](native/README.md): the computer prototype (moving to the RC mod), its results and its ABI coverage ## Credits and license diff --git a/docs/README.md b/docs/README.md index 6aa1c89..dab764b 100644 --- a/docs/README.md +++ b/docs/README.md @@ -2,9 +2,10 @@ | Document | Contents | | --- | --- | -| [building.md](building.md) | Requirements; building the native core (`native/scripts/build.ps1`, `build.sh`, what `verify` checks); building the Maven modules and their profiles; CI workflows; known issues. | +| [building.md](building.md) | Requirements; building the Maven modules and their platform profiles; building the `native/` prototype (`native/scripts/build.ps1`, `build.sh`, what `verify` checks); CI; known issues. | | [java-binding.md](java-binding.md) | The `shuttle-java` module: Wasmtime API (engine, module, store, linker, instance, memory, functions), value types, Component Model support, the Weave JNI layer for checkpointable guests, and the architecture (Java API / JNI layer in Rust / Wasmtime). | | [wit-codegen.md](wit-codegen.md) | `shuttle-wit-parser` and `shuttle-wit-maven-plugin`: generating Java component-context interfaces from WIT files. | -| [../native/README.md](../native/README.md) | The native core prototype (`shuttle-core`, `shuttle-runner`, C guests): layout, build, test results, and which parts of the firmware ABI draft are implemented. | +| [design/runtime-extensions.md](design/runtime-extensions.md) | Design (not implemented): yielding imports, memory limits and nested-instance primitives for the Weave binding. | +| [../native/README.md](../native/README.md) | The computer prototype (`shuttle-core`, `shuttle-runner`, C guests), which will move to the Retro Computing mod: layout, build, test results, firmware ABI coverage. | Back to the [project README](../README.md). diff --git a/docs/design/runtime-extensions.md b/docs/design/runtime-extensions.md new file mode 100644 index 0000000..4e4400c --- /dev/null +++ b/docs/design/runtime-extensions.md @@ -0,0 +1,171 @@ +# Design: runtime extensions for the Weave binding + +Status: **design, not implemented** (2026-10-15). API names marked *(proposed)* need +the owner's approval before implementation; names marked *(approved)* were approved +on 2026-10-17. + +## Scope + +Shuttle provides a Weave-integrated, persistable Wasmtime runtime and its Java +binding, in the same role that JNLua plays for OpenComputers. Everything at the level +of "a computer" (firmware ABI, devices, ticks, power, BIOS/OS boot, the SHTL save +format) belongs to the host mod; the prototype in `native/shuttle-core` is going to +move to the Retro Computing mod. + +Two features of that prototype are generic runtime features and belong in Shuttle's +Java binding (`shuttle-java`, package `jp.akkiserver.shuttle`): + +1. **Weave extensions**: yielding imports, unwinding from inside a host call, and + per-instance memory limits. +2. **Nested instances**: the primitives that a host needs to run one guest from + inside another guest's host call, with a shared execution budget (what + `native/shuttle-core` uses for its `co_*` imports). + +The existing Weave layer (`WeaveModule`, `WeaveInstance`, `WeaveCaller`, +`shuttle-java/src/main/rust/wasmtime-binding/src/weave.rs`) is extended; nothing is +removed. + +## 1. Weave version + +`shuttle-java` pins Weave `0f26893`, which lacks all three extensions. They exist in +`677cce6` (branch `shuttle-host-hooks` of `akkiserver-dev/weave`), which `native/` +already uses: + +| Weave change | API | +| --- | --- | +| `0286544` yielding imports, per-instance memory limits | `TransformOptions::yielding_imports`, `weave_wasmtime::request_unwind`, `WeaveInstance::set_memory_limit`, `Ctx::set_memory_limit` | +| `677cce6` change the limit from inside a host call | `Ctx::set_memory_limit` usable in host functions | + +The Weave-side names in the table are kept as they are (approved). `Cargo.toml` pins +Weave by commit (`rev`), so the branch name `shuttle-host-hooks` does not matter to +Shuttle and stays. + +Changes: + +- `wasmtime-binding/Cargo.toml`: all four Weave crates to rev + `677cce64cf9728485849af21f7bf36f801175d4f`; wasmtime to 46.0.3 (same as `native/`). +- `weave.rs` (`module_create`): the `TransformOptions` literal gains + `..Default::default()`, otherwise it stops compiling once the new field exists. + +## 2. Yielding imports + +**Problem.** `WeaveCaller.requestUnwind()` sets the poller to `UnwindNext`: the host +call returns normally, and the guest stops at its next instrumented poll. A host +function cannot stop the guest *at the call* and produce its result later (for +example, a disk read that completes on the server thread). + +**Weave's mechanism.** A module transformed with +`TransformOptions::yielding_imports = [(module, name), ...]` treats calls to those +imports as unwind points. Inside such a host call, +`weave_wasmtime::request_unwind(caller)` makes the guest unwind right at the call +site. On `resume`, the guest calls the **same import again** with the same +arguments, and that second call's return value is used. + +**Java API** *(approved)*: + +```java +WeaveModule module = WeaveModule.load(engine, wasm, + WeaveModule.Options.builder() + .pollPeriod(64) + .stackPages(4) + .yieldingImport("dev", "read") // may be repeated + .build()); + +builder.define("dev", "read", "ii>i", (caller, args) -> { + Result r = pending.get(key(args)); + if (r == null) { // first call: start the work, stop the guest + submitToServerThread(args); + caller.suspend(); + return new long[] {0}; // ignored: the guest unwinds + } + return new long[] {r.value()}; // re-call after resume: the real result +}); +``` + +- `WeaveModule.Options` (builder) with `pollPeriod`, `stackPages`, `yieldingImport(module, + name)`. The existing `load(engine, wasm)` and `load(engine, wasm, pollPeriod, + stackPages)` stay and map to it. +- `WeaveCaller.suspend()` calls `request_unwind`. `call`/`resume` then return + `isUnwound()`. +- Calling `suspend()` from an import that is not declared as yielding throws. Weave + does not check this itself: `request_unwind` only refuses calls during module + initialization, and otherwise just sets the unwind state. The binding therefore + records, per defined host function, whether its import is in the module's + yielding list, and `suspend()` checks that flag. +- Because the import is called again after `resume`, and possibly after a + `snapshot` + `restore` in a later session, the host keeps the request state on its + side and makes the second call idempotent. +- `requestUnwind()` keeps its current meaning (stop at the next poll). + +## 3. Memory limits + +Without a limit, a guest can grow its memory until the host runs out. + +**Java API** *(approved)*: + +| Method | Maps to | Notes | +| --- | --- | --- | +| `WeaveInstance.setMemoryLimit(long bytes)` | `WeaveInstance::set_memory_limit(Some(n))`, or `None` for a negative value | any negative value (conventionally `-1`) means unlimited; there is no separate `clear` method | +| `WeaveInstance.memorySize()` | `mem_view().size(0)` | current size of memory 0, for accounting | +| `WeaveCaller.setMemoryLimit(long bytes)` | `Ctx::set_memory_limit` (same rule) | inside a host call, e.g. to shrink a parent's limit after creating a child | + +- A `memory.grow` past the limit returns -1 to the guest; it does not trap. +- The limit is not part of a Weave snapshot. After `restore`, the host sets it again. + +## 4. Nested instances + +`native/shuttle-core` runs a child program from inside the kernel's `co_resume` host +call (`native/crates/shuttle-core/src/imports.rs`, `instantiate_child` and +`co_resume`). The runtime part of that is small: + +1. **Re-entrancy.** A host function may `call` or `resume` another `WeaveInstance` + on the same thread. Each instance has its own Store, so this already works in + Rust. The Java binding has to allow it (no global lock, the outer `WeaveCaller` + stays valid) and a test has to cover it. +2. **Shared execution budget.** The child's polls come out of the parent's + remaining budget. The host needs: + - `WeaveCaller.pollBudget()` / `WeaveCaller.setPollBudget(long)` *(approved)*: the + parent's `Poller::UnwindAfter(n)`, read and written inside the host call. + - `WeaveInstance.pollBudget()` *(approved)*: after `resume`, the child's leftover + budget (`take_poller`), so the host can compute how many polls it used. + - The existing `unwindAfter(n)` keeps its name. + + The host gives the child `min(requested, parent remaining)` with + `unwindAfter(...)`, resumes it, and subtracts the polls the child used from the + parent: + + ```java + long remaining = caller.pollBudget(); + long given = Math.min(requested, remaining); + child.unwindAfter(given); + WeaveResult r = child.resume(); + long used = given - child.pollBudget(); + caller.setPollBudget(remaining - used); + ``` + +3. **Snapshots** stay per instance: `snapshot()` on the parent and on each child, plus + `WeaveModule.hash()` to identify the module. Putting them into one save file is + the host's job. + +Not part of Shuttle: the `co_*` import names and their ABI, child handles, the +call/yield record format, and the policy for how much memory a kernel may use. + +## Out of scope for now + +- Worker threads and stack sizes (`max_wasm_stack`), epoch-based time limits. +- A versioned envelope around snapshots (today the bytes are gzip of Weave's encoding). +- Moving `native/` to the Retro Computing mod repository. + +## Tests for the implementation + +Extend `shuttle-java/src/test/java/jp/akkiserver/shuttle/WeaveTest.java`: + +- Yielding import: `suspend()` stops the guest at the call; after `resume()` the + import is called again and its value is used; the same across `snapshot()` + + `restore()`; `suspend()` from a non-yielding import throws. +- Memory limit: `memory.grow` past the limit returns -1 and the guest keeps running; + the limit set from inside a host call takes effect; `memorySize()` matches. +- Nested instances: a host function resumes a child; the child's polls are taken + from the parent's budget; parent and child snapshots restore and continue. + +Run `mvn test` on Windows and on WSL2. diff --git a/docs/java-binding.md b/docs/java-binding.md index 7b244ad..4746fd2 100644 --- a/docs/java-binding.md +++ b/docs/java-binding.md @@ -281,7 +281,9 @@ Rules that follow from the API: ### Relation to `native/` -The Weave layer in `shuttle-java` and the native core in `native/` are separate. `shuttle-java` pins Weave commit `0f26893`, which does not have the `shuttle-host-hooks` additions (yielding imports, memory limits). `native/` pins `677cce6` on branch `shuttle-host-hooks`. How the Java layer will connect to `shuttle-core` (a Machine API over JNI) has not been designed yet. +`native/` is a prototype of a whole computer (firmware ABI, devices, BIOS → OS → child programs) built directly on Weave in Rust. That level belongs to the host mod, so `native/` will move to the Retro Computing mod and is not connected to `shuttle-java`. + +`shuttle-java` pins Weave commit `0f26893`, which does not have the `shuttle-host-hooks` additions (yielding imports, memory limits) that `native/` uses (`677cce6`). Bringing those, plus the primitives for nested instances, into this binding is designed in [design/runtime-extensions.md](design/runtime-extensions.md). ## Architecture diff --git a/native/HANDOFF.md b/native/HANDOFF.md index ab9d4cc..32cd575 100644 --- a/native/HANDOFF.md +++ b/native/HANDOFF.md @@ -7,24 +7,27 @@ files in the repo stay in English. ## 1. What the project is -Shuttle is a **WebAssembly runtime library** (Java binding + native Rust) -for Minecraft computer mods. It is **not a mod itself**. Planned consumers: +**Scope (owner, 2026-10-15): Shuttle only provides a Weave-integrated, +persistable Wasmtime runtime and its Java binding**, the way JNLua provides +the Lua runtime for OpenComputers. It is **not a mod**. Planned consumers: - **OpenComputers (OC)**, through a separate addon mod. - **A new Retro Computing mod (RC)** by the owner, which includes Shuttle by default. -Neither integration exists yet. Everything a computer runs is Wasm: a BIOS -(player-made, e.g. stored on an EEPROM item), an OS kernel, and the OS's -child programs. +Neither integration exists yet. Everything at the level of "a computer" +belongs to the host mod: firmware and its ABI, devices, ticks, power, +BIOS -> OS -> child programs, the save format of a whole machine. - Runtime: **Wasmtime**, made checkpointable with **Weave** (a Wasm transformer that can unwind a running program into its linear memory and - rebuild it later). This is how a host mod saves a computer with the world - and resumes it mid-execution after a reload. -- Layers: firmware (implemented by Shuttle) -> BIOS -> OS -> child programs. - The firmware talks to guests only through the firmware ABI, never through - Wasmtime APIs. + rebuild it later). This is how a host mod saves running guests with the + world and resumes them after a reload. +- Kept in Shuttle as generic runtime features (designed, not implemented): + Weave extensions (yielding imports, `request_unwind`, memory limits) and + the primitives for nested instances. See `docs/design/runtime-extensions.md`. +- `native/` (`shuttle-core`, `shuttle-runner`, C guests, firmware ABI) is a + prototype of the RC side and will move to the RC repository. - Native code: Rust (Wasmtime + Weave + JNI). Java side: `shuttle-java`. - Targets: Windows, macOS, Linux, each x86_64 and arm64. @@ -33,10 +36,11 @@ child programs. | What | Where | | --- | --- | | This repo | `akkiserver-dev/shuttle` (forked from StefanRichterHuber/wasmtime-ng) | -| Working branch | `feature/unified-ci` (from `main` after PR #2 was merged as `7109d4f`) | -| PRs | #1 closed (CI check only); #2 merged (native core, rename, docs, CI fixes) | +| Working branch | `feature/runtime-extensions-design` (from `main` at `99956f1`, after PR #3 was merged) | +| PRs | #1 closed (CI check only); #2 merged (native core, rename, docs, CI fixes); #3 merged (unified CI, host profiles) | | Weave fork | `akkiserver-dev/weave`, branch `shuttle-host-hooks` (commit `677cce6`) | -| Firmware ABI spec (Japanese) | https://claude.ai/code/artifact/05272d17-ac4c-4f56-933a-374281672f79 | +| RC firmware ABI spec (Japanese) | https://claude.ai/code/artifact/05272d17-ac4c-4f56-933a-374281672f79 | +| RC Java-side device layer spec (Japanese) | https://claude.ai/code/artifact/f10cc73d-71d3-4ac4-8508-891479a7b215 | | Docs | `README.md` (overview), `docs/` (building, java-binding, wit-codegen), `native/README.md` | Modules (Maven groupId `jp.akkiserver`, parent artifactId `shuttle`): @@ -52,7 +56,7 @@ Modules (Maven groupId `jp.akkiserver`, parent artifactId `shuttle`): Old names (before commit `983dc25`): `wasmtimejavang`, `wasmtimejavang-example`, `wit-parser`, `wit-codegen-maven-plugin`, groupId `io.github.stefanrichterhuber`, packages `io.github.stefanrichterhuber.{wasmtimejavang,witparser,witcodegen}`. -Remote branches `feature/native-build` and `feature/shuttle-core` are merged/obsolete (not deleted yet). +Remote branches `feature/native-build`, `feature/unified-ci` and `feature/shuttle-core` are merged/obsolete (not deleted yet). ## 3. Current state (2026-10-15) @@ -70,9 +74,15 @@ Remote branches `feature/native-build` and `feature/shuttle-core` are merged/obs Snapshots 5.3 KB (tick 1) and about 7.6 KB later, matching the Linux numbers in `native/README.md`; so the larger Windows snapshots (about 9.3 KB) are an OS difference, cause not investigated. -- **WSL2 Maven not run yet**: needs `sudo apt install openjdk-21-jdk maven` - and `rustup target add wasm32-unknown-unknown`. The owner's bar for "done" - is build + tests passing on the Windows host and on WSL2. +- **WSL2 Maven verified**: `mvn test` in `/mnt/d/Projects/shuttle` (no `-P`; + `host-linux-x86-64` activates itself) passes for all five modules + (4m07s). Needs `openjdk-21-jdk`, `maven` and the `wasm32-unknown-unknown` + Rust target. The owner's bar for "done" (build + tests on the Windows host + and on WSL2) is met. +- **CI (`ci.yml`, PR #3)**: all 13 jobs pass. One earlier `java + (macos-aarch64)` failure was transient: Maven could not download + `maven-filtering:3.4.0` from Central; a re-run passed. Further CI work is + on hold by the owner's decision. - **Maven on Windows** (IntelliJ's bundled Maven, JDK 25, no `-P`; the `host-windows-x86-64` profile activates itself), `RUSTFLAGS=-D warnings`: all five modules pass: `shuttle-java` 64/64, `shuttle-wit-parser` 4/4, @@ -84,18 +94,34 @@ Remote branches `feature/native-build` and `feature/shuttle-core` are merged/obs replaced by six OS-activated `host-*` profiles in the root pom, also used by `shuttle-wit-parser`. - **Two Weave versions coexist**: `shuttle-java` pins `0f26893` (no - `shuttle-host-hooks`), `native/` pins `677cce6`. How the `d400a75` Weave - JNI layer and `native/shuttle-core` fit together is **not decided**. + `shuttle-host-hooks`), `native/` pins `677cce6`. Decided: they are not + connected; `shuttle-java` moves to `677cce6` as part of the + runtime-extensions design, and `native/` moves to the RC side. ## 4. Decided with the owner -- Rust/Java boundary: Weave, `co_*`, VM exits, memory pool and snapshot - capture live in Rust. Device implementations live in Java and are called - directly from Rust (JNI) on worker threads; world-touching work is queued - to the server thread. Host mod code sees a high-level Machine API only. -- Threads: Rust owns a worker pool (reliable stack sizes for - `max_wasm_stack`). Saving is split: capture on the worker at a tick - boundary, compress and write on a separate save thread. +The first group was decided while Shuttle was still thought to include the +machine. Under the clarified scope (section 1) it belongs to the RC mod and +is kept here for reference until `native/` moves. + +### 4a. Machine level (belongs to the RC mod) + +- Firmware language (2026-10-15, replaces the earlier Rust/Java boundary): + the whole firmware (`fw.*`, `co.*`, the machine, devices, the save format) + is written in Java in the RC mod, as in OpenComputers. It is built on + Shuttle's host functions, yielding imports, memory limits and + nested-instance primitives (`docs/design/runtime-extensions.md`). + `native/shuttle-core` (Rust) followed the earlier plan, where `co_*`, VM + exits, the memory pool and snapshot capture lived in Rust; it is now only + a reference for the Java rewrite. +- Threads (2026-10-17, replaces "Rust owns a worker pool"): Shuttle creates + no threads. The host (RC, Java) owns the worker pool and creates its + threads with an explicit stack size + (`new Thread(group, task, name, stackSize)`), with `max_wasm_stack` set + below that stack size. Virtual threads are not used for guests. Device + calls that touch the world are queued to the server thread. Saving is + split: capture at a tick boundary, compress and write on a separate save + thread. - Persistence: NBT/DataComponents hold only a UUID and small metadata; the snapshot and file systems are separate files in the world folder. - ABI shape (see the spec): balanced realism. Kept: `cpuid` strings, slots @@ -106,6 +132,17 @@ Remote branches `feature/native-build` and `feature/shuttle-core` are merged/obs other CPU properties come from parts. Feature flags map to Wasm features. - Runtimes are switchable through an OC-style Architecture abstraction. Lua comes later (Wasm-hosted Lua preferred; JNLua as a compatibility option). + +### 4b. Shuttle (runtime and repository) + +- Scope (2026-10-15): Shuttle = Weave-integrated persistable Wasmtime runtime + + Java binding only. Kept as runtime features: Weave extensions (yielding + imports, `request_unwind`, memory limits) and nested-instance primitives; + design in `docs/design/runtime-extensions.md` (API names there still need + approval). `native/` moves to the RC side. +- Threads (2026-10-17): Shuttle creates no threads. Guests and host + functions run on the Java thread that calls `call`/`resume`; the host mod + owns the threads and their stack sizes. - Build: option C. Local scripts for Windows and WSL2 build the host target; release binaries for all six platforms come from CI. - Weave may be modified freely. @@ -132,6 +169,9 @@ Remote branches `feature/native-build` and `feature/shuttle-core` are merged/obs ## 5. NOT approved yet (decided by Claude without asking; review each) +Item 3 concerns Shuttle (Weave API names). Items 1, 2, 4 and 5 concern the +machine prototype in `native/` and go to the RC side with it. + 1. `dev_call` result that does not fit `rcap`: held in one slot and fetched with `dev_result(-1, ...)`; replaced by the next `dev_call`. 2. Kernel memory: the kernel may use whatever its children have not @@ -167,11 +207,14 @@ Remote branches `feature/native-build` and `feature/shuttle-core` are merged/obs The owner picks the order. Candidates: -1. Finish `feature/unified-ci` (CI green on all six platforms, PR, merge). -2. WSL2 native build (after the owner installs the tools). -3. Decide how the Weave JNI layer (`shuttle-java`, Weave `0f26893`) and - `native/shuttle-core` (`677cce6`) relate, then design the Machine API JNI - layer. Review section 5 as part of it. -4. All-platform jar / distribution (Maven Central, GitHub Releases, bundled - in the RC mod, ...). -5. Smaller known issues from section 6. +1. Implement `docs/design/runtime-extensions.md` after the owner approves its + API names (and section 5 item 3): Weave bump to `677cce6`, yielding + imports, memory limits, nested-instance primitives, tests. +2. Move `native/` to the RC repository (when that repository exists). +3. All-platform jar / distribution (Maven Central, GitHub Releases, bundled + in the RC mod, ...). CI work is on hold until the owner asks. +4. Smaller known issues from section 6. + +Done on 2026-10-15: Windows and WSL2 builds + tests, rename to Shuttle, +docs, Java CI fix (PR #2), unified build-only CI and host profiles (PR #3), +scope clarified to runtime-only and the runtime-extensions design. diff --git a/native/README.md b/native/README.md index cfddb22..cd43807 100644 --- a/native/README.md +++ b/native/README.md @@ -1,9 +1,9 @@ # Shuttle (prototype) -Native core of Shuttle, the Wasm runtime library for computer mods: runs Wasm guests (BIOS → OS kernel → +A prototype of a whole computer, built on Wasmtime + Weave: runs Wasm guests (BIOS → OS kernel → child programs) on Wasmtime 46, makes them checkpointable with Weave, and implements the firmware ABI draft 0.1 (`fw.*`, `co.*`, `sys.call`). -Nothing here depends on Minecraft or Java yet. +Nothing here depends on Minecraft or Java yet. This level (firmware, devices, the machine) belongs to the host mod, not to the Shuttle runtime: this directory will move to the Retro Computing mod and is kept here as a reference. ## Layout