Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 85 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
# Builds everything on all six target platforms. CI only builds; tests are run
# on the developer machine.
#
# - guests: the C test guests (wasm32, identical everywhere), built once
# - native: the native/ Cargo workspace (shuttle-core, shuttle-runner)
# - java: the Maven reactor; each runner builds the JNI libraries for its own
# platform through the host-* profile the root pom activates
#
# setup-rust-toolchain sets RUSTFLAGS=-D warnings, so Rust warnings fail the build.

name: CI

on:
push:
branches: [main]
pull_request:
workflow_dispatch:

jobs:
guests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- name: Install clang and lld
run: sudo apt-get update && sudo apt-get install -y clang lld
- name: Build guests
run: sh native/guests/build.sh

native:
strategy:
fail-fast: false
matrix:
include:
- { name: linux-x86_64, os: ubuntu-latest }
- { name: linux-aarch64, os: ubuntu-24.04-arm }
- { name: windows-x86_64, os: windows-latest }
- { name: windows-aarch64, os: windows-11-arm }
- { name: macos-x86_64, os: macos-15-intel }
- { name: macos-aarch64, os: macos-latest }
name: native (${{ matrix.name }})
runs-on: ${{ matrix.os }}
timeout-minutes: 30
defaults:
run:
shell: bash
working-directory: native
steps:
- uses: actions/checkout@v5
- uses: actions-rust-lang/setup-rust-toolchain@v1
with:
cache-workspaces: native
- name: Build
run: cargo build --locked --release

java:
strategy:
fail-fast: false
matrix:
include:
- { name: linux-x86_64, os: ubuntu-latest }
- { name: linux-aarch64, os: ubuntu-24.04-arm }
- { name: windows-x86_64, os: windows-latest }
- { name: windows-aarch64, os: windows-11-arm }
- { name: macos-x86_64, os: macos-15-intel }
- { name: macos-aarch64, os: macos-latest }
name: java (${{ matrix.name }})
runs-on: ${{ matrix.os }}
timeout-minutes: 45
defaults:
run:
# bash on every OS: PowerShell would split -Dcargo.profile=release at the dot
shell: bash
steps:
- uses: actions/checkout@v5
- uses: actions-rust-lang/setup-rust-toolchain@v1
with:
target: wasm32-unknown-unknown # the valuetest fixture is built even with -DskipTests
- name: Set up JDK 21
uses: actions/setup-java@v5
with:
java-version: "21"
distribution: temurin
cache: maven
- name: Build with Maven
run: mvn -B -DskipTests -Dcargo.profile=release -Dcargo.profile.folder=release package
55 changes: 0 additions & 55 deletions .github/workflows/maven.yml

This file was deleted.

59 changes: 0 additions & 59 deletions .github/workflows/native.yml

This file was deleted.

7 changes: 5 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,8 +51,11 @@ will include it by default. Current status and open decisions:
- Maven: there is no `mvn` on PATH. Use IntelliJ's bundled Maven,
`%LOCALAPPDATA%\Programs\IntelliJ IDEA\plugins\maven-plugin\lib\maven3\bin\mvn.cmd`,
with `JAVA_HOME=%USERPROFILE%\.jdks\temurin-25.0.4` (the global
`JAVA_HOME` points at `bin\` and is wrong), and the profile
`-P build-host-windows-x86_64`. The default profile uses cross + Docker.
`JAVA_HOME` points at `bin\` and is wrong). No `-P` is needed: the root
pom activates the `host-windows-x86-64` profile automatically. Quote
`-D...` arguments in PowerShell (`"-Dcargo.profile=release"`).
- Tests run on this machine only; CI (`.github/workflows/ci.yml`) only builds,
on six platforms.
- JNI class names are also written as strings in Rust (`java_type = "..."`).
After renaming Java classes or packages, grep the `.rs` files and run the
tests: a wrong name compiles but fails at run time.
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
[![Native (Shuttle)](https://github.com/akkiserver-dev/shuttle/actions/workflows/native.yml/badge.svg)](https://github.com/akkiserver-dev/shuttle/actions/workflows/native.yml)
[![CI](https://github.com/akkiserver-dev/shuttle/actions/workflows/ci.yml/badge.svg)](https://github.com/akkiserver-dev/shuttle/actions/workflows/ci.yml)

# Shuttle

Expand Down Expand Up @@ -49,14 +49,14 @@ Native core (requires Rust, plus clang and wasm-ld for the C guests):

`verify` boots BIOS → OS → child, then repeats the run with capture → restore into a fresh machine at every tick boundary and checks that the logs are identical.

Java bindings on Windows (requires JDK 21, Maven and Rust; no Docker):
Java bindings (requires JDK 21, Maven and Rust; Maven picks the native build for your OS and CPU automatically):

```powershell
mvn -P build-host-windows-x86_64 -pl shuttle-java,shuttle-java-example -am install
```sh
mvn -pl shuttle-java,shuttle-java-example -am install
mvn -pl shuttle-java-example exec:java
```

The second command runs the Weave example. Linux builds, the other Maven profiles, CI and known problems are covered in [docs/building.md](docs/building.md).
The second command runs the Weave example. Platform profiles, CI and known problems are covered in [docs/building.md](docs/building.md).

## Documentation

Expand Down
81 changes: 36 additions & 45 deletions docs/building.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Neither build depends on the other.
| clang + wasm-ld | the C guests in `native/guests`. Windows: the [LLVM installer](https://github.com/llvm/llvm-project/releases). Linux/WSL2: `sudo apt install clang lld` |
| JDK 21 and Maven | the Maven reactor (`maven.compiler.release` is 21) |
| Rust target `wasm32-unknown-unknown` | the `shuttle-java` tests (the `valuetest` fixture is built for it): `rustup target add wasm32-unknown-unknown` |
| [cross](https://github.com/cross-rs/cross) + Docker | only the Maven profiles that cross-compile (see below), and `build.sh --cross` |
| [cross](https://github.com/cross-rs/cross) + Docker | only `native/scripts/build.sh --cross` (the Maven build does not use it) |

## Native core (`native/`)

Expand Down Expand Up @@ -63,23 +63,30 @@ The results of the last measured run are in [native/README.md](../native/README.

## Maven reactor

### Profiles
### Host platform profiles

The native library of `shuttle-java` (`wasmtime-binding`, built as `wasmtimebinding`) is compiled during the `compile` phase by a profile, and copied into the jar under `jp/akkiserver/shuttle/libs/<platform>/`, where [jar-jni](https://github.com/questdb/jar-jni) finds it at runtime. Profiles defined in [`shuttle-java/pom.xml`](../shuttle-java/pom.xml):
The two JNI libraries, `wasmtimebinding` (in `shuttle-java`) and `witparserbinding` (in `shuttle-wit-parser`), are built with plain `cargo` during the `compile` phase, for the platform Maven runs on. They are copied into the jar under `.../libs/<platform>/`, where [jar-jni](https://github.com/questdb/jar-jni) finds them at runtime.

| Profile | Rust target | Built with | Jar folder |
The root [`pom.xml`](../pom.xml) has one profile per platform. Maven activates the right one automatically from the JVM's `os.name` / `os.arch`, so no `-P` is needed:

| Profile | Rust target | Jar folder | Library |
| --- | --- | --- | --- |
| `build-linux-x86-64` (**active by default**) | `x86_64-unknown-linux-gnu` | `cross` (Docker) | `linux-x86-64` |
| `build-linux-aarch64` | `aarch64-unknown-linux-gnu` | `cross` (Docker) | `linux-aarch64` |
| `build-windows-x86_64` | `x86_64-pc-windows-msvc` | `cross` (Docker) | `win32-x86-64` |
| `build-windows-aarch64` | `aarch64-pc-windows-msvc` | `cross` (Docker) | `win32-aarch64` |
| `build-host-windows-x86_64` | `x86_64-pc-windows-msvc` | local `cargo` (no Docker) | `win32-x86-64` |
| `host-linux-x86-64` | `x86_64-unknown-linux-gnu` | `linux-x86-64` | `.so` |
| `host-linux-aarch64` | `aarch64-unknown-linux-gnu` | `linux-aarch64` | `.so` |
| `host-windows-x86-64` | `x86_64-pc-windows-msvc` | `win32-x86-64` | `.dll` |
| `host-windows-aarch64` | `aarch64-pc-windows-msvc` | `win32-aarch64` | `.dll` |
| `host-macos-x86-64` | `x86_64-apple-darwin` | `darwin-x86-64` | `.dylib` |
| `host-macos-aarch64` | `aarch64-apple-darwin` | `darwin-aarch64` | `.dylib` |

Each profile only sets three properties (`native.rust.target`, `native.libdir`, `native.lib.ext`); the cargo and copy steps are written once in each module's `pom.xml`. Check which profile is active with:

There is no macOS profile yet, so `shuttle-java` cannot currently be packaged with a macOS library.
```sh
mvn -N help:active-profiles
```

Maven turns off an `activeByDefault` profile as soon as another profile of the same POM is selected with `-P`. So `-P build-host-windows-x86_64` builds only the Windows library and does not call `cross`.
A jar built this way contains the library for one platform only. Collecting the libraries of all six platforms into one jar is not set up yet.

Profiles defined in the root [`pom.xml`](../pom.xml):
Other profiles in the root `pom.xml`:

| Profile | Effect |
| --- | --- |
Expand All @@ -94,53 +101,37 @@ mvn -Dcargo.profile=release -Dcargo.profile.folder=release install

### Commands

Windows, local cargo only:

```powershell
# shuttle-java and the example (with shuttle-java's tests)
mvn -P build-host-windows-x86_64 -pl shuttle-java,shuttle-java-example -am install

# the whole reactor, skipping tests (see Known issues for why)
mvn -P build-host-windows-x86_64 -DskipTests install
```

Linux, default profile (needs `cross` and a running Docker):
The same on every platform:

```sh
mvn install
mvn test # build and run all tests
mvn install # build, test and install into ~/.m2
mvn -pl shuttle-java,shuttle-java-example -am install # only shuttle-java and the example
mvn -pl shuttle-java-example exec:java # run the Weave example (after install)
```

Several platforms in one jar:

```sh
mvn -P build-linux-x86-64,build-linux-aarch64,build-windows-x86_64 install
```

Run the Weave example (after `shuttle-java` is installed in the local repository):

```sh
mvn -pl shuttle-java-example exec:java
```

### `shuttle-wit-parser`

`shuttle-wit-parser` always builds its JNI library (`witparserbinding`) with a plain host `cargo build`, without `cross` or a profile, because it only runs on the build machine while the WIT plugin is generating sources.
On Windows, run these from PowerShell with each `-D...` argument quoted (`"-Dcargo.profile=release"`), because PowerShell splits unquoted arguments at the dot.

### Tests

Tests are run on the developer machine; CI only builds.

- `shuttle-java`: JUnit 5 tests (`WeaveTest`, `WasmtimeValueTest`, `WasmtimeComponentTest`, `WasmtimeMemoryTest`, ...). In `generate-test-resources`, `exec-maven-plugin` builds the Rust fixture `src/test/rust/valuetest` for `wasm32-unknown-unknown` into `target/rust-test/valuetest`. The other fixtures are inline WAT or inline Wasm bytes.
- `shuttle-wit-parser` and `shuttle-wit-maven-plugin`: parse the WIT files under `src/test/resources/wit` (`greet.wit`, `wasi-cli`, `wasi-sockets`) and compile the generated sources.
- Coverage is recorded with JaCoCo (`prepare-agent`, report in `prepare-package`).

## CI

| Workflow | Trigger | What it does |
One workflow, [`ci.yml`](../.github/workflows/ci.yml) "CI", runs on pushes to `main`, on every pull request, and on manual dispatch. It only builds; it runs no tests. `setup-rust-toolchain` sets `RUSTFLAGS=-D warnings`, so Rust warnings fail the build.

| Job | Runs on | What it does |
| --- | --- | --- |
| [`native.yml`](../.github/workflows/native.yml) "Native (Shuttle)" | push to `main` or PR touching `native/**` or the workflow; manual dispatch | Builds the C guests once on Ubuntu and uploads them as an artifact. Then, on Linux, Windows and macOS (x86_64 and arm64 each, six jobs): `cargo build --locked --release` and `shuttle-runner verify`. |
| [`maven.yml`](../.github/workflows/maven.yml) "Java CI with Maven" | push or PR to `main` | Ubuntu, JDK 21 (Temurin): installs `cross`, then runs `mvn -P build-linux-x86-64,build-linux-aarch64,build-windows-x86_64 -Dcargo.profile=release -Dcargo.profile.folder=release -B package`. |
| `guests` | Ubuntu | Builds the C guests (`native/guests/build.sh`). |
| `native (<platform>)` | the six platforms below | `cargo build --locked --release` in `native/`. |
| `java (<platform>)` | the six platforms below | JDK 21 (Temurin): `mvn -B -DskipTests -Dcargo.profile=release -Dcargo.profile.folder=release package`, which builds the JNI libraries for that platform. |

Platforms (GitHub-hosted runners): `ubuntu-latest`, `ubuntu-24.04-arm`, `windows-latest`, `windows-11-arm`, `macos-15-intel`, `macos-latest`.

## Known issues

- **`shuttle-wit-parser` tests fail on Windows.** Its `pom.xml` only copies the Linux x86_64 library (`*.so` into `jp/akkiserver/shuttle/wit/parser/libs/linux-x86-64`). On Windows the DLL is built but never copied into the jar folder, so loading fails with `cannot find .../win32-x86-64/witparserbinding.dll`. The `shuttle-wit-maven-plugin` tests use the parser too. On Windows, skip tests for these modules (`-DskipTests`) or build only `shuttle-java` (`-pl shuttle-java -am`).
- **"Java CI with Maven" is failing**, also on `main`. The cause has not been investigated yet. Possible factors: the workflow installs `wasm32-wasip1`/`wasm32-wasip2` targets (left over from the removed WASI modules) but not `wasm32-unknown-unknown`, which the `valuetest` fixture needs, and it cross-compiles `x86_64-pc-windows-msvc` with `cross`.
- **No macOS Maven profile.** `native/` builds on macOS in CI, but `shuttle-java` has no profile that builds or packages a macOS library.
- The JNI crates allow the `deprecated` lint: jni 0.22.4's `bind_java_type!` expands to `AtomicBool::fetch_update`, deprecated in Rust 1.99 ([jni-rs/jni-rs#846](https://github.com/jni-rs/jni-rs/issues/846)). Remove the `#![allow(deprecated)]` lines once jni ships a fix.
7 changes: 1 addition & 6 deletions docs/wit-codegen.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ The result types are records:

Supported: records, variants, enums, flags, options, results, tuples, lists, resources and all scalar types. Unsupported constructs throw: the WIT 0.3 async surface, `stream`/`future`, `map`, and fixed-length lists.

The native library is only packaged for Linux x86_64. On Windows the parser cannot load it (see [building.md](building.md#known-issues)).
The native library is built for the platform Maven runs on (Linux, Windows or macOS, x86_64 or arm64); see [building.md](building.md#host-platform-profiles).

## `shuttle-wit-maven-plugin`

Expand Down Expand Up @@ -181,8 +181,3 @@ public class MyBundledContext implements FooContext, BarContext {
// interfaces cannot hold state.
}
```

## Known issues

- **Generated imports for `variant`, `enum`, `result` and resources are broken.** The rename turned the import lines in `WitCodeGenerator.requiredImports` into `import component.jp.akkiserver.shuttle.WitVariant;` (and the same for `WitEnum`, `WitResult`, `WitResource`). The correct package is `jp.akkiserver.shuttle.component`. Generated code that uses any of these types does not compile until this is fixed. `greet.wit` is not affected.
- The parser only loads on Linux x86_64 (see [building.md](building.md#known-issues)).
Loading
Loading