Share a link, get a swarm. NuΓ©e turns the laptops and phones of a group that trusts each other into a private compute and file cluster: for one job, or as a permanent group that is its own server, running its builds and tests, keeping its files and serving its sites, and still there when any one machine is off. No server of ours, no account, no payment.
brew install maxgfr/tap/nuee
nuee run demo:mandelbrot
# open the printed link (or scan the QR code) on a phone: the tab starts computingThe web page is https://maxgfr.github.io/nuee/. It can join a swarm, run one, share a file or open any NuΓ©e link, with nothing to install.
Machines that stay together form a permanent group that survives the loss of any of them, then build and test on each other, inside a sandbox:
nuee group create "team" # prints a link that stays valid
nuee group invite - <machine id> --name alice --keeper
nuee group up - --allow-repo https://github.com/acme/app.git --exec-network \
--allow-command 'npm ci && npm run build' \
--allow-command 'npm ci && npm test -- --shard={i}/{n}'
# from a checkout whose commit is pushed:
nuee build --on - --keep dist -- 'npm ci && npm run build' # dist/ comes back
nuee shard 8 --on - -- 'npm ci && npm test -- --shard={i}/{n}' # eight piecesA machine only runs a command written exactly as one it allows. On Linux, group up needs
bwrap (bubblewrap) for its sandbox.
Or from a workflow, with the Action in this repository:
- uses: maxgfr/nuee@v0.7.2
with:
group: ${{ secrets.NUEE_GROUP }}
key: ${{ secrets.NUEE_KEY }}
command: npm ci && npm run build
keep: distnuee group status - --watch shows which machines are on line and the jobs as they run, and the
page of the group's link shows the same in a browser. nuee group host - -o nuee-server adds an
always-on keeper and writes a cloud-init file to paste when creating a Linux server at any host.
See docs/GROUPS.md, including what allowing a repository means. To try it alone in two minutes on this repository's own tests, follow this recipe.
The same group keeps files and sites, and another keeper takes over when the coordinating machine dies:
nuee group put - ./photos # the group keeps a copy; turn this machine off
nuee site publish ./public --on - --name blog # a link that stays the same across versions
nuee site loader - -o deploy/ # three files for GitHub Pages or any static hostThe loader serves the site, scripts and pages included, from the static host's own address: the
content comes from the group and is checked against its hash, and a returning visitor keeps it
with every machine off. The group also works from Python and coding agents (nuee mcp).
To see it with nothing installed, host the sample site from a tab: the tab plays the group, gives a stable link and the loader for it. Its preview runs no script; the loader runs the site whole.
More recipes, each run as written: real examples.
- Verified CPU jobs in WebAssembly. A coordinator (the CLI or the web page) splits a job into
tasks; CLI workers (
nuee join) and browser tabs run them. The same module gives the same output bytes in wasmtime, Chromium, Firefox and Safari (WebKit); the end-to-end tests check this on a golden corpus. Results are checked by recomputation (see Security model). - Demos.
demo:mandelbrot(a fractal, written out as an image),demo:mandelbrot-gpu(the same on GPUs),demo:skycmd(closed-loop evaluation of the skycmd s3-pico flight model, downloaded and checked against its BLAKE3 hash),demo:skycmd-dagger(oracle-labelled DAgger records from the same model, merged into one JSONL file) anddemo:train(a small digit recognizer trained with DiLoCo). - Your own jobs. A job TOML file, or any
.wasmmodule that follows the task ABI plus a directory of inputs (one file per task). JSONL inputs are cut into shards and merged back;[wasm] assetshands files (a model, a dataset) to every task. - GPU tasks and models. WGSL compute shaders (wgpu natively, WebGPU in browsers),
sentence embeddings (
nuee embed), and LLM option scoring through a local endpoint or a browser model, each verified with a measured tolerance. See docs/GPU.md. - Native commands, opt-in.
execjobs run a command from an exactrepo@commiton machines whose operator approved that exact command line and environment (join --allow-execasks on a terminal,--exec-approvalslists them ahead), for MLX or CUDA work;nuee sweepruns one variant per task and ranks them. See docs/SKYCMD.md. - Training.
nuee trainruns DiLoCo with WebAssembly trainers (recomputed bit for bit, browsers included) or native trainers (robust aggregation). See docs/TRAINING.md. - Files.
nuee shareencrypts files or folders with age before they leave the machine and prints a link;nuee getor the web page downloads, decrypts and checks them against BLAKE3.nuee pinkeeps a ciphertext-only copy and serves it, so the original sender can go offline. - Durable storage.
share --replicas N --on <swarm>keeps copies on members that offered space (join --store), repairs them when a holder disappears and checks holders with storage challenges;--erasure 4+2keeps a Reed-Solomon set instead. Only the peer that asked for a blob can release it; challenges prove possession of ranges, not independent copies. See docs/STORAGE.md. - Hosting.
nuee expose/nuee connecttunnel a local service through a link, the web page opens it as a JSON console, andnuee site publishshares a static site with an isolated preview. See docs/HOSTING.md. - An anchor.
nuee serve --anchoris an always-on coordinator, relay, store member and HTTP gateway, with Docker and systemd files. See docs/ANCHOR.md. - Your own relay.
nuee relayruns an iroh relay with Let's Encrypt, an optional bearer token, a connection cap and a per-client rate limit. - A direct lane between browsers. When a presenter running the web page turns on direct connections (WebRTC), browser workers that accept send their results straight to the presenter's page over a data channel, signalled through iroh, with iroh as the fallback. No TURN server; STUN is configurable.
- Permanent groups.
nuee groupkeeps a link valid while machines come and go, with a roster signed by the group's own key. When the coordinating keeper dies, the next one takes over with its copy of the files and sites.nuee buildandnuee shardrun builds and test suites across the machines in a sandbox, also from the GitHub Action, andnuee site publish --namegives a site a stable link.group statusand the page of the link show the machines on line and the jobs;group hostsets up an always-on server. See docs/GROUPS.md. - Interrupted runs resume. Outputs already in the output directory are kept; run the same command again.
All verbs print human text on stderr. --json prints NDJSON events on stdout instead.
The final done event of nuee run carries peers: [{id, browser, done, verified, banned}]:
per connected peer, done counts the accepted outputs it delivered first and verified those
of its results another execution agreed with. by in task, verified and results.json names only the execution that arrived
first, so a slower peer whose results agreed shows up only in these counters.
$ nuee run demo:mandelbrot
https://maxgfr.github.io/nuee/#/J/AAQ...
fingerprint π π π πΎ π² π§
peer joined: 5b1c⦠(nuee-web/0.1.0)
verified 212 Β· 3 peers Β· 38.5/s
done: 256 tasks, results in nuee-out/mandelbrot
image: nuee-out/mandelbrot/mandelbrot.ppmThe link and a QR code appear only once the relay is reachable (10 seconds at most, otherwise
exit code 3). Useful options: --verify full|sample|none (default sample), --sample 0.10,
--canary 0.03, --approve (admit each peer by hand), --wait-peers N, --local-slots N,
--deadline 30s, --out DIR, --no-resume, --qr-svg FILE.
A custom module:
nuee run my-task.wasm --inputs inputs/ # one file per task, sorted by name$ nuee join - # paste the link on stdin, it stays out of your shell history
coordinator fingerprint π π π πΎ π² π§ (compare with the coordinator's screen)
joined with 5 slots
job 1: mandelbrot
task 17 done in 41 ms (1 so far)
left the swarm after 87 tasksCompare the six emoji with the coordinator's screen before leaving it running. --slots N sets
how many tasks run at once (default: half the cores).
$ nuee share report.pdf photos/
https://maxgfr.github.io/nuee/#/F/AAQ...
serving 48213990 bytes (encrypted) until Ctrl-C
$ nuee get -o downloads/ - # link on stdinnuee get refuses to overwrite existing files unless given --force, and refuses archive
entries that would land outside the output directory.
nuee pin - # keep a ciphertext copy of a file or pin link
nuee pin --list
nuee pin --rm <HASH>
nuee serve # serve every pinned item until Ctrl-Cnuee relay --hostname relay.example.org --acme-contact you@example.org --token-file token.txt
nuee relay --dev --bind 127.0.0.1:3340 # plain HTTP, for local use and tests
nuee --relay https://relay.example.org run demo:mandelbrot$ nuee selftest # offline: no key, no network, no files
β wasm golden: 3 golden outputs match
β demo modules: embedded: mandelbrot, toy-trainer, skycmd-arena, optdigits
β links: job, file and pin links round-trip
β manifest: manifest signatures and admission proofs hold
β encryption: age encryption round-trips and detects tampering
β swarm: in-memory swarm finished 6 tasks
selftest ok
$ nuee doctor # identity, data directory, relay, wasm, GPU adapter, sandbox, clock
$ nuee id # node id and its fingerprint
$ nuee completions zshGlobal options: --relay URL, --relay-only (no direct connections), --web-base URL,
--config-dir DIR, --data-dir DIR, --ephemeral (throwaway identity, writes nothing),
-v, -q, --json.
| Guide | Covers |
|---|---|
| docs/SKYCMD.md | skycmd evaluation, teacher labelling with exec, sweeps, DiLoCo on Macs |
| docs/TRAINING.md | nuee train, DiLoCo, WebAssembly and native trainers |
| docs/GPU.md | WGSL tasks, WebGPU support, embeddings, LLM scoring, tolerances |
| docs/STORAGE.md | share, get, pin, replicas, repair, challenges, Reed-Solomon |
| docs/HOSTING.md | expose, connect, the console, sites, the gateway |
| docs/ANCHOR.md | running an always-on anchor |
| docs/GROUPS.md | permanent groups, nuee shard, what --allow-repo trusts |
--exec-mirror URL=PATH (on run, join and sweep, next to --allow-exec) fetches the
allowlisted repository from a local clone instead of the network. It is meant for operators of
air-gapped machines and for tests; the commit is checked the same way.
- The link is the capability. A job link carries the coordinator's public key, its relay
and a 16-byte secret. Anyone holding the link can join; treat it like a password. Links can
be read from stdin (
-) so they do not end up in shell history. - Admission is an HMAC-SHA256 proof over both peers' keys, compared in constant time. Connections are QUIC with mutual TLS 1.3 through iroh, end to end.
- Fingerprints. Six emoji (48 bits) derived only from the coordinator's key. Workers show them so people can compare with the coordinator's screen.
- Signed manifests and results. The coordinator signs each job manifest; workers sign each result. Signatures use domain separation.
- Verification by recomputation. In
samplemode the coordinator reruns a share of the results itself and mixes in known-answer canary tasks (they catch naive cheaters; identities sharing state can recognise a repeated input, so the hidden recompute is the guarantee). Infullmode every task also runs on two peers, but two agreeing peers may be one attacker: the coordinator still recomputes the same sampled share and sends canaries. A mismatch is settled by the coordinator's own run; only then is the peer banned and all its unverified results redone. This holds with--local-slots 0too: the coordinator then takes no tasks but still recomputes. Where it cannot run the kind at all (see docs/THREAT-MODEL.md), only agreement between two peers is left, andnuee runsays so. With a single peer able to run that kind, tasks then wait rather than being trusted, andnuee runsays it needs a second such device (or--verify none). - Worker-side caps. Workers enforce their own limits on memory, module size, inputs, outputs and time, whatever the manifest says. Modules must have zero imports and a declared memory maximum; Wasm features outside a fixed list (relaxed SIMD, threads and others) are refused.
- Files are encrypted with age before leaving the machine and checked with BLAKE3 on arrival. Pinners and store members only ever hold ciphertext.
- Tolerances where floats differ. GPU, embedding and LLM outputs are compared with tolerances measured across backends, so a dishonest worker has that much room: up to 5 % of a GPU fractal tile can be wrong unnoticed. Native trainers are not recomputed at all; robust aggregation bounds what one worker can do. Details in docs/GPU.md and docs/TRAINING.md.
execis off by default. A worker runs native code only for a command its operator approved exactly (repository, commit, argv and environment), in a throwaway checkout, with a cleaned environment, time limits andsandbox-execorbwrapwhen present. Approving a command means trusting the commit's code.--allow-repotrusts a whole repository. Onnuee group up, off by default and refused without a sandbox, it runs one shard command at any commit submitted by a machine on the roster. Anyone who can push to that repository can run code in the sandbox of those machines.- Hosted content never runs in the app's origin. The console shows JSON, the site preview
is an inert sandboxed frame, the gateway sends
Content-Security-Policy: sandbox.
Details: SECURITY.md (reporting, scope) and docs/THREAT-MODEL.md. The wire format is in docs/PROTOCOL.md.
- Relay operators see browser IP addresses. Traffic through a relay is end-to-end encrypted, but the relay sees who connects. Other peers do not see a browser's IP, unless the presenter turned WebRTC on: then the presenter and each participant who accepts see each other's IP address. Both the switch and the join card say so.
- The n0 public relays are for development and hobby use. For anything else, run your own
with
nuee relay, or use n0's paid relays. - Browser keys are ephemeral. One person holding the link can open many tabs, so "two
different peers agree" proves nothing. Real assurance comes from the coordinator's own
recomputation, or from
--approve. - Mobile tabs stop in the background. Keep the screen on while contributing.
- No Windows binary. Windows machines join through the browser.
- Groups keep files and static sites, not databases. A site with a server side runs on one
machine through
nuee expose, and stops with it. - A first visit to a group's site needs one of its keepers online, and a relay. Returning visitors of the loader do not.
- Whoever holds a group's link sees its activity: machine names, which are on line, job names and progress, how many files and which sites. Not the files, commands or links.
- Keepers are trusted. They can read what the group keeps and change what a name shows. Members are trusted too: one submission can bring up to 1 GiB of inputs, which a keeper holds in memory.
- A takeover is not instant. A keeper waits about 10 seconds per keeper ranked before it; a site or file sent to the group in that gap may be missed, so send it again.
- NuΓ©e is for groups that trust each other enough to share a link. It is not an open market.
| Project | What it is for | Difference with NuΓ©e |
|---|---|---|
| Akash | A marketplace to rent cloud compute, paid in tokens | NuΓ©e has no market, no payment, no provider to rent from |
| Salad | Paid rental of consumer GPUs to companies | NuΓ©e runs among people who know each other, unpaid |
| Mesh LLM | Pooling machines to serve a large language model, built on iroh | NuΓ©e runs verified batch jobs and shares files; it does not split a model across peers |
| Buzz (Block) | A peer-to-peer application built on iroh | NuΓ©e is a compute and file swarm joined from a link or a browser tab |
| BOINC | Volunteer computing for research projects, run from a project server | NuΓ©e needs no server: the coordinator is a laptop or a web page, joined from a link |
| Self-hosted CI runners | One of your machines takes whole jobs from GitHub or GitLab | nuee shard cuts one suite across several machines, from any CI or a terminal, and a teammate who builds the same commit gets the result back |
| Syncthing | Keeps folders in sync between your own devices | A group keeps copies on its keepers and gives a link that a browser opens with nothing installed |
| GitHub Pages, Netlify | Host a static site on their servers, free | The site comes from your machines under a link that stays the same; the loader can sit on Pages |
| Code | Meaning |
|---|---|
| 0 | ok |
| 1 | error |
| 2 | usage (bad arguments, truncated or wrong link) |
| 3 | network or relay unreachable |
| 4 | verification mismatch or banned |
| 5 | admission refused |
| 6 | integrity failure (hash or decryption) |
| 7 | cancelled or interrupted |
| 8 | swarm ended by the coordinator |
| Variable | Effect |
|---|---|
NUEE_CONFIG_DIR |
identity key directory (default ~/.config/nuee) |
NUEE_DATA_DIR |
data directory (default ~/.local/share/nuee) |
NUEE_ONLINE_TIMEOUT_MS |
how long to wait for the relay (default 10000) |
NUEE_DURABLE_ABSENT_MS |
coordinators: replace a stored copy whose holder has been gone this long (default 600000); for drills and tests |
NO_COLOR |
disable colors |
Needs the Rust toolchain pinned in rust-toolchain.toml (with the wasm32-unknown-unknown
target and the llvm-tools component), wasm-pack, wasm-opt and Node.
cargo build --release -p nuee # the CLI
scripts/build-tasks.sh # task modules into web/tasks/ and target/tasks/, golden checks
scripts/build-web.sh # browser package into web/pkg/<version>/
scripts/e2e.sh # Playwright on Chromium, Firefox and WebKit, local relay only
scripts/check.sh # the full gate CI runsEvery build embeds the mandelbrot and toy-trainer modules and the optdigits dataset from
tasks/golden/ (scripts/build-tasks.sh checks that a rebuild reproduces them byte for byte).
demo:skycmd and demo:skycmd-dagger also need the skycmd-arena module: release builds embed
it with cargo build --release -p nuee --features embed-tasks after scripts/build-tasks.sh;
without the feature, nuee looks for target/tasks/skycmd-arena.wasm in the checkout.
scripts/check.sh takes a group (static, test, e2e-1, e2e-2, e2e-3, fuzz), as CI does to run
them side by side. Without one it runs, in order: cargo fmt --check; clippy with -D warnings on the
workspace and on the wasm32 crates; the task build; cargo test; cargo deny and
cargo audit; clippy on tasks/; the web build; node --test web/test/; the end-to-end
tests. It stops at the first failure and names the stage.
crates/nuee the CLI
crates/nuee-proto wire types, links, manifests, limits, module validation
crates/nuee-core coordinator and worker state machines (no I/O)
crates/nuee-net iroh endpoints, admission, the network driver
crates/nuee-store blobs, encrypted file sharing, pinning, durable storage, Reed-Solomon
crates/nuee-exec wasmtime task runner and the opt-in native runner
crates/nuee-gpu WGSL, embeddings, LLM scoring and their tolerance comparators
crates/nuee-train DiLoCo: outer step, delta codec, robust aggregation
crates/nuee-host TCP tunnel, HTTP console requests, the anchor gateway
crates/nuee-web the browser build (wasm-bindgen)
tasks/nuee-task SDK for writing task modules
tasks/ task modules (mandelbrot, skycmd-arena, toy-trainer) and their golden copies
web/ the web page (no build step besides web/pkg)
e2e/ Playwright tests
fuzz/ cargo-fuzz targets
docs/ usage guides, protocol and threat model
deploy/ anchor Dockerfile, Compose file and systemd unit
examples/skycmd/ job files for skycmd
MIT OR Apache-2.0, at your option. See LICENSE-MIT and LICENSE-APACHE.