Part of magmalake — data lake building blocks in Mojo.
Minimal OS threads for Mojo: spawn and join
pthreads, share state through atomics and a mutex, fan a loop out over cores
with parallel_for, and keep long-lived workers running with WorkerPool. No
dependencies — the Mojo standard library and the pthread symbols every libc
already exports, nothing else. No dlopen, no C shim, no build step.
It exists because Mojo currently ships no way to use more than one core from
Mojo code: parallelize was removed from std.algorithm, and no thread pool
is reachable from std. If you want a second core, you call pthread_create
yourself. This tin is that call, done once, carefully, and tested.
Works on Mojo 1.3.0.dev2026111805 (the nightly every magmalake tin is pinned to), on linux-64 and osx-arm64 — both in CI.
pixi shelf add threads-mojoWorking with a coding agent? npx skills add mojoshelf/mojoshelf --skill mojoshelf-consume --yes teaches it to find and install tins itself — it installs the shelf CLI too.
That resolves the tin from mojoshelf and adds it — along with the tins it depends on — as pixi git source dependencies. magmalake tins are not published to a conda channel, so pixi add threads-mojo will not find them.
As a dependency declaration:
# pixi.toml
[dependencies]
threads-mojo = { git = "https://github.com/magmalake/threads.mojo" }or, like the rest of magmalake, by source path: check the repo out next to
yours and add two include paths to your mojo build — src, and compat:
mojo build … -I ../threads.mojo/src -I ../threads.mojo/compatThe second one holds a single comptime Cell = … line and is how this tin
compiles on every supported toolchain without forking; see
Why this does not just re-export std.atomic
below. Consumers taking the published tin need neither — the conda package is
precompiled.
from threads import AtomicCounter, parallel_for
@fieldwise_init
struct Totals(Copyable, Movable):
var sum: Int64
def task(i: Int, mut totals: Totals) -> None:
_ = AtomicCounter.at(Int(Pointer(to=totals.sum))).fetch_add(Int64(i))
def main() raises:
var totals = Totals(0)
parallel_for[task](1000, totals)
print(totals.sum) # 499500The state goes in by ref and every task gets it by mut. Because it is an
argument, it is alive for the whole call — no need to mention it afterwards
to keep last-use destruction at bay — and a read argument or a temporary is
rejected where the call is made.
The same call on a raw block of cells, when the context is not a struct:
from std.memory.alloc import unsafe_alloc
from threads import OpaquePtr, i64_ptr, opaque_ptr, parallel_for
def square(i: Int, ctx: OpaquePtr) -> None:
var cells = i64_ptr(Int(ctx))
cells[unsafe_offset=i] = cells[unsafe_offset=i] * cells[unsafe_offset=i]
def main() raises:
var n = 1_000_000
var data = unsafe_alloc[Int64](n)
for i in range(n):
data[unsafe_offset=i] = Int64(i)
parallel_for[square](n_tasks=n, ctx=opaque_ptr(Int(data)))
print(data[unsafe_offset=999]) # 998001
data.unsafe_free()pthread_create takes a C function pointer and one void *. That is the whole
channel, and this library does not pretend otherwise. Three rules, and they are
the same three everywhere in the API:
- Thin functions only. A thread body is
def(Int, OpaquePtr) thin -> None(ordef(OpaquePtr) thin -> OpaquePtrfor a raw thread). It cannot capture and it cannot raise — pthread has no exception channel. - Shared state travels through the
ctxpointer. One pointer, for every task. Lay out a block of 64-bit cells and index it; that is what the tests and the demo do. - The caller guarantees the context outlives the call.
parallel_forjoins every worker before it returns, so "outlives the call" is all you need. The typed form gets this from the compiler: the state is arefargument, so it lives across the call by construction. The opaque form does not — Mojo destroys a value at its last use, not at the end of the scope, and an untracked pointer is not a use, so a local you stop mentioning after the spawn can be freed underneath a running thread. Mention it after the join, or heap-allocate it and free it after the join.test_parallel_for_context_outlives_the_join,test_parallel_for_typed_state_lives_across_the_joinandtest_mutex_gives_mutual_exclusionexist to catch a regression here.
Neither form checks that the state is safe to mutate from several threads at
once. Every task has mut access to the same value; anything that is not an
atomic, a mutex-guarded region, or a slot only one task writes is a data race
the compiler will not see.
A task cannot raise, so failures come back through the context. The pattern —
an AtomicFlag plus one error-code cell per task — is documented in
threads/parallel.mojo and exercised by test_parallel_for_error_cell_pattern.
from threads import (
ThreadHandle, ThreadGroup, spawn_n, join_all,
num_cpus, current_thread_id, yield_now,
AtomicCounter, AtomicFlag,
Mutex, MutexRef, CondVar, CondVarRef,
parallel_for, WorkerPool, TypedPool,
OpaquePtr, i64_ptr, opaque_ptr,
)parallel_for fits a bounded set of indices: hand out tasks, drain the
queue, join. A server is the other shape — the work never runs out, and each
worker loops on its own until something says stop. WorkerPool is that shape:
spawn_n + AtomicFlag + ThreadGroup.join_all() bundled so callers stop
re-deriving the header layout and the free-after-join rule. It is deliberately
thin — no queue, no task type, no scheduler.
TypedPool is the same pool with the shared state typed, and it is the one to
reach for unless the context is a hand-laid-out block of cells:
from threads import AtomicCounter, AtomicFlag, TypedPool
def serve(worker: Int, mut app: App, stop: AtomicFlag) -> None:
while not stop.is_set():
app.handle_one_request()
def main() raises:
var app = App()
var pool = TypedPool.start[serve](n=8, state=app)
wait_for_sigint()
pool.shutdown() # request_stop + join
print(pool.state().served) # the state is still there, by constructionfrom threads import AtomicCounter, AtomicFlag, OpaquePtr, WorkerPool
def serve(worker: Int, ctx: OpaquePtr, stop: AtomicFlag) -> None:
while not stop.is_set():
handle_one_request(ctx) # whatever `ctx` addresses
def main() raises:
var pool = WorkerPool.start[serve](n=8, ctx=my_ctx)
wait_for_sigint()
pool.shutdown() # request_stop + join| item | signature | notes |
|---|---|---|
WorkerFn |
def(Int, OpaquePtr, AtomicFlag) thin -> None |
worker index, shared context, stop flag |
PoolTaskFn[T] |
def(Int, mut T, AtomicFlag) thin -> None |
the same, with the state typed |
WorkerPool.start |
start[work: WorkerFn](n, ctx) raises -> WorkerPool |
n >= 1 (a zero-worker pool is refused, not silently created); an n-less overload uses num_cpus() |
TypedPool.start |
start[task: PoolTaskFn[T]](n, ref state: T) raises -> TypedPool[T, origin] |
T and origin are inferred from state; same n >= 1 rule and same n-less overload |
.state() |
-> ref[origin] T |
TypedPool only; read totals after shutdown() without naming the local again |
.request_stop() |
-> None |
release-store to the flag; returns immediately |
.join() |
(mut self) raises |
joins every worker; does not set the flag |
.shutdown() |
(mut self) raises |
request_stop then join |
.num_workers() / .is_stopping() / .stop_flag() |
stop_flag() is a view, so a third party holding its address() can end the pool |
|
.pin_all() |
raises |
worker i to CPU i % num_cpus() (a no-op on macOS) |
Everything after start is identical on both, with the same names and
signatures, so moving between the two forms is a change of construction and
nothing else.
Workers are symmetric: every one runs the same body, and the index exists only so a worker can address its own slot in your context. It is drawn from an atomic counter at thread start, not baked in.
Two things about the stop flag that have surprised people, both of them consequences of it being cooperative rather than preemptive:
- A worker blocked in a syscall does not observe the flag until that syscall
returns. A thread parked in
recv,accept, or a blocking FFI call stays parked;request_stop()does not interrupt it. If your workers block, the thing they block on needs its own wake mechanism — a sentinel, a closed channel, a timeout — and the flag is only the second half of the handshake. (restate.mojo's served mode is the worked example: its Rust shim grew anrst_stopfor exactly this.) - A worker that never checks the flag never stops, so
join()never returns. Loop onnot stop.is_set().
Lifetimes are handled rather than documented-at: the pool heap-allocates one
small header (index counter, stop cell, user-context address), and its
destructor sets the flag, joins, and only then frees. Dropping a pool without
calling shutdown() is therefore safe — which matters, because Mojo destroys a
value at its last use, so a pool whose last mention is request_stop() is
destroyed on that line and the destructor's join is what keeps the header alive
under the still-running workers. Your ctx is not owned by the pool and must
outlive it; since the destructor joins, outliving the pool value is enough.
With WorkerPool that last sentence is a rule for you to keep: an OpaquePtr
carries no origin, so nothing checks it, and the failure is silent — build the
context out of a local, hand over its address, never mention the local again,
and it is destroyed on the line where you took the address, with the workers
still reading it.
TypedPool is the same sentence made a guarantee. parallel_for's typed
overload gets there by taking ref state, because the state is an argument and
the call joins before it returns; a pool outlives its start call, so a ref
argument alone buys nothing. So the origin rides out of start on the returned
value: TypedPool[T, origin] names it in its type and holds a
Pointer[T, origin] field. A live pool value is a live use of that origin, so
the compiler will not destroy state before the pool — and the pool's
destructor joins every worker before it returns. The
typed pool state outlives the pool test poisons the state in its destructor
and asserts that no worker ever read the poison; written against WorkerPool
and a hand-erased pointer, the same program segfaults.
| item | signature | notes |
|---|---|---|
parallel_for (typed) |
parallel_for[task: def(Int, mut T) thin -> None](n_tasks, ref state: T, num_workers=0) raises |
starts min(n_tasks, num_workers or num_cpus()) threads that pull indices off one shared atomic counter; joins all before returning; state is alive for the whole call and must be a mutable binding |
parallel_for (opaque) |
parallel_for[work: def(Int, OpaquePtr) thin -> None](n_tasks, ctx, num_workers=0) raises |
the same scheduler on a raw pointer; the caller keeps ctx alive |
spawn_n |
spawn_n[start](n, ctx, stride=0) raises -> ThreadGroup |
thread i gets ctx + i * stride; a partial failure joins what already started |
ThreadGroup |
.join_all(), .pin_all(), len() |
move-only; join_all attempts every join even if one fails |
The work queue is one shared counter, so scheduling is dynamic — a slow task does not stall the others — at a cost of one atomic RMW per task. Make tasks chunky: tens to thousands of tasks, not one per element.
| item | signature | notes |
|---|---|---|
ThreadHandle.spawn |
spawn[start: def(OpaquePtr) thin -> OpaquePtr](arg) raises -> ThreadHandle |
move-only handle over pthread_t |
ThreadHandle.join |
join(mut self) raises |
idempotent — zeroes the id, so a second call is a no-op instead of a second pthread_join on a stale id (which is UB, not an error) |
ThreadHandle.detach |
detach(mut self) raises |
leaves the handle inert |
ThreadHandle.pin_to_cpu |
pin_to_cpu(cpu) raises |
real on Linux (pthread_setaffinity_np); a documented no-op on macOS — Darwin exposes only an affinity hint, never a hard pin |
num_cpus |
-> Int |
sysconf(_SC_NPROCESSORS_ONLN), at least 1 |
current_thread_id |
-> UInt64 |
pthread_self |
yield_now |
-> None |
sched_yield, for spin loops |
Everything operates on a naturally aligned 64-bit cell at an address you
own — atomics are only interesting between threads, and threads only get a
raw pointer, so the useful shape is a view over shared memory rather than a
value you hold. AtomicCounter and AtomicFlag are both views: copying one
copies the view, not the cell.
| item | notes |
|---|---|
AtomicCounter |
.fetch_add(delta=1), .load(), .store(v), .at(address), .at(ctx, slot), .alloc(initial) / .unsafe_free() |
AtomicFlag |
.set() (release), .is_set() (acquire), .clear(), .set_value(v) / .raw() for the flag-as-error-code pattern |
| free functions | atomic_fetch_add, atomic_load_relaxed, atomic_load_acquire, atomic_store_relaxed, atomic_store_release, atomic_fence_acquire, atomic_fence_release |
| item | notes |
|---|---|
Mutex |
owns a 64-byte pthread_mutex_t blob; move-only; .lock(), .unlock(), .try_lock(), .with_lock[body](ctx), .address(), .ref() |
MutexRef |
copyable non-owning view; what a worker rebuilds from an address in its context |
CondVar / CondVarRef |
the same split over pthread_cond_t: .wait(mutex), .signal(), .broadcast() |
There is deliberately no RAII lock guard. Mojo destroys a value at its last
use, so a guard whose only job is to unlock in its destructor would unlock at
the point you stop mentioning it — usually the first line of the critical
section. That is a silent correctness bug rather than a compile error. Use
lock()/unlock(), or with_lock, which brackets a thin function properly.
It does, now — but not from a single file, because two toolchains this tin
has had to support disagree about what Atomic is:
| toolchain | declaration | the only spelling it accepts |
|---|---|---|
| Mojo 1.0.0 | struct Atomic[dtype: DType, ...] |
Atomic[DType.int64] |
| nightly | struct Atomic[T: Deinitable & Movable, ...] |
Atomic[Int64] |
Neither spelling compiles on the other compiler, Mojo exposes no
compiler-version constant to branch on, and a type cannot be chosen by a
comptime if at module scope in any case. Everything else is identical:
fetch_add[ordering=…], load[ordering=…], store[ordering=…] and fence
compile unchanged against either declaration. So the divergence is confined to
one line, in a file the include path selects:
# compat/stable/threads_compat.mojo # compat/nightly/threads_compat.mojo
comptime Cell = Atomic[DType.int64] # comptime Cell = Atomic[Int64]Bridging that gap is half of what this tin is for: your code writes one
spelling and keeps compiling across the split. The cost is the extra -I in
Install.
Until September 2026 this was solved one level down instead, by reaching past
std.atomic to the pop.atomic.rmw / pop.load / pop.store / pop.fence
intrinsics it is itself written on top of, on the theory that a compiler
primitive would be steadier than a stdlib signature. It was not: nightlies from
26.6.0.dev2026090105 onward reject the __mlir_attr syntax those calls need
(modular/modular#7094), and
because threads.atomic was the only file in all of magmalake touching raw
MLIR, it was single-handedly holding every tin downstream of it below that
nightly. There is no raw MLIR left in this repo.
Expect compat/ to grow rather than shrink — 1.0 and nightly are diverging,
and async is the next thing this tin will want that they spell differently.
The rule for anything added there is the rule the Cell split follows: the
divergent file holds the declaration that differs and nothing else, so the
entire delta between the two toolchains fits on a screen.
Two smaller portability notes baked in for the same reason: Pointer is
non-nullable on both toolchains now (constraint failed: Pointer is non-nullable), so C NULL is passed as a plain Int — identical in the
register under both the SysV and AAPCS64 ABIs. And List iteration requires a
Copyable element on both, so ThreadGroup iterates its move-only handles by
index.
threads-mojo bench, on an Apple M4 (10 logical CPUs, Mojo 1.0.0):
parallel sum over 100000000 Int64 parallel memcpy of 762 MiB
workers ms speedup workers ms speedup GB/s
1 32.03 1.00x 1 75.96 1.00x 21.06
2 19.21 1.67x 2 25.25 3.01x 63.36
4 13.50 2.37x 4 18.95 4.01x 84.45
8 8.82 3.63x 8 18.46 4.12x 86.68
10 8.66 3.70x 10 18.25 4.16x 87.68
The memcpy row is in the demo on purpose. It is bandwidth bound, so it stops scaling at four workers no matter how many cores you throw at it — about 4× is the ceiling for anything memory-bound on this machine. A parallel-for demo that only shows a straight line is measuring the wrong thing.
Overheads, measured on the same machine: ~14 µs per spawn+join round trip
(500 sequential threads in 6.9 ms), ~2.4 ns per uncontended fetch_add,
and 14–21 ns per fetch_add under 8-way contention on one cache line
(800,000 of them in 11–17 ms across repeated runs, on both toolchains). That
last spread is why parallel_for wants chunky tasks: the counter is a shared
cache line, and every task costs one trip through it.
The point of the tin is other people's inner loops, so here is one. Using an
unmodified parquet.mojo
checkout on the include path, decoding a 1M-row / 4-column Parquet file one
row group per parallel_for task:
| file | workers | ms | speedup | rows/s |
|---|---|---|---|---|
| 1M rows, 4 row groups | 1 | 5.02 | 1.00× | 199 M |
| 2 | 2.84 | 1.77× | 352 M | |
| 4 | 2.07 | 2.43× | 484 M | |
| 1M rows, 28 row groups | 1 | 4.49 | 1.00× | 223 M |
| 2 | 2.54 | 1.77× | 394 M | |
| 4 | 1.82 | 2.47× | 549 M | |
| 8 | 1.52 | 2.96× | 660 M | |
| 16 | 1.59 | 2.83× | 629 M |
Method, so you can weigh it. Apple M4, 10 logical CPUs, Mojo 1.0.0, best of
5 runs. The file is parquet.mojo's own build/bench-wide.parquet
(1,000,000 rows; int64, double, dictionary-encoded int64, dictionary-
encoded string; UNCOMPRESSED; 4 row groups of 250k). The 28-row-group variant
is the same data rewritten through parquet.mojo's writer with
row_group_size = 62500, also UNCOMPRESSED, in a scratch directory. One
ParquetReader per row group is built outside the timed region, each over
its own copy of the file bytes and restricted with select_row_groups; the
timed region is parallel_for over read_table(), and every run checks that
the decoded row count still sums to 1,000,000. Both the 1-worker and the
N-worker rows go through the same parallel_for path, so the comparison is
worker count and nothing else.
Read it honestly. 2.96× is not 8×, and it should not be: parquet.mojo's
decoder already runs at 232 M rows/s on one core and is largely moving bytes,
so it hits the same memory ceiling the memcpy column above shows at ~4×. Going
past 8 workers gets slower — 28 row groups over 16 threads is more contention
than parallelism. The experiment lives in a scratch directory and changed
nothing in parquet.mojo at the time. It is no longer the only way to get this:
parquet.mojo now threads its own reads, across row groups and column chunks
together, behind num_workers.
Four tins in the org build on it, which is the tin's own regression suite as much as anything:
- parquet.mojo —
num_cpusandparallel_forbehindnum_workers, fanning a read out over (row group, column chunk) pairs and assembling Arrow on the workers. - iceberg.mojo —
parallel_forover a scan's files, behindScanOptions.num_workers. - postgres.mojo —
MutexandCondVarunder a connection pool, so a checkout blocks on a condition variable rather than spinning. - restate.mojo — a
TypedPoolof serve workers sharing an origin-tracked handler state.
$ pixi run test # Mojo 1.3.0.dev2026111805The tests run on both platforms. The load-bearing ones are the races, each written so a broken primitive produces a wrong number rather than a flake:
AtomicCounter— 8 threads × 100,000fetch_addmust equal exactly 800,000. A non-atomic read-modify-write loses updates here every run.AtomicFlag— a writer does a plain write and then a release store; four spinning readers acquire-load the flag and must all see the payload.Mutex— 8 threads × 10,000 unguarded-looking increments of a plainIntmust equal exactly 80,000; and a separate timing test that the lock really blocks (hold for 50 ms, assert the waiter waited > 10 ms).- Allocator under threads — 8 workers building and dropping Mojo
Lists andStrings concurrently, 64 tasks × 20 rounds each. Every worker in this library rests on the Mojo runtime allocator being usable from several threads at once; this test is the evidence, and it passes on both CI legs. WorkerPool— nine tests: workers that tick a shared counter until stopped and then join;request_stop()racing the spawns; the 16 indices each claimed exactly once (a duplicate leaves one slot at 2 and another at 0);n = 1;n < 1refused; a worker that fails and returns without ever consulting the flag still joins; a pool destroyed withoutshutdown()still stops, joins, and frees in that order; and a third party setting the flag throughstop_flag().address().TypedPool— six tests: the typed spellings of the tick, unique-indices andn = 1cases;n < 1still refused through the wrapper; a pool dropped rather than shut down; and the one the design exists for — a state whose destructor poisons its heap cell to -1, started over a local that is never named again, with the workers counting every poisoned read they see. The count must be 0.- Stress — 500 sequential spawn/joins, 50 back-to-back
parallel_forrounds, 50WorkerPools started and shut down back to back.
A test that gets the right number proves the number, not the ordering: a race that happens to interleave benignly on this machine, this run, still passes. So the suite has a second half that is a detector rather than an assertion.
$ pixi run stress # no sanitizer, 300 rounds
$ pixi run stress-tsan # --sanitize thread, zero tolerance
$ pixi run stress-asan # --sanitize address, + LeakSanitizertests/stress_threads.mojo runs the typed and opaque parallel_for at 1, 2,
4, 10 and an oversubscribed 64 workers, starts and stops 50 TypedPools and
50 WorkerPools — alternating shutdown() with dropping the pool, so the
destructor's stop/join path is covered as often as the explicit one — and
hammers one AtomicCounter from every core. TSan only reasons about code that
actually ran, so the sweep is the point. Every phase also checks its
arithmetic, because a lost update must fail even on a run where the sanitizer
saw nothing.
These are zero-tolerance gates: tests/run_stress.sh greps the run for
WARNING: ThreadSanitizer / ERROR: AddressSanitizer / ERROR: LeakSanitizer and fails the task on any hit, rather than trusting an exit code
that has moved between sanitizer versions. It also puts a POSIX-sh watchdog
around the run — macOS ships no timeout(1) — so a hung join fails in minutes
instead of eating a runner, and the watchdog self-tests against sleep on
every invocation.
The gate is only worth having because the library is clean under it: the
atomics here lower to instructions TSan models, so a correct program produces
zero warnings and any warning is a real finding. The same build on a
deliberately racy task — a plain t.sum = t.sum + i in a parallel_for body —
reports the race and names both accesses in _parallel_worker.
Which sanitizer runs where is the toolchain's decision, not a preference. With Mojo 1.0.0:
| osx-arm64 | linux-64 | |
|---|---|---|
--sanitize thread |
works | links, then aborts before main |
--sanitize address |
does not link | works, LeakSanitizer included |
On linux-64 a TSan binary dies in startup with MmapAligned() failed … TCMalloc assumes a 48-bit virtual address space — the runtime's bundled TCMalloc cannot
find a 1 GiB-aligned mapping inside what TSan has reserved. A hello-world fails
identically and disabling ASLR does not help, so it is the toolchain rather
than any program. On osx-arm64 an ASan build fails to link with undefined
___asan_version_mismatch_check_v8, with or without --shared-libasan. CI
therefore runs stress on both platforms, stress-tsan on macOS and
stress-asan on Linux, which between them covers both tools.
The threading primitives here are distilled from
ehsanmok/flare's runtime (MIT) — its
ThreadHandle shape, its opaque pthread_mutex_t blob, the heap-allocated
worker context that must outlive the join, and its hard-won note that the
scheduler's stop flag really wanted to be an atomic. The code is rewritten for
current Mojo rather than copied: flare pins mojo==1.0.0b1, and this tin is
pinned to a Mojo 1.3 nightly.
Licensed under the Apache License, Version 2.0 — see LICENSE and NOTICE.