Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,3 +31,8 @@ jobs:
grep -q 'sum: 499500' origins.txt
pixi run run-post | tee post.txt
grep -q 'sum: 499500' post.txt

# Deliberate misuse: four cases the compiler must reject, three it must
# still accept (and get wrong in the documented way).
- name: Caught and uncaught
run: pixi run check
30 changes: 30 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,36 @@ which wrote the `threads-mojo = { git = …, rev = … }` line in
[`pixi.toml`](pixi.toml). magmalake tins are not on a conda channel, so
`pixi add threads-mojo` would find nothing.

## What the compiler catches, and what it does not

`pixi run check` builds seven deliberately wrong programs under
[`tests/`](tests/) and holds the compiler to a claim about each one.

[`tests/caught/`](tests/caught/) must fail to compile, with the diagnostic the
file names:

| file | misuse | diagnostic |
| --- | --- | --- |
| `share_immutable` | `share(x)` on a `read` argument | `cannot be converted from 'Totals' to ref 'Totals'` |
| `share_temporary` | `share(Totals(0))` | same |
| `return_ctx_to_local` | `return share(local)` | `cannot implicitly convert 'Ctx[Totals, origin_of(t)]' value to 'Ctx[Totals]'` |
| `ctx_outlives_block` | a `Ctx` assigned in a block, used after it | same |

[`tests/uncaught/`](tests/uncaught/) must compile with no diagnostic at all,
and then print the wrong answer the file predicts:

| file | what goes wrong | output |
| --- | --- | --- |
| `untracked_ctx_drops_early` | the post's `Ctx` erases the origin; `totals` is destroyed at its last visible use, before any thread starts | `Totals dropped` printed first; sum `499499` (−1 + 499500, the tasks adding into the poisoned cell) |
| `opaque_escapes_origin` | `share(totals).opaque()` outside `run` — the boundary of what origins cover | same |
| `field_deref_after_last_use` | `totals.cell[]` copies the pointer field, which is the struct's last use, and the deref reads a destroyed object | `-1`; the method read `totals.sum()` gives `499500` |

The uncaught set is the list a reviewer has to check by hand. If a newer
compiler starts rejecting one, `check` fails on it — move the file to
`caught/`, record the diagnostic, and the list gets shorter. The destructor in
those tests poisons the cell rather than freeing it, so the misuse stays a
number that can be asserted instead of undefined behaviour.

## Comments

The post has no comment box; this repo is where the code can be discussed.
Expand Down
4 changes: 3 additions & 1 deletion pixi.toml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,9 @@ preview = ["pixi-build"]
run = { cmd = "mkdir -p build && mojo build src/origins.mojo -o build/origins && ./build/origins" }
# … and exactly as the post prints it (src/post.mojo).
run-post = { cmd = "mkdir -p build && mojo build src/post.mojo -o build/post && ./build/post" }
fmt = { cmd = "mojo format src" }
# Deliberate misuse: what the compiler rejects, and what it lets through.
check = { cmd = "bash tests/check.sh" }
fmt = { cmd = "mojo format src tests" }

[dependencies]
mojo = "==1.0.0"
Expand Down
13 changes: 13 additions & 0 deletions tests/caught/ctx_outlives_block.mojo
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Same refusal, one scope down: `t` dies with the block, `c` does not.
# expect-error: cannot implicitly convert 'Ctx[Totals, origin_of(t)]' value to 'Ctx[Totals]'
from origins import Ctx, Totals, run, share, task


def main() raises:
var c: Ctx[Totals]
var n = 1
if n == 1:
var t = Totals(0)
c = share(t)
run[task](1000, c)
print(c[].sum)
15 changes: 15 additions & 0 deletions tests/caught/return_ctx_to_local.mojo
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# `share(t)` has type `Ctx[Totals, origin_of(t)]`, and the compiler will not
# quietly widen that to the untracked origin on the way out of the function.
# expect-error: cannot implicitly convert 'Ctx[Totals, origin_of(t)]' value to 'Ctx[Totals]'
from origins import Ctx, Totals, run, share, task


def make() -> Ctx[Totals]:
var t = Totals(0)
return share(t)


def main() raises:
var c = make()
run[task](1000, c)
print(c[].sum)
12 changes: 12 additions & 0 deletions tests/caught/share_immutable.mojo
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# A `read` argument has an immutable origin; `share` demands a mutable one.
# expect-error: cannot be converted from 'Totals' to ref 'Totals'
from origins import Totals, run, share, task


def sum_of(totals: Totals) raises -> Int64:
run[task](1000, share(totals))
return totals.sum


def main() raises:
print(sum_of(Totals(0)))
9 changes: 9 additions & 0 deletions tests/caught/share_temporary.mojo
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# A temporary dies at the end of the statement; there is nothing to share.
# expect-error: cannot be converted from 'Totals' to ref 'Totals'
from origins import Ctx, Totals, run, share, task


def main() raises:
var c = share(Totals(0))
run[task](1000, c)
print(c[].sum)
80 changes: 80 additions & 0 deletions tests/check.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
#!/usr/bin/env bash
# Deliberate misuse, sorted by what the compiler does with it.
#
# tests/caught/*.mojo must FAIL to build, and the error text must contain
# the file's `# expect-error:` line.
# tests/uncaught/*.mojo must build with no diagnostics at all, run, and
# print the file's `# expect:` lines in that order —
# the wrong answer, reproduced on purpose.
#
# Both directions are a claim about the compiler. If a caught case starts
# compiling, a guarantee the README states has gone. If an uncaught case
# stops compiling, the compiler has learned to see something it could not —
# move the file to caught/, record the diagnostic, and shorten the list of
# things a reviewer has to check by hand.
set -u
cd "$(dirname "$0")/.."
command -v mojo >/dev/null || { echo "mojo not on PATH — run as: pixi run check"; exit 2; }
mkdir -p build/tests
fail=0
pass=0

expect_lines() { # $1 file, $2 marker → the text after each "# <marker>: "
sed -n "s/^# $2: //p" "$1"
}

# in_order OUTPUT EXPECTED — every expected line occurs in OUTPUT, each one
# strictly after the previous match.
in_order() {
local out=$1 cursor=0 hit
while IFS= read -r want; do
hit=$(awk -v n="$cursor" -v s="$want" 'NR > n && index($0, s) { print NR; exit }' <<<"$out")
[ -n "$hit" ] || return 1
cursor=$hit
done <<<"$2"
return 0
}

for f in tests/caught/*.mojo; do
name=$(basename "$f" .mojo)
want=$(expect_lines "$f" expect-error)
if out=$(mojo build "$f" -I src -o "build/tests/$name" 2>&1); then
echo "FAIL caught/$name: compiled; expected an error containing: $want"
fail=$((fail + 1))
elif ! grep -qF -- "$want" <<<"$out"; then
echo "FAIL caught/$name: failed for a different reason; wanted: $want"
echo "$out" | grep 'error:' | head -3
fail=$((fail + 1))
else
echo "ok caught/$name"
pass=$((pass + 1))
fi
done

for f in tests/uncaught/*.mojo; do
name=$(basename "$f" .mojo)
if ! out=$(mojo build "$f" -I src -o "build/tests/$name" 2>&1); then
echo "FAIL uncaught/$name: no longer compiles — the compiler may have caught it:"
echo "$out" | grep 'error:' | head -3
fail=$((fail + 1))
continue
fi
if grep -q 'warning:' <<<"$out"; then
echo "FAIL uncaught/$name: compiled with a diagnostic:"
echo "$out" | grep 'warning:' | head -3
fail=$((fail + 1))
continue
fi
got=$("build/tests/$name")
if in_order "$got" "$(expect_lines "$f" expect)"; then
echo "ok uncaught/$name"
pass=$((pass + 1))
else
echo "FAIL uncaught/$name: output did not match in order. Got:"
echo "$got" | sed 's/^/ /'
fail=$((fail + 1))
fi
done

echo "$pass passed, $fail failed"
[ "$fail" = 0 ]
45 changes: 45 additions & 0 deletions tests/uncaught/field_deref_after_last_use.mojo
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# Nothing to do with threads. `totals.cell[]` copies the pointer field, and
# that copy is the struct's last use — so `totals` is destroyed between the
# copy and the deref, and the deref reads the poisoned cell. A method that
# reads through `self` borrows the whole struct for the call and gets the
# right number.
# expect: via method: 499500
# expect: Totals dropped
# expect: via field deref: -1
from std.memory.alloc import unsafe_alloc
from origins import Ctx, run, share
from threads import AtomicCounter, OpaquePtr


struct Totals(Movable):
"""Owns one heap cell and announces its own destruction.

`__deinit__` poisons the cell to -1 instead of freeing it: a write into a
freed block is undefined behaviour and corrupts the heap on some runs,
while a write into a poisoned, leaked block is a number we can assert.
"""

var cell: Pointer[Int64, MutUntrackedOrigin]

def __init__(out self):
self.cell = unsafe_alloc[Int64](1)
self.cell[] = 0

def sum(self) -> Int64:
return self.cell[]

def __deinit__(deinit self):
print("Totals dropped")
self.cell[] = -1


def task(i: Int, ptr: OpaquePtr) -> None:
var t = Ctx[Totals].of(ptr)
_ = AtomicCounter.at(Int(t[].cell)).fetch_add(Int64(i))


def main() raises:
var totals = Totals()
run[task](1000, share(totals))
print("via method:", totals.sum())
print("via field deref:", totals.cell[])
42 changes: 42 additions & 0 deletions tests/uncaught/opaque_escapes_origin.mojo
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# The tracked `Ctx` fixes the drop — until `opaque()` is called outside `run`.
# The `void *` carries no origin, the temporary `Ctx` dies with the statement,
# and `totals` goes with it. This is the boundary of what origins can cover.
# expect: Totals dropped
# expect: before parallel_for
# expect: after parallel_for: 499499
from std.memory.alloc import unsafe_alloc
from origins import Ctx, share
from threads import AtomicCounter, OpaquePtr, parallel_for


struct Totals(Movable):
"""Owns one heap cell and announces its own destruction.

`__deinit__` poisons the cell to -1 instead of freeing it: a write into a
freed block is undefined behaviour and corrupts the heap on some runs,
while a write into a poisoned, leaked block is a number we can assert.
"""

var cell: Pointer[Int64, MutUntrackedOrigin]

def __init__(out self):
self.cell = unsafe_alloc[Int64](1)
self.cell[] = 0

def __deinit__(deinit self):
print("Totals dropped")
self.cell[] = -1


def task(i: Int, ptr: OpaquePtr) -> None:
var t = Ctx[Totals].of(ptr)
_ = AtomicCounter.at(Int(t[].cell)).fetch_add(Int64(i))


def main() raises:
var totals = Totals()
var ptr = share(totals).opaque()
print("before parallel_for")
parallel_for[task](1000, ptr)
# Not `totals.cell` — a use of `totals` here would move the drop past it.
print("after parallel_for:", Ctx[Totals].of(ptr)[].cell[])
67 changes: 67 additions & 0 deletions tests/uncaught/untracked_ctx_drops_early.mojo
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
# The post's `Ctx` erases the origin, so the compiler destroys `totals` at its
# last visible use — `Ctx.to(totals)` — before a single thread has started.
# Compiles with no diagnostic. Expected here: the drop printed FIRST, and the
# thousand additions landing on the poisoned cell: -1 + 499500.
# expect: Totals dropped
# expect: before parallel_for
# expect: after parallel_for: 499499
from std.memory.alloc import unsafe_alloc
from threads import AtomicCounter, OpaquePtr, parallel_for


@fieldwise_init
struct Ctx[T: AnyType](Copyable, Movable):
var _ptr: Pointer[Self.T, MutUntrackedOrigin]

@staticmethod
def to(ref state: Self.T) -> Self:
return Self(
Pointer[Self.T, MutUntrackedOrigin](
unsafe_from_address=Int(Pointer(to=state))
)
)

@staticmethod
def of(ptr: OpaquePtr) -> Self:
return Self(
Pointer[Self.T, MutUntrackedOrigin](unsafe_from_address=Int(ptr))
)

def __getitem__(self) -> ref[MutUntrackedOrigin] Self.T:
return self._ptr[]

def opaque(self) -> OpaquePtr:
return OpaquePtr(unsafe_from_address=Int(self._ptr))


struct Totals(Movable):
"""Owns one heap cell and announces its own destruction.

`__deinit__` poisons the cell to -1 instead of freeing it: a write into a
freed block is undefined behaviour and corrupts the heap on some runs,
while a write into a poisoned, leaked block is a number we can assert.
"""

var cell: Pointer[Int64, MutUntrackedOrigin]

def __init__(out self):
self.cell = unsafe_alloc[Int64](1)
self.cell[] = 0

def __deinit__(deinit self):
print("Totals dropped")
self.cell[] = -1


def task(i: Int, ptr: OpaquePtr) -> None:
var t = Ctx[Totals].of(ptr)
_ = AtomicCounter.at(Int(t[].cell)).fetch_add(Int64(i))


def main() raises:
var totals = Totals()
var ctx = Ctx[Totals].to(totals).opaque()
print("before parallel_for")
parallel_for[task](1000, ctx)
# Not `totals.cell` — a use of `totals` here would move the drop past it.
print("after parallel_for:", Ctx[Totals].of(ctx)[].cell[])
Loading