Skip to content

feat(compiler): make outputs write-only with live input references - #104

Open
thiremani wants to merge 16 commits into
masterfrom
feat-write-only-outputs
Open

thiremani wants to merge 16 commits into
masterfrom
feat-write-only-outputs

Conversation

@thiremani

@thiremani thiremani commented Sep 11, 2026

Copy link
Copy Markdown
Owner

Declared outputs are write-only inside their template. A caller can explicitly share an input with an output by reusing the same binding; later input reads observe earlier output writes in both ordinary and ranged calls. This replaces the seed-read analysis proposed in the closed #102 and fixes #103.

Behavior

For out, seen = Fold(current, item), statement order determines what seen receives:

Body order Plain call from 10 with item 5 Ranged call from 10 with items 1:3
seen = current, then out = current + item 15 10 13 11
out = current + item, then seen = current 15 15 13 13

An unaliased input keeps its own value. Sharing stays inside the call's staged result slots: sibling right-hand sides still read the caller's pre-assignment values. Empty ranges and skipped writes preserve existing destinations; fresh destinations start at zero.

What master did. Ordinary calls returned 15 10 for both body orders, while ranged calls returned 13 13 for the output-first body and 15 15 for a one-element range 5:6: the same read observed different values depending on whether the argument was 5 or a range containing only 5. Master's documented rule was the pre-call snapshot for ordinary calls; the live behavior in ranged variants was an artifact of routing the input to the output's staged slot. This PR removes that inconsistency by making every call follow the live-reference rule; the earlier commits on this branch had resolved it the other way, toward the snapshot.

Contract. Inputs cannot be assigned through their input names. Writing a shared output changes what subsequent input reads observe. Every simultaneous assignment evaluates its reads before applying its writes, inside a body as at the call site, so out, before = current + item, current still gives before the prior value. Keeping an old value across a write is an explicit assignment that creates an independent value; any copy it costs sits at that assignment.

With two shared inputs, sequential a = y then b = x produces 2 2 from 1 2. Simultaneous a, b = y, x swaps them to 2 1.

Output-name reads remain errors, including conditions, arguments, prints, and formatting markers. Inputs are read-only bindings, so %n writes into an input or iterator parameter are now rejected; acc_fmt uses a writable local copy.

Implementation and migration

  • Remove per-iteration input snapshots. Whether an argument shares a binding with a destination is decided at compile time from the call's names and lowered as a private variant of the specialization (internal linkage) named with the mangling scheme's marker-plus-count suffixes, _aN_<slots> for the alias pattern and _oN_<types> for widened output storage, both demangled by Demangle: inside it a direct scalar input reads the output's current value and a compatible indirect input receives the staged output pointer. Nested calls and caller-driven ranges forward sharing by name through the same alias bindings, so there are no hidden arguments, no run-time selects, and no pointer comparisons.
  • Specialize binding arguments on settled storage types and revisit calls after storage widening. Private output-storage variants preserve sharing across compatible ownership or array-shape representations. This fixes the static-to-heap and nested refinement cases in Call arguments are specialized on the binding's flow type, not its merged slot type #103.
  • CFG liveness is exact per alias context. Settlement analyzes each type specialization's unshared context; the script walk derives each call's context from names, forwards it through nested calls by the rule lowering uses (one shared function, aliasPattern, with the body's outputs bound to the storage the enclosing call widened them to), visits each lowered variant once, analyzes shared contexts on first reach, and caches them on the specialization by alias pattern. A read of a shared input counts as a read of its output's latest write, so out = current + item written twice is accepted when called as value = BumpTwice(value, 5) and reported as an unused overwrite when called as other = BumpTwice(value, 5). Local copies retain their own binding permissions.
  • Native ABI 2.1: exported prototypes carry no alias selectors: a direct-return function is (params..., seed) and an indirect-return function keeps its leading result carrier followed by params..., with no seed. Functions without a range parameter are unchanged from master. Range-bearing functions on master carried a hidden i32 alias selector per direct scalar parameter before the seed; those selectors are removed, so their prototype changes and the seed moves one position earlier. This is a documented break in the C ABI spec ("Changes in 2.1"); there are no shipped native callers. An intermediate revision of this PR had instead added selectors to every function and resolved aliasing at run time, which cost about 10% on fib_tail.
  • Native pointer sharing holds within the called body's own statements; a nested Pluto call inside it stages its outputs, so it does not extend the sharing to a native caller. Pluto callers get sharing across nested calls statically. Documented in the spec; a generic pointer entry that resolves unknown sharing at run time, beside the private variants, is recorded as outstanding in the effects plan.
  • Templates that previously read output names use locals or single expressions. The Rebuild rewrite in this PR changes [26] to [37] and [2] to [6]. Live sharing also changes mem_alias_refine from Sibling: hello! to Sibling: static!.
  • Preserve existing LLVM loop metadata when adding unroll hints (separate commit). It was required while the selector signatures changed the optimized Fib loop's shape; with the original prototypes restored it is a robustness fix and stays.

The memory model, README, ABI specifications, IR plan, and effects plan describe the final rule. The effects plan distinguishes the implemented %n input restriction from pending write-effect modeling.

Verification

  • Race-enabled lexer, parser, compiler, and root tests pass; go vet ./... passes.
  • Full suite: 76 passed, 0 failed; all 75 runtime leak checks report zero leaks.
  • Existing fixtures cover both Fold orders, ownership, nested sharing, conditional writes, array resets, and caller-side staging. The only additional behavior pair from review is sequential versus simultaneous swap.
  • IR tests cover the array-copy regressions, the alias variant reading its output's storage, an unaliased call using the public specialization, and a variant skipping an incompatible output; metadata tests cover preservation, explicit policies, and idempotence.
  • Benchmarks against master (bench repo fib, fib_tail, harmonic; median of 10 with 5 warm-ups, native CPU, output parity checked): fib 0.97x, fib_tail 1.01x, harmonic 1.01x. With the earlier run-time selectors fib_tail was 1.10x slower (stable across three re-runs); restoring the plain prototype through compile-time variants recovered it.

thiremani and others added 6 commits September 11, 2026 21:31
…-stable

Declared outputs can no longer be read inside their template: values,
conditions, call arguments, prints, and formatting markers (including
dynamic width and precision) are rejected with
`output "y" is read inside its function; outputs are write-only, use a
local`. A body transforms inputs into outputs and never observes an
output's value, so the incoming destination seed can never leak in; the
seed and destination-seeded staging slots stay as an unobservable
keep-old carrier and the ABI is unchanged. This supersedes the seed-read
effect analysis of the closed #102: the reproducer `y = x > 0 x` then
`y = y + 1` is now a template error.

Range-bearing variants snapshot every non-iterator input at the start of
each scalar iteration. A direct scalar aliased to an output reads the
carried output once; an indirect input the body reads after an output
write keeps a private copy for the iteration. Before, an aliased input
read the output's storage on every read, so `out = current + item` then
`seen = current` observed the new value: `value, before = Fold(value,
1:3)` printed `13 13` and now prints `13 11`, and a ranged heap-string
accumulator prints `abc ab` instead of `ac`. The promoted-alias path is
deleted; nothing reaches it once inputs are snapshotted.

Fixtures that read an output are rewritten with locals or single
expressions. `Rebuild` changes from reading its freshly written output
to reading the previous iteration's result (`[26]` -> `[37]`, `[2]` ->
`[6]`), and `shareStaticOutput(0)` no longer blanks its second output.
The flow-versus-slot call specialization gap found in review is #103.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
snapshotIterationInputs copied every indirect input read after an
output write, even one no output could share storage with, so
`count, value = Read(data, 0:100000)` with integer outputs copied and
freed the whole array on every iteration: 2.2s for the probe against
milliseconds on master. The caller aliases an input to an output only
when the two types lower identically (setCallArgAliasSelectors), so
mirror that check before allocating a snapshot. The probe is back under
the timer's resolution and its IR carries no arr_i64_copy; a heap-string
input read after a heap-string output write still copies per iteration.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The structural CFG recorded which inputs a template reads after its
first output write so lowering could skip the per-iteration copy for
inputs read only before it. Measurement shows ranged string
accumulation is quadratic with or without that copy, because today's
lowering allocates a fresh string every iteration; the amortized
in-place append is planned, not implemented. The fact therefore bought
about a 2x constant on an already-quadratic path (0.99s to 1.97s at
160000 appends) while making correctness depend on a read-ordering
analysis being complete.

Remove the fact and its plumbing from the CFG, CodeCompiler, and
FuncArgs. An indirect input that some output could back is now copied
at the start of every scalar iteration; direct scalars still re-read
the carried output once. Inputs no output can alias are still left in
place. Linear accumulation will come from consuming the input on its
last use when carried appends land.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Live input alias selectors change the optimized Fib tail loop so its latch
already carries llvm.loop.peeled.count. Preserve existing loop properties
when adding scalar unroll hints, while respecting explicit unroll policies.

Read metadata operands through a native C pointer buffer rather than relying
on the layout of llvm.Value. Existing metadata tests cover preservation,
policy handling, and idempotence.
Keep outputs write-only while allowing explicitly aliased inputs to observe
earlier output writes in ordinary and ranged calls. Remove iteration
snapshots, preserve nested reference identity, and specialize calls on
settled binding storage, including compatible wider output slots.

Reject formatting %n writes to input and iterator parameters; writable local
copies retain their own permissions. Document the live-reference rule and
cover both Fold orders and sequential versus simultaneous swaps without
adding duplicate test combinations.

BREAKING CHANGE: native ABI 2.1 adds an i32 alias selector for every direct
scalar parameter in ordinary as well as ranged variants. Native callers
must supply zero for inputs that do not alias an output.

Fixes #103.
@thiremani thiremani changed the title feat(compiler): make function outputs write-only and inputs iteration-stable feat(compiler): make outputs write-only with live input references Sep 12, 2026
thiremani and others added 10 commits September 12, 2026 18:32
Reads precede writes in every simultaneous assignment, inside a body as at
the call site, so a shared input still yields its prior value within the
statement that writes the output. Keeping an old value across a write is
an explicit assignment; that is where any copy is paid.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Whether a call's argument shares a binding with one of its destinations is
known from the call's names, so it no longer travels as a hidden i32 alias
selector on every direct scalar parameter. A call whose argument names its
own destination lowers to a private variant of the specialization,
`<mangled>$alias$<pattern>` with internal linkage, in which a direct scalar
input reads the output's current value and a compatible indirect input
receives the staged output pointer. Nested calls and caller-driven ranges
forward the sharing by name through the same alias bindings, replacing the
run-time selects and pointer comparisons.

The exported prototypes return to ABI 2.0, `(params..., seed)`, and the
plain variant's IR matches master's, which recovers the 10% loss the
selectors had caused on fib_tail: master-relative timings are now fib
0.97x, fib_tail 1.01x, harmonic 1.01x. Behavior is unchanged; every alias
fixture prints the same output.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The private alias and output-storage variants were named with `$`, which
LLVM accepts but which is not a C identifier character and did not follow
the mangling scheme. Name them with the scheme's lowercase-marker-plus-
count form, alongside `_fN`, `_tN`, and the reserved `_cN`: `_oN_<types>`
lists every output slot's storage type for a widened-storage variant and
`_aN_<slot>...` carries the per-parameter alias pattern, `_oN` before
`_aN` when both apply. MangleVariant builds the suffixes, Demangle parses
them back into OutputStorage and AliasPattern and renders them as
`-> (StrH, StrH)` and `[in1->out1]`, and the C ABI spec gains §5.2 plus
grammar rules for VariantSym. Emitted symbols change; behavior does not.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…onals

Two lowerings replaced the name a shared input was recorded under, so a
nested call inside them selected the unshared variant. Ranged staging
rebinds an aliased input to the staged output slot, which no longer
matched paramAlias.Base; `out, seen = Fold(current, (1:3) + 0)` in a
wrapper called with a shared input gave `13 11` instead of `13 13`, and
a second ranged call in the same body gave `20 16` instead of `20 20`.
Conditional lowering targets synthetic `$c_cond_<name>` destinations
that never equal the recorded output name; `item > 0 Fold(current, item)`
gave `15 10` instead of `15 15`.

A parameter may now carry several alias bases, and ranged staging
registers the staged slot as one. Conditional temps record the source
destination they commit into, and call-site aliasing resolves through
that map before comparing names. Fixtures cover both ranged shapes,
arrays, and the conditional call taken and skipped.

The C ABI spec is versioned to 2.1: master's range-bearing variants
carried hidden alias selectors before the seed, and removing them moves
the seed, so those prototypes change. Native pointer sharing is scoped
to the called body's own statements; a nested Pluto call stages its
outputs and does not extend it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Only direct-return functions end in a seed; indirect returns keep their
leading result carrier and have none. The effects plan and the ABI
optimization plan no longer claim the calling convention is unchanged:
range-bearing prototypes change in 2.1, and the generic pointer entry for
native callers with unknown sharing is recorded as outstanding. The alias
fixture gains the two-level array wrapper from review.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…tor wording

NestedArrayRange now stages the ranged call and makes its nested call
inside that loop, which is the shape that lost the alias before ceb8a89:
it prints `[10 1 2] [10 1]` on 8c862a2 and the expected `[10 1 2]
[10 1 2]` on the fix. The previous extra wrapper forwarded the alias
before staging and passed on both. The mismatched-sibling IR test now
shares the input with its second output so the variant has to skip the
incompatible first one, and the unrelated-array test asserts the public
specialization is called rather than a retired selector name. A stale
comment describing a run-time pointer select is corrected.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The CFG scope existed only to answer whether a name is defined, yet it
stored a VarEvent{Kind: Write} for every declared parameter, output, and
target, which read as a write that never happened. The scope now holds
name membership, and publishTarget becomes declareName; the events that
reach the dataflow passes are the only VarEvents left. Outputs are still
declared before the body so a formatting marker naming one is rejected
as a read rather than passing as literal text.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The helper that widens an input read to every output it could share is
renamed possibleInputOutputAliases, and its comment and the IR plan state
the policy it implements: one CFG result serves every alias pattern of a
type specialization, so a body's unused-write diagnostics do not depend on
a particular call's sharing, at the cost of leaving a write observable
only under sharing undiagnosed in calls that do not share. Per-pattern
warnings remain a possible refinement. No behavior change.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The CFG treated every compatible input as a possible reader of every
output, so a dead store observable only under sharing went unreported in
calls that did not share. The dataflow now runs per alias context.
Settlement analyzes each type specialization's unshared context as
before; the script walk derives every call's context from its names,
forwards it through nested calls by the rule lowering uses to select a
variant, analyzes a shared context on first reach, and caches it on the
specialization for later scripts. That rule, including output-storage
widening, moves into one function, aliasPattern, that lowering and the
CFG both call, so a body is analyzed exactly as it is lowered.

Diagnostics become exact per calling context: `out = current + item`
written twice is accepted for `value = BumpTwice(value, 5)` and reported
as an unused overwrite for `other = BumpTwice(value, 5)`, directly or
through a wrapper. The exact analysis immediately found one such dead
store in the alias fixture itself, where two nested calls each assigned
`seen` and nothing read the first; the fixture discards it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The script CFG walk passed a callee's own variable types into its body,
so a nested call whose destination is an output lost the storage the
enclosing call had widened it to. Lowering keeps that storage through
outputSlotTypes, so a wrapper receiving a heap string (or a typed array)
was lowered with the nested call shared while the CFG analyzed it
unshared and reported a live intermediate write as unused.

The walk now computes each site's output storage the way lowering does,
binds the body's outputs to it before visiting nested calls, and visits
each lowered variant once, keyed by its variant symbol. Diagnostics stay
cached per alias pattern, which is all the body analysis depends on.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Call arguments are specialized on the binding's flow type, not its merged slot type

1 participant