Skip to content

Give the .acx format a cookie and a version number - #53

Merged
gitosaurus merged 2 commits into
mainfrom
feat/acx-format-header
Aug 12, 2026
Merged

Give the .acx format a cookie and a version number#53
gitosaurus merged 2 commits into
mainfrom
feat/acx-format-header

Conversation

@gitosaurus

@gitosaurus gitosaurus commented Aug 12, 2026

Copy link
Copy Markdown
Owner

Two commits: a format header, and the bounds checks the header led me to.

1. A cookie and a format version

Nothing at the head of an .acx said what it was. The only thing separating a game binary from any other file was whether deserialization happened to trip over something — and often it didn't:

$ ./build/archetype --perform=games/starship_lex.arch     # an Archetype *source* file
ERROR: No 'main' object

That error comes from game logic, not from the format. A whole universe had been read out of a text file and believed. drivers/web/play.js:601 is a file picker that hands the wasm deserializer whatever the player chose off their own disk, so an arbitrary file reaching operator>> is not hypothetical.

Files now begin with 7F 41 43 58 followed by a format version varint. 0x7F is neither printable nor a legal UTF-8 lead byte, so a text file is turned away on byte one.

The version is the layout's, not the interpreter's. A release that moves no bytes has to leave every .acx byte-for-byte as it was, or the goldens churn and byte comparison stops being the oracle it became in August. CurrentFormatVersion moves only when operator<< changes shape.

Older files still load. The pre-header layout opens with ended_, a bool written as a one-byte varint, so its first byte can only be 0x00 or 0x02. Enough of a gate to turn away an image or an archive while letting every real headerless .acx through — drivers/gorreven.acx and drivers/spacebits.acx are both pre-header and both still read. When there are none left to care about, that branch can be deleted.

Storage grows a peek to make the branch possible: a read that can be taken back.

2. Check every count before believing it

The first version of this PR noted the unchecked value.resize(size) as a follow-up. Looking properly turned up four sites, and one of them was considerably worse than a bad allocation.

IdIndex::read has each record name the slot it belongs in, and that number went into a deque subscript, unchecked. Thirteen crafted bytes — a header, one slot, one record naming slot fifty million — wrote out of bounds:

$ ./build/archetype --perform=oob.acx ; echo $?
139                                          # SIGSEGV, no output at all

The slot count itself was believed too: twelve bytes could ask for a deque of four hundred million strings, and the machine would go and try.

No stream can back a count larger than the bytes left in it, since every element costs at least one byte to encode, so readCount refuses one that cannot be honoured. Exact for strings and records; slightly stricter than the format demands for a registry's slot count, since a free slot costs nothing to write — but free slots are reused rather than accumulated, and if a game ever does hold more objects than its save has bytes, the answer is to write the holes down rather than to stop checking.

Also: the string case in Value.cc had no check at all on what it managed to read, so a truncated literal came back as a run of NUL bytes rather than as an error.

input before after
Archetype source file No 'main' object Not an Archetype binary: …
random bytes Could not fully read string declared as 7841 bytes Not an Archetype binary: …
PNG (read as a universe) Not an Archetype binary: …
record naming slot 50,000,000 SIGSEGV Record names slot 50000000 in a registry of 1
12 bytes claiming 400M entries allocates until it thrashes A registry size of 400000000 is more than the 1 bytes remaining can supply
.acx from the future format version N is not one this interpreter understands

Testing

--test: 18 suites, 0 failures, with new coverage for peek-does-not-consume, header round-trip, headerless-is-untouched, truncated cookie, version-from-the-future, and each bounds case including the exact shape that segfaulted.

Goldens regenerated: bare.acx and cherry.acx are each exactly five bytes longer and their .ttl is unchanged — the container changed, the contents did not. check.sh passes. drivers/gorreven.acx still inspects to the same 980 lines of Turtle.

What this does not fix

Fuzzing 400 mutants of a valid bare.acx (1–6 random byte flips, a quarter of them also truncated):

signalled rejected cleanly loaded and ran
header only 233 102 65
header + bounds 171 166 63

So the bounds checks convert 62 crashes into clean rejections, and 171 remain. Those are a different class: assert failures and segfaults inside expression and statement reconstruction, where opcodes, value tags, and object ids are trusted exactly the way counts used to be. Worth its own issue rather than more scope here — and worth noting that the assert ones are only diagnosable in a debug build.

🤖 Generated with Claude Code

https://claude.ai/code/session_013Ui8UMgev1U8LW5iyQdSsJ

gitosaurus and others added 2 commits August 11, 2026 21:14
Nothing at the head of an .acx said what it was, so the only thing
separating a game binary from any other file was whether deserialization
happened to trip over something.  Often it didn't: feeding the
interpreter a text file got as far as "No 'main' object", meaning a whole
universe had been read out of Archetype source and believed.  The web
driver hands the deserializer a file the player picked off their own
disk, so that is not a hypothetical.

Files now begin with 7F 41 43 58 and a format version.  The version is
the layout's, not the interpreter's: a release that moves no bytes must
leave every .acx as it was, or the goldens churn and comparing bytes
stops being an oracle.

Reading an older headerless file still works.  That layout opens with
ended_, a bool written as a one-byte varint, so its first byte can only
be 0 or 2 -- enough of a gate to turn away an image or an archive while
letting every real pre-header file through.  When there are none of those
left to care about, the branch can go.

Deciding between the two needs a look at bytes that may belong to either,
so Storage grows a peek: a read that can be taken back.

The goldens are five bytes longer apiece and their Turtle is untouched,
which is the whole claim -- the container changed and the contents did
not.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Ui8UMgev1U8LW5iyQdSsJ
A count in an .acx is a promise about bytes that follow, and the reader
believed all of them on sight.  Four places:  a string's length, a string
value's length, a registry's slot count, and an identifier map's size.

The registry was the one that mattered.  Its records each name the slot
they belong in, and that number went into a deque subscript unchecked, so
thirteen crafted bytes -- a header, one slot, one record naming slot fifty
million -- wrote out of bounds and took the process down with SIGSEGV.
The slot count itself was believed too:  twelve bytes could ask for a
deque of four hundred million strings, and the machine would go and try.

No stream can back a count larger than the bytes left in it, since every
element costs at least one byte to encode, so readCount refuses one that
cannot be honoured.  That is exact for strings and records.  For a
registry's slot count it is slightly stricter than the format demands --
a free slot costs nothing to write -- but free slots are reused rather
than accumulated, and if a game ever does hold more objects than its save
has bytes, the answer is to write the holes down rather than to stop
checking.

The string value in Value.cc had no check at all on what it managed to
read, so a truncated literal came back as a run of NUL bytes instead of
as an error.  It has one now.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Ui8UMgev1U8LW5iyQdSsJ
@gitosaurus
gitosaurus merged commit 9f5969c into main Aug 12, 2026
2 checks passed
@gitosaurus
gitosaurus deleted the feat/acx-format-header branch August 12, 2026 04:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant