Give the .acx format a cookie and a version number - #53
Merged
Conversation
Nothing at the head of an .acx said what it was, so the only thing separating a game binary from any other file was whether deserialization happened to trip over something. Often it didn't: feeding the interpreter a text file got as far as "No 'main' object", meaning a whole universe had been read out of Archetype source and believed. The web driver hands the deserializer a file the player picked off their own disk, so that is not a hypothetical. Files now begin with 7F 41 43 58 and a format version. The version is the layout's, not the interpreter's: a release that moves no bytes must leave every .acx as it was, or the goldens churn and comparing bytes stops being an oracle. Reading an older headerless file still works. That layout opens with ended_, a bool written as a one-byte varint, so its first byte can only be 0 or 2 -- enough of a gate to turn away an image or an archive while letting every real pre-header file through. When there are none of those left to care about, the branch can go. Deciding between the two needs a look at bytes that may belong to either, so Storage grows a peek: a read that can be taken back. The goldens are five bytes longer apiece and their Turtle is untouched, which is the whole claim -- the container changed and the contents did not. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Ui8UMgev1U8LW5iyQdSsJ
A count in an .acx is a promise about bytes that follow, and the reader believed all of them on sight. Four places: a string's length, a string value's length, a registry's slot count, and an identifier map's size. The registry was the one that mattered. Its records each name the slot they belong in, and that number went into a deque subscript unchecked, so thirteen crafted bytes -- a header, one slot, one record naming slot fifty million -- wrote out of bounds and took the process down with SIGSEGV. The slot count itself was believed too: twelve bytes could ask for a deque of four hundred million strings, and the machine would go and try. No stream can back a count larger than the bytes left in it, since every element costs at least one byte to encode, so readCount refuses one that cannot be honoured. That is exact for strings and records. For a registry's slot count it is slightly stricter than the format demands -- a free slot costs nothing to write -- but free slots are reused rather than accumulated, and if a game ever does hold more objects than its save has bytes, the answer is to write the holes down rather than to stop checking. The string value in Value.cc had no check at all on what it managed to read, so a truncated literal came back as a run of NUL bytes instead of as an error. It has one now. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Ui8UMgev1U8LW5iyQdSsJ
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two commits: a format header, and the bounds checks the header led me to.
1. A cookie and a format version
Nothing at the head of an
.acxsaid what it was. The only thing separating a game binary from any other file was whether deserialization happened to trip over something — and often it didn't:That error comes from game logic, not from the format. A whole universe had been read out of a text file and believed.
drivers/web/play.js:601is a file picker that hands the wasm deserializer whatever the player chose off their own disk, so an arbitrary file reachingoperator>>is not hypothetical.Files now begin with
7F 41 43 58followed by a format version varint.0x7Fis neither printable nor a legal UTF-8 lead byte, so a text file is turned away on byte one.The version is the layout's, not the interpreter's. A release that moves no bytes has to leave every
.acxbyte-for-byte as it was, or the goldens churn and byte comparison stops being the oracle it became in August.CurrentFormatVersionmoves only whenoperator<<changes shape.Older files still load. The pre-header layout opens with
ended_, a bool written as a one-byte varint, so its first byte can only be0x00or0x02. Enough of a gate to turn away an image or an archive while letting every real headerless.acxthrough —drivers/gorreven.acxanddrivers/spacebits.acxare both pre-header and both still read. When there are none left to care about, that branch can be deleted.Storagegrows apeekto make the branch possible: a read that can be taken back.2. Check every count before believing it
The first version of this PR noted the unchecked
value.resize(size)as a follow-up. Looking properly turned up four sites, and one of them was considerably worse than a bad allocation.IdIndex::readhas each record name the slot it belongs in, and that number went into a deque subscript, unchecked. Thirteen crafted bytes — a header, one slot, one record naming slot fifty million — wrote out of bounds:The slot count itself was believed too: twelve bytes could ask for a deque of four hundred million strings, and the machine would go and try.
No stream can back a count larger than the bytes left in it, since every element costs at least one byte to encode, so
readCountrefuses one that cannot be honoured. Exact for strings and records; slightly stricter than the format demands for a registry's slot count, since a free slot costs nothing to write — but free slots are reused rather than accumulated, and if a game ever does hold more objects than its save has bytes, the answer is to write the holes down rather than to stop checking.Also: the string case in
Value.cchad no check at all on what it managed to read, so a truncated literal came back as a run of NUL bytes rather than as an error.No 'main' objectNot an Archetype binary: …Could not fully read string declared as 7841 bytesNot an Archetype binary: …Not an Archetype binary: …Record names slot 50000000 in a registry of 1A registry size of 400000000 is more than the 1 bytes remaining can supply.acxfrom the futureformat version N is not one this interpreter understandsTesting
--test: 18 suites, 0 failures, with new coverage for peek-does-not-consume, header round-trip, headerless-is-untouched, truncated cookie, version-from-the-future, and each bounds case including the exact shape that segfaulted.Goldens regenerated:
bare.acxandcherry.acxare each exactly five bytes longer and their.ttlis unchanged — the container changed, the contents did not.check.shpasses.drivers/gorreven.acxstill inspects to the same 980 lines of Turtle.What this does not fix
Fuzzing 400 mutants of a valid
bare.acx(1–6 random byte flips, a quarter of them also truncated):So the bounds checks convert 62 crashes into clean rejections, and 171 remain. Those are a different class:
assertfailures and segfaults inside expression and statement reconstruction, where opcodes, value tags, and object ids are trusted exactly the way counts used to be. Worth its own issue rather than more scope here — and worth noting that theassertones are only diagnosable in a debug build.🤖 Generated with Claude Code
https://claude.ai/code/session_013Ui8UMgev1U8LW5iyQdSsJ