The //json module parses general JSON with a checked stage2 walk over simdjson 5.1.1's structural index directly
into a compact word tape, with no DOM, Serde value or Java object tree in
between. Guests read that index in place through InteropLibrary, embedders can
export it into their own objects in one pass, and snapshot() detaches it into
immutable heap storage. Known-schema scalar records decode into fixed native slots
instead. Native details and ABI notes are in json/native/README.md;
the delivery plan is JSON_PLAN.md.
The module publishes (from 0.8.0) as dev.elide:truffledata-json
(//json:json: the JSON entry points, SVM backend and typed plans, plus .a and .so/.dylib
attachments for Linux x86-64, Linux AArch64 and macOS AArch64), truffledata-json-jvm (//json:jvm: FFM backend) and
truffledata-json-bindings, on top of the shared truffledata-tape and
truffledata-tape-jvm; see publishing.
The format-neutral half of this backend lives in //tape (package
dev.elide.truffledata.tape), which the YAML backend shares. JSON produces its tapes;
the owners, views, sinks, snapshots, session and stream plumbing and typed records
that read them are the same classes for every format. //json re-exports //tape (and
//json:jvm re-exports //tape:jvm), so a JSON dependency brings them along.
Shared (//tape) |
Was (JSON) |
|---|---|
TapeOwner (owners: liveness, confinement, pins, export cursor, array index, directory lookups, alias-resolution hook) |
JsonOwner |
TapeValue, TapeSink, TapeText, TapeKey |
JsonValue, JsonSink, JsonText, JsonKey |
TapeSnapshot, TapeExportNode, JavaSink, TruffleSink |
JsonSnapshot, JsonExportNode, JavaSink, TruffleSink |
TypedOwner, TypedRecord, TypedArray, TypedAccess, TypedLayout |
JsonTypedOwner, JsonRecord, JsonArray, JsonAccess, JsonPlan.Layout |
TapeSession, TapeStream (borrow, cursor, text-arena and stream bookkeeping) |
the shared half of JsonSession and JsonStream |
TapeBorrow, TapeLexical, TapeTextArena, TapeFormat |
JsonBorrow, JsonMessage.Lexical, JsonTextArena, new |
HeapValueTape, SvmValueTape, TapeSource; SegmentValueTape in //tape:jvm |
same names |
The JSON entry points keep their names and behaviour: JsonSession is a
TapeSession<JsonMessage> (so decode returns JsonMessage and with lends a
TapeValue), JsonStream a TapeStream<JsonMessage> that adds skip,
truncatedBytes and options, and JsonMessage, JsonPlan, JsonLimits,
JsonSvm, JsonJvm and JsonDecoder are unchanged apart from the renamed types.
Messages still name JSON ("JSON message closed"), because every owner reports its
TapeFormat; only the format-independent objects (sinks, text cursors and typed
records and accessors) say no format name. TapeSink gains YAML's tag, anchor
and alias events with no-op defaults; JSON never emits them, and TapeOwner.resolve
is the identity for JSON owners, which carry no alias tables.
try (JsonDecoder decoder = JsonJvm.open(libraryPath);
JsonMessage message = decoder.decode(utf8)) {
TapeValue root = message.root();
long id = root.member("id").longValue();
JavaSink sink = new JavaSink();
root.member("items").export(sink); // one walk, no per-node native calls
TapeSnapshot kept = message.snapshot(); // outlives the message and decoder
}Parsing is strict: no comments, trailing commas, NaN or Infinity. Every value is visited and validated before a message is published, including fields nobody reads. Duplicate member names are rejected after escape decoding, trailing input is rejected, and every root kind is accepted. Member order is preserved; missing and null are distinct. Malformed input, limit failures and ABI mismatches throw; there is no fallback to another parser.
| Failure | Exception |
|---|---|
| Empty input, syntax, duplicate key, invalid argument | IllegalArgumentException |
JsonLimits exceeded |
IllegalArgumentException("JSON resource limit") |
| Native allocation failure | OutOfMemoryError |
| Binding fingerprint or compact tape ABI mismatch | IllegalStateException |
Choose the backend explicitly; nothing searches for a library or falls back to Java.
| Runtime | Entry point | Notes |
|---|---|---|
| Native Image (SVM) | JsonSvm.open(), JsonSvm.decode(...) |
Links //json/native:svm statically (the C ABI and shared simdutf transcoder). Throws UnsupportedOperationException outside image runtime code. |
| JVM | JsonJvm.open(Path library) |
Loads json_shared.so through FFM. Keep //json:jvm out of static images. |
Both check the generated binding fingerprint before the first decode. Both
implement JsonDecoder:
decode(byte[])/decode(byte[], JsonLimits)copy the input once into padded storage the message retains. On SVM the native side makes that copy; on the JVM Java copies into a padded confined arena and the native side adopts it.adopt(MemorySegment data, long length, Runnable release)(JVM) treatsdata.byteSize()as capacity. With at least 64 bytes of padding beyondlengththe input is parsed in place, otherwise it is copied once. SVM uses the explicitly unsafeJsonSvm.adopt(address, length, capacity, release); itsJsonDecoder.adoptthrowsUnsupportedOperationException.
release runs exactly once, on the owning thread, when the message closes, even if
the input was copied. If adoption fails, it does not run and the caller keeps the
region. The caller must not mutate or free the region before release. Adoption
currently always uses JsonLimits.DEFAULT.
The SVM decoder handle holds no state; closing it is a no-op. The JVM decoder is confined to the thread that opened it. Closing it never invalidates returned messages or typed records: the library stays loaded until the last message, typed plan and typed record close.
A JsonSession is an explicit, thread-confined decode context: one native
simdjson parser, a padded input buffer and a block arena reused across documents,
so a small document makes no native allocation in steady state. Nothing opens a
session implicitly: JsonSvm.decode keeps decoding each call independently (no
hidden per-thread state); embedders that decode repeatedly open a session per
thread and close it when done. In a static image open one with JsonSession.open();
on the JVM with JsonJvm.openSession(libraryPath, limits) (JsonSession.open() throws
UnsupportedOperationException outside an image). Both run the same native session; see
"JVM sessions" below for the differences.
try (JsonSession session = JsonSession.open()) { // or open(JsonLimits)
session.parse(utf8, sink); // borrowed: decode, export, done
long id = session.with(utf8, root -> root.member("id").longValue());
JsonMessage kept = session.decode(utf8); // retained: outlives the session
session.validate(utf8); // strict check, nothing retained
}parse(byte[], TapeSink)andparse(byte[], offset, length, TapeSink)decode into session storage and export through the sink. Nothing is retained.parse(TruffleString, TapeSink)decodes a guest string without a Java copy: the string's own storage goes to the native session as UTF-8 (ASCII code range), Latin-1 (Latin-1 range) or UTF-16LE (anything else), and simdutf validates and transcodes it into the padded parser buffer. An unpaired surrogate anywhere is invalid JSON (IllegalArgumentException); limits apply to the UTF-8 form, which lexical spans andnumberText()describe. A string in another encoding is first transcoded to UTF-8 byTruffleString, and invalid UTF-8 is rejected.- Text arenas: when the sink's
sharedText()returns true (TruffleSinkdoes), the same native call also writes every key and string once into a per-documentbyte[]presized to2 × length + 16: Latin-1 when all its code points are at most U+00FF, UTF-16LE otherwise.TapeText.toTruffleString(encoding)then builds each string over that array without a copy (a Latin-1 string becomes UTF-16 without one too), so a parse costs one array plus oneTruffleStringper string. Strings of 64 KiB or more are copied, so one small retained string never keeps a large document's text alive. Once any string shares the array, the session gives the next parse a fresh one; an array nobody took is reused. with(byte[], Function<TapeValue, R>)lends the root to a reader. The root and every view derived from it belong to a borrowed owner that is closed (and its assumption invalidated, so cached compiled reads fail too) when the call returns; a view that escapes throwsIllegalStateExceptionafterwards.decode(...)moves the decoded storage (tape, side tables and input) into aJsonMessagewithout copying the tape. The message has the ownership rules below and survives the session's later decodes and itsclose().- While a borrow is in progress (inside a sink event or a reader), every other call
on the session, including
close(), throwsIllegalStateException. - Limits apply per document. Parsing semantics are identical to
JsonSvm.decode.
JsonSession.stream(JsonStreamOptions) reads whitespace-separated documents
(NDJSON, JSON Lines, concatenated JSON) one at a time through the session:
try (JsonSession session = JsonSession.open();
JsonStream stream = session.stream(JsonStreamOptions.DEFAULT)) { // 1 MiB documents
int at = 0;
while (at < input.length) {
at += stream.feed(input, at, input.length - at); // accepted bytes: fewer means drain first
TapeValue root;
while ((root = stream.next()) != null) consume(root); // lent until the next call
}
stream.finish(); // a final number or literal completes
while (stream.next(sink)) {} // or export each document
}- The carry-over buffer holds at most
2 × maxDocumentBytes;batchBytes(the stage1 streaming batch limit) must be at leastmaxDocumentBytes.feedaccepts nothing while buffered documents are still being read: drain withnextuntil it returnsfalseornull, then feed the rest. next(TapeSink)exports a document;next()lends its root until the next call on the stream, another stream of the session or the session (views throw afterwards);retain()detaches the lent document into aJsonMessagewithout copying the tape. Streams never share a text array: asharedText()sink gets each string copied.- Only whitespace separates documents:
[1],[2]is a syntax error (simdjson's comma format accepts doubled and missing commas). A BOM is accepted only at the start. - An invalid document throws
IllegalArgumentExceptionfromnext, and the stream stays failed untilskip()resumes after that document. Invalid UTF-8 and raw control bytes are reported by simdjson for a whole batch and cannot be skipped. Afterfinish(), a final incomplete document throws ("truncated") andtruncatedBytes()is its length. - Each document is decoded with the session's limits and duplicate-key and number rules; closing the session closes its streams.
JsonJvm.openSession(Path library, JsonLimits limits) opens the same native session
through Myna's generated FFM LibraryBinding; the library stays loaded until the
session and every message it retained are closed. Differences from static images:
- Normal FFM downcalls take native memory only, so array input (and a guest string's storage) is first copied into a reusable session-owned native staging buffer.
parse(MemorySegment data, long length, TapeSink)borrows a live native segment the calling thread can access for the call (copied natively; nothing retained). Heap, closed and other threads' segments are rejected before entry.- No text arenas: a Java array cannot be the target of a normal downcall, so a
sharedText()sink receives strings copied once each. JsonStream.feed(byte[], …)stages at most2 × maxDocumentBytesper call natively;feed(MemorySegment)passes a live native segment directly (heap segments rejected).- Inputs of at most
JsonSession.CRITICAL_LIMIT(4,096) bytes decode through Myna'sCriticalLibraryBinding(session_decode_small), andresetalways does: both are declared native leaves. The safety argument: the input is bounded (4 KiB at the slowest measured decode rate, ≈ 150 MB/s, is ≈ 30 µs of delayed safepoint), the native side takes no lock, makes no callback and lets no exception escape, and the staged input is native memory (Myna's critical adapters forbid heap segments). Pass a smallercriticalLimittoopenSession(Path, JsonLimits, int), or 0 to disable;criticalDecodes()counts the decodes that took the path. A diagnostic timing found no measurable difference against normal transitions for the 210-byte fixture document on HotSpot.
JsonLimits(maxInputBytes, maxIndexBytes, maxNodes, maxTextBytes, maxDepth) bounds
one decode. All values must be positive and maxDepth must be within 1..128.
| Limit | DEFAULT |
Bounds |
|---|---|---|
maxInputBytes |
64 MiB | Logical input, checked before copying |
maxIndexBytes |
64 MiB | Compact words, directories, traversal and duplicate-key storage, including reallocation overlap |
maxNodes |
1,000,000 | Index nodes (keys count) |
maxTextBytes |
64 MiB | Decoded text bytes for strings and keys |
maxDepth |
128 | Container nesting |
The index budget is charged per arena block and large allocation (see the native
README), so it also counts unused block tails. Parser scratch is owned by a bounded
process-wide pool (at most eight idle parsers), or by the session, and is not
counted. These limits are not a total native-memory budget. The typed
scalar path does not take JsonLimits; it uses fixed 64 MiB input and index caps.
Ownership follows the Buffa model: thread-confined scoped owners, explicit detachment, and Truffle assumptions invalidated on close.
- A
JsonMessageretains its input, index, decoded strings and lexical spans. Parsers return to the pool before decode completes, so no message depends on parser scratch and parser reuse cannot overwrite live data. - Closing the input source or the decoder does not invalidate a message.
- A message is confined to the thread that decoded it. Cross-thread reads and
cross-thread
close()throwIllegalStateException. Asynchronous transfer is not implemented. close()is idempotent on the owning thread. It invalidates everyTapeValuederived from the message, the message'sassumption()and therefore cached compiled reads, then releases native memory.close()throwsIllegalStateException("JSON message is being exported")while an export of that message is running, so a sink cannot free the tape under the walk. Close after the export returns.- Sibling messages have independent lifetimes.
- There is no finalizer or Cleaner; close messages explicitly.
TapeValue is a reusable view of one node. Every read checks its owner first, and
that check is inlined into compiled code. TapeOwner.CheckOpenNode caches up to
three owners behind their assumptions for nodes that hoist the liveness check.
Retain a value beyond its message only through a snapshot.
message.snapshot() and value.snapshot() return a TapeSnapshot: immutable,
unconfined and never closed. Its assumption() is Assumption.ALWAYS_VALID, and its
reads never touch native memory. A snapshot remains valid after its message and
decoder close, and can be read from any thread.
A subtree snapshot copies the subtree's nodes, re-based to node 0, but copies every
string region and the whole retained input, so its cost follows the document's text
size rather than the subtree's. Lexical spans are copied, so numberText() and
rawText() behave as on the message. snapshot() of a whole snapshot returns the
same object. Owners index arrays for random access in a concurrent cache, so a
shared snapshot is safe to read concurrently.
| Method | Behavior |
|---|---|
kind() |
NULL, BOOLEAN, NUMBER, STRING, ARRAY or OBJECT |
longValue() / doubleValue() |
I64 or F64 nodes only; other number kinds throw rather than convert |
stringValue() |
Decoded Java String |
numberText() |
Exact token of any number, with no conversion |
rawText() |
Source spelling of a number or string (keys included), with quotes and escapes |
member(String) / member(Utf8Key) |
Lookup; throws NoSuchElementException when missing. Retain a ValueTape.Utf8Key for repeated lookups. |
keyAt(i) / memberAt(i) |
Members in source order |
element(i) / elements() |
Random access, or a sequential iterator that builds no index |
size() |
Member or element count |
Arrays with more than eight elements build a child-offset index on first random
access and keep it while the owner lives; smaller arrays are walked.
member(String) throws IllegalArgumentException for a name with an unpaired
surrogate.
The decoder classifies each token once. numberText() always returns the exact
token, whatever its node kind.
| Token | Node | TapeSink event |
JavaSink |
Interop |
|---|---|---|---|---|
Integer in int64 (-42) |
I64 | longValue |
Long |
Integer types by range; BigInteger; double when |v| ≤ 2^53; float when also exact |
-0 |
F64 -0.0 |
doubleValue(-0.0) |
Double -0.0 |
asDouble is -0.0; no integer type fits (the sign would be lost) |
Integer above int64 within uint64 |
U64 | unsignedValue(bits) |
BigInteger |
BigInteger only |
Other token simdjson converts to a finite binary64 (1.5, 1e2) |
F64 | doubleValue |
Double |
double; float when exact; integer types when integral and in range |
Anything else (1E400, integers beyond uint64) |
DECIMAL, referencing the input | decimalValue(token) |
BigDecimal |
BigInteger when integral with at most 4096 integer digits (huge exponents such as 1E999999999 fit nothing and are never expanded); double only when exactly representable (1180591620717411303424, 2^70); never long |
F64 holds simdjson's nearest binary64, so a token such as
1.0000000000000000001 reads as 1.0. Interop fits* contracts describe the
node value, not the token; use numberText() when the spelling matters.
JavaSink throws NumberFormatException for DECIMAL exponents outside
BigDecimal's scale range.
TapeValue exports InteropLibrary. It is read-only: no member or element writes,
removals or invocations. Every message checks liveness and thread first.
| Kind | Messages |
|---|---|
| OBJECT | hasMembers, getMembers (source-ordered key strings), isMemberReadable, readMember |
| ARRAY | hasArrayElements, getArraySize, isArrayElementReadable, readArrayElement |
| STRING | isString, asString, asTruffleString (UTF-8, copied once without an intermediate String) |
| NUMBER | isNumber, fitsIn* / as* per the number policy above |
| BOOLEAN | isBoolean, asBoolean |
| NULL | isNull |
readMember and readArrayElement return TapeValue views, including for scalars.
Member reads cache the encoded name for up to three name identities per site, so
compiled reads by constant name neither re-encode nor allocate. A member name with
an unpaired surrogate has no UTF-8 encoding and matches nothing: isMemberReadable
is false and readMember throws UnknownIdentifierException. It is never encoded
lossily as ?. Reads after close fail with IllegalStateException, which polyglot
callers see as a PolyglotException.
Member lookup uses compact object directories. Objects with at least two members carry
one (hash, key word offset) entry per member, sorted by unsigned FNV-1a hash and then
offset. Objects without directories are scanned. A cached member read remembers a rank
in that directory, verifies the current key's kind, length and bytes before using it, and
falls back to a binary search on a stale hint. Second words are skipped when moving from
an escaped key to its value. TapeKey.of(name) prepares a key for repeated lookups.
JsonMessage.toMyna() and TapeSnapshot.toMyna() create detached Myna ABI v4 tapes on
demand. This copy is for interchange; ordinary reads, exports and snapshots read compact
words directly. Snapshots copy the subtree's referenced input and pool ranges and rebase
its spans and directories. rawText() therefore keeps the original spelling after the
source owner closes.
TapeValue.export(TapeSink) and TapeOwner.export(TapeSink) (the whole document)
walk the tape once, in source order. A TapeExportNode does the same inside guest
code; adopt it as a child node and call execute(value, sink). The walk is a loop
with an explicit container stack: no recursion, no per-node native calls, one
liveness check per export, and no view per key or string.
The sink receives:
beginObject(members)orbeginArray(elements)with the child count, so it can presize. Each is matched by oneend().- Inside an object, a
key(TapeText)before each member value. - One event per scalar:
nullValue,booleanValue,longValue(I64),doubleValue(F64, including-0.0),unsignedValue(U64 as unsigned two's-complement bits),decimalValue(TapeText)orstringValue(TapeText).
Keys, strings and decimals arrive through one reusable TapeText cursor per export,
rebound for each event, so an export creates no object per string. The sink chooses
how to materialize: toJavaString() (a decimal gives its exact token),
toTruffleString(encoding), copyUtf8(byte[], offset), utf8Equals(byte[]),
utf8Length(), isAscii() (from the word's ASCII flag, no scan) and hash()
(the low 15 bits of the key's FNV-1a hash; 0 for strings and decimals). The cursor is valid
only during the event that passed it: afterwards every method throws
IllegalStateException. view() returns a TapeValue that lives as long as the
owner. The owner cannot close until the export returns. An exception from the sink
ends the export and releases that hold.
toJavaString() decodes ASCII text into a Latin-1 String from a reusable scratch
buffer (one copy) and other short text with the JDK's UTF-8 decoder. Non-ASCII
strings of 256 bytes or more use the shared native simdutf transcoder
(td_tape_utf8_to_utf16) on SVM, writing UTF-16 into a pinned character array.
The outermost export on a confined message reuses one cursor (and its 129-slot
container stack) held by the message; a session's borrowed parses share one cursor.
Nested exports and snapshot exports allocate their own. JavaSink builds
LinkedHashMap (source order), ArrayList, String, Long, Double,
BigInteger, BigDecimal, Boolean and null; one sink builds one tree.
new JavaSink(keyCache) takes member names from a JavaSink.KeyCache shared by the
sinks of one thread: a bounded (1,024 names) table keyed by the key hash and the
decoded bytes, so documents of one schema stop allocating key Strings.
TruffleSink is the reference sink for Truffle guests: the same tree shape with
UTF-16 TruffleString keys (from a TruffleSink.KeyCache, copied once out of the
document) and string values. It asks for shared text, so through a JsonSession
parse its strings share the document's text arena; exporting a retained message or
snapshot (no arena) copies each string once.
This shows the intended shape for an embedder such as Elide. GraalJS internal factory names are illustrative; check the pinned GraalJS revision for exact signatures.
final class JsSink implements TapeSink {
private final JSContext context;
private final JSRealm realm;
private final ArrayDeque<Object> open = new ArrayDeque<>(); // JSObject or ArrayFrame
private TruffleString key;
Object result;
private record ArrayFrame(Object[] items, int[] size) {}
public void beginObject(long members) { push(JSOrdinary.create(context, realm)); }
public void beginArray(long elements) { push(new ArrayFrame(new Object[(int) elements], new int[1])); }
public void end() {
Object done = open.pop();
if (done instanceof ArrayFrame f) add(JSArray.createConstantObjectArray(context, realm, f.items()));
}
public void key(TapeText name) { key = js(name); }
public void stringValue(TapeText text) { add(js(text)); }
public void nullValue() { add(Null.instance); }
public void booleanValue(boolean v) { add(v); }
public void longValue(long v) { add(v == (int) v ? (Object) (int) v : (double) v); }
public void doubleValue(double v) { add(v); } // keeps -0
public void unsignedValue(long bits) { add(Double.parseDouble(Long.toUnsignedString(bits))); }
public void decimalValue(TapeText t) { add(Double.parseDouble(t.toJavaString())); } // 1E400 -> Infinity
public boolean sharedText() { return true; } // session parses fill a text arena
// In a session parse: a UTF-16 view of the document's text arena, no copy. Otherwise the
// UTF-8 bytes are copied once. (A real sink interns keys, as TruffleSink.KeyCache does.)
private static TruffleString js(TapeText v) {
return v.toTruffleString(TruffleString.Encoding.UTF_16);
}
private void push(Object container) {
if (!(container instanceof ArrayFrame)) add(container); // arrays join their parent at end()
open.push(container);
}
private void add(Object value) {
Object parent = open.peek();
if (parent == null) result = value;
else if (parent instanceof ArrayFrame f) f.items()[f.size()[0]++] = value;
else JSObject.set((JSObject) parent, key, value); // key -> property write
}
}Presized arrays, one property write per member, and no intermediate Java tree are
the point. The sink controls number coercion: this one follows JSON.parse, which
rounds integers above 2^53 and DECIMAL tokens to double.
Known-schema JSON compiles through decoder.compile(schema) or
decoder.compile(root, recordTypes) on either backend's JsonDecoder into a
reusable, thread-confined native plan (JsonPlan.compile(...) is the static SVM
shortcut, and JsonSvm.open().compile(...) returns the same plan).
plan.decode(utf8) returns the scoped root TypedRecord: validated slots with
separate presence and null masks, read with primitive getters or through
DataLibrary. Strings and unsigned BigIntegers materialize on request.
Records hold up to 64 fields of INT64, FLOAT64, BOOLEAN, STRING, unsigned 64-bit
EXACT, RECORD and ARRAY, with required/nullable, integer range, UTF-8/UTF-16/code
point length and binary32 representability contracts. A RECORD field names its
schema with Constraints.recordType(), a key of recordTypes whose schema has that
same typeName. An ARRAY field takes its element contract from
Constraints.element(): a scalar or RECORD that is neither nullable nor optional
(JSON array elements are never null; nullability applies to the array). Element
counts (Length on ARRAY) and arrays of arrays are not supported. Recursive record
types and nesting deeper than 32 records (the root counts as 1) are rejected when
the plan compiles. Unknown fields, escaped duplicates, missing required fields, type
mismatches and trailing input are rejected at every level.
Every record and array of one decode shares a TypedOwner (thread guard,
assumption, close). record(i) returns a nested TypedRecord, array(i) a
TypedArray (BorrowedArray with interop array exports), and DataLibrary.read
returns those for RECORD and ARRAY fields. Closing any record closes the owner and
invalidates all of them. TypedRecord.snapshot() and TypedArray.snapshot() copy into
owned immutable Batch rows and DataArrays that survive close.
TypedAccess binds a schema and an ordinal path (new TypedAccess(root, recordTypes, 1, 0, 2): RECORD hops, then an INT64/FLOAT64/BOOLEAN leaf) for allocation-free
compiled primitive reads that walk the record table without intermediate
TypedRecords; keep it in a final node field. Records outlive their plans.
On the JVM the plan table is written through the generated FFM layouts from the
same backend-neutral layout the SVM plan uses, and slot and record-entry reads go
through bounded MemorySegments with an exact-class cast. A JVM plan rejects
decoding once its decoder closes; records it returned stay valid, the plan must
still be closed, and the library stays loaded until the last plan and record close.
-Dtruffledata.json.cells=true routes the document decode's native calls through Truffle cells
that Myna generates from this module's seam, instead of the generated @CFunction/FFM calls: on
Native Image td_json_decode, td_json_decode_adopted and td_json_release (JsonSvm), on
HotSpot td_json_decode_adopted and td_json_release (JsonJvm, whose decode(byte[]) copies
into an adopted arena). Every check, limit, status mapping, error and owner lifetime stays the
backend's own. The route is off by default; it is support for Myna cells and makes no
performance claim.
Measured: see docs/RELEASE_0_11_0.md
-
Artifact.
dev.elide:truffledata-json-cells(Bazel//json/cells) carries the route (CellsSvm,CellsFfm,JsonCellsFeature), both generated cell sets and Myna's Native Image metadata for the Native Image cells; it depends ontruffledata-cells(Myna's cells runtime),truffledata-jsonandtruffledata-json-bindings. Nothing in the default JSON artifacts depends on it; with the flag off it need not be on the class path. It uses the same natives astruffledata-json. -
Native Image. Pass
-Dtruffledata.json.cells=trueand--features=dev.elide.truffledata.json.JsonCellsFeaturetonative-image, with the cells artifact on the image class path. The Feature installs the cells at image build time andJsonCellsRoute.ENABLEDfolds, so an image built without the flag contains no cells code; setting the property when running an image has no effect. The flag and the route must agree or the image build fails, with no silent fallback: the build-time holderJsonCellsRouteoftruffledata-jsonrejects the flag without the Feature, and the Feature cannot load without the artifact. The Feature without the flag installs nothing. -
JVM. Pass the property to the JVM with
truffledata-json-jvmand the cells artifact on the class path. It is read once, whenJsonCellsRouteinitializes (astatic final), andJsonJvm.openbinds the cells; the flag without the artifact fails there. Myna's FFM cells bind through the class loader's lookup, so the cells alsoSystem.loadthe fingerprint-checked library: one library path per class loader. -
Call shape. Both routes call the generated cells' static
invokeentry (Myna 0.16.0) for decode and adopted decode, and release through the same cells class's raw binding (below): native carriers and rawlongaddresses, with no cell instance,Object[]argument array, boxed carrier,NativePointerorBorrow. The routes are host code (compiled ahead of time on Native Image), where a cell'sexecute(Object[])path is not scalar-replaced;invokecoerces nothing and allocates nothing, so the caller keeps every address valid for the call (JsonSvmkeeps the input pinned or adopted, and the result allocated). The cells'executemethods remain the entry for guest code. On HotSpot,invokeis a@TruffleBoundarystatic over the FFM downcall and does not track the caller's segment scope, soCellsFfmfirst touches the input segment, which raises the sameWrongThreadExceptionorIllegalStateExceptionthe baseline downcall raises for a foreign confined or closed segment.//json/cells/tests:jvmscans both routes for this shape,//json/cells/tests:aot_shapechecks the compiled Native Image route calls noexecutemethod, and//json/cells/tests:native_testchecks the route allocates exactly what the production static bindings allocate per call. -
Seam and fingerprint. The cells come from seams derived in the build from
native/json.seam(bazel/cells_seam.py): the module options change (truffle_cells=true,truffle_backendnative/ffm,truffle_transitions=direct, package and class) andjson/cells/ownership.seamadds Myna ownership declarations, which only the cells carry. The production seam, bindings, ABI fingerprint and natives are unchanged; the backends check the library against the production fingerprint, and//bazel:myna_contractchecks that each derived descriptor equals the production one apart from those options and the ownership delta (the declarations, explicitnounwind/nocapture/readonlycontracts,ptr<void>narrowed to the named resource or struct, and ownership transferred on each destroy). -
Ownership (Myna 0.18.0).
json/cells/ownership.seamdeclares, each with its contract checked againstnative/api.cc:- resources
JsonSession(session_create/session_release) andJsonStream(stream_create/stream_release; a guest's stream holds its session open), and in-place resultsJsonResult(decode_adopted/release),JsonPlan(plan_create/plan_release) andJsonTypedResult(typed_decode/typed_release), each created on status 0 only; - call borrows on
decode,typed_decode,session_decode,session_decode_small(data, and the nullabletext_out) andstream_feed, so a guest passes each buffer and the cell derives its length; - a borrow of
decode_adopted's input held until itsJsonResultis released. Its extent iscapacity, notlength: with enough padding the native side adopts and reads the whole region.
Not declared:
decode,session_retainandresult_mynaas creates. They initializeJsonResulttoo, and Myna maps a struct to one resource with one create, so the guest-facingreleasecell takes onlydecode_adopted's owners;decode_adoptedcoversdecodefor guests (with too little padding it copies its input).nounwindis declared only where every throwing path is caught (stream_finish,stream_nextand the other imports need none). Every resource isconfined. Guest code that asks for UTF-16 input or a text arena lends 2-byte-aligned buffers: misalignment isTD_JSON_ARGUMENT. The routes are host code and keepinvoke;release, a resource's destroy, has noinvoke, so they callJsonCells.release(Native Image) andJsonCellsJvmFFM.release(HotSpot, with the result segment itself), the bindings the destroy cell and the formerinvokereach. - resources
These interfaces exist for follow-on adapters: ordered members (keyAt,
memberAt), random lookup, exact and raw token text (numberText, rawText),
immutable snapshots, single-pass export and typed plans. Still to do:
- JEP 540: durable, thread-safe JDK
TapeValueimplementations on a pinned JDK 28 early-access lane, backed initially by snapshots; JSON generation with exact number emission; Java UTF-16 factory semantics for unpaired surrogates; and differential conformance against the pinned JDK, including its surrogate policy. No JEP compliance is claimed. - Jackson 3, then JSON-P, Gson on demand: parser and generator adapters over the completed index. These are token iteration, not bounded-memory incremental parsing; stream or batch parsing needs its own buffering and lifetime design.
- kotlinx: a
Decoderadapter in Komodo, consuming ordered members, raw text, typed plans and snapshots. - Cross-thread ownership transfer, caller-chosen limits for adoption, and published Maven artifacts.
bazel test //json/tests:jvm # JVM FFM, interop, export, snapshots
bazel test //json/tests:native_test # SVM, compiled read/export gates (Linux x86-64, RBE)
bazel test //json/native:test //json/native:api_test //json/native:typed_testThe SVM compiled gates cover typed and generic member reads, interop reads and the
export loop, bound Java allocation per iteration, and require compiled reads to
fail after close. They measure
retained reads, not parse cost or native memory. Comparison results against Gson
are in benchmarks/comparison/JSON.md. Its
generic cells (Gson trees, JavaSink export, interop reads, snapshots) gate the
compiled interop and export loops on graph shape and a fixed per-document allocation
budget. The first measurement found generic reads slower than reading Gson trees
and snapshot reads allocating heavily; see that page before relying on either path
for speed.
The generic native producer, borrowed sessions and streams share the structural stage2 walk. Typed plans use the same walk with checked schema-key lookup and typed slot contracts; nested/repeated plans add bounded record/array frames. On-Demand remains the test oracle and runs after a failed nested decode to preserve error diagnostics; a structural failure cannot publish a result. The previous generic producer is a test-only differential oracle. Production string kernels remain portable: measured AVX2 and AVX-512 variants did not clear the adoption gate on unclemax. The nested/repeated admission report records paired SVM ingest gains and control regressions. Earlier qualification and explicitly unmet criteria remain in the producer ledger.