Skip to content

Merge train 179: #10168, #10171, #10147, #10150, #10136, #10145 - #10188

Merged
proggeramlug merged 22 commits into
mainfrom
train179
Sep 13, 2026
Merged

Merge train 179: #10168, #10171, #10147, #10150, #10136, #10145#10188
proggeramlug merged 22 commits into
mainfrom
train179

Conversation

@proggeramlug

@proggeramlug proggeramlug commented Sep 13, 2026

Copy link
Copy Markdown
Contributor

Merge train 179: lands #10168, #10171, #10147, #10150, #10136 and #10145, plus changelog fragments for #10147 and #10150 (lint's changeset gate requires one; neither PR adds its own) and the workspace version bump to 0.5.1549.

All PR commits were cherry-picked onto 6874a9eb73 (main after merge train 177 / #10187) without conflicts, at heads #10168 f5f651d9b4, #10171 2b77e7d4fe, #10147 7f55a7d742, #10150 549a7c4414, #10136 718239be8d, #10145 0c2876b193. The regex stack (#10176, #10181) was taken out of this train because its author's lane is landing it; see the compile-break note on #10176.

Validation (macOS arm64)

Full validation ran on a superset tree: these six PRs plus #10174/#10176/#10181, with the #10176 None fix. Outside the regex files, that tree differs from this train only by one #[cfg(feature = "regex-engine")] line's position relative to a doc comment.

Confirmation on this exact tree: RUSTFLAGS=-Dwarnings cargo check -p perry --bins, the regex-free runtime check, and cargo test --release -p perry-runtime (3726 passed / 0 failed).

CI attribution (each PR's own run vs main's run at b5a82cfeae)

Review notes

Summary by CodeRabbit

  • New Features

    • Improved optimization of loops over JSON record arrays, including constant, counter-based, and derived indexes.
    • JSON parsing now adapts between lazy and eager array materialization based on access patterns.
  • Performance

    • Reduced memory usage and CPU overhead for large JSON documents and record arrays.
    • Improved garbage-collection efficiency when reclaiming large collections and relocating lazy JSON data.
    • Deeply nested JSON now falls back more efficiently to safe parsing.
  • Release

    • Updated version to 0.5.1549.

Ralph Küpper added 22 commits September 13, 2026 13:13
…nning

The direct JSON parser re-read every byte of its input before parsing,
to decide whether the recursive descent could exceed the 1000-level
native-stack bound. The "already validated" shortcut lived on the
string-token reuse cache, which only exists for a source under 2 MB
holding one large string value, so no record document ever hit it and
every direct parse paid a whole-document scan on top of the parse.

DirectParser now counts open containers as it descends, on every
recursive entry including the shaped-record path and the typed
top-level array, and aborts with `depth_exceeded` when the bound would
be crossed. Valid documents never scan. A failed direct parse goes to
the heap-stack parser when it hit the bound or when the cold classifier
says the document is deep, so malformed deep input keeps its error
kinds. The only remaining pre-scan is the forced tape above the lazy
size ceiling, where an over-budget document must fail before its
native tape is reserved.

(cherry picked from commit 44d28cf)
(cherry picked from commit 0ff443d)
A debug test build's parser frames are several times larger than the
release runtime's, so 1001 nested objects on the harness's default 2 MB
thread overflowed the stack in CI (SIGSEGV after the iterator_helpers
tests). The bound under test is the release runtime's; the check itself
runs on a 256 MB worker like the 300 000-level test does.

(cherry picked from commit f5f651d)
The per-array homogeneous element-shape invariant (#7480) keyed every
proof on a class id, and `element_identity_of_bits` refused `class_id ==
0` outright. Every `JSON.parse`'d record is class 0 with an ordinary
birth ShapeId (`object/json_construction.rs`), so no parsed record array
could ever carry a proof — the one array shape the invariant's consumer
most wants to reason about was structurally excluded.

Admit class 0, keyed on the EXACT ordinary ShapeId the descriptor probe
already validated. That identity is strictly narrower than the
class-level one it replaces, and it has to be: "same class" is vacuous
when the class is 0, so `element_matches_record` now declines the two
class-level fallbacks for a class-0 record rather than relying on both
of them failing closed by coincidence.

Two new generated-code-facing entry points:

* `js_array_ensure_element_shape_ordinary` — establish-or-confirm,
  returning the proven ordinary ShapeId for a class-0 proof and 0 for a
  class-keyed one. The two proofs are deliberately not interchangeable:
  a class-keyed record matches at class level, so its
  `ordinary_shape_id` is the first element's shape and not a per-element
  guarantee.
* `js_shape_ordinary_inline_slot_for_key` — the inline slot a PLAIN
  ordinary shape assigns to a key, or -1. Four conjuncts make "slot k ==
  key position k" true (ordinary kind, generation 0, no holes, every key
  inline); dropping any one would produce a wrong offset rather than a
  missed optimization, so each is asserted separately. The key arrives as
  the whole NaN-box rather than a masked pointer, because a short
  property name reaches the string pool as an SSO immediate whose masked
  low bits are packed characters and not an address.

`js_array_ensure_element_shape` still returns the class id, so every
existing consumer reads a class-0 proof exactly as "no proof".

(cherry picked from commit 84ee63f)
`for (let i = 0; i < count; i++) sum += rows[7].id` and
`for (let i = 0; i < count; i++) { const index = i % length; sum += rows[index].id; }`
over a `JSON.parse`'d record array now run in the #7480 call-free element-shape
fast clone. They previously ran the full element-read + field-read diamond
pair.

The clone keyed every proof on a compile-time class, which is exactly what a
parsed record array does not have. Four things kept it dark, each independently
sufficient:

* the matcher needed a resolvable element class, and `rows: any` has none;
* the preheader's `GC_TYPE_ARRAY` brand ran BEFORE the growth-forwarding
  repair, so the `GC_TYPE_LAZY_ARRAY` header `JSON.parse` returns for a
  top-level array in [1 KB, 16 MB] was rejected before the repair that would
  have materialized it;
* the residual per-element check required `GC_OBJ_TYPED_LAYOUT_INTACT`, which
  a parsed record never has — the clone would have been emitted, entered, and
  then side-exited on the first element of every loop, with every IR-census
  assertion still passing;
* the index had to be exactly the counter, so neither `rows[7]` nor
  `const d = i % n; rows[d]` was admitted.

The second arm proves the same thing about a different identity: the preheader
asks `js_array_ensure_element_shape_ordinary` for the exact ordinary ShapeId
every element carries, then `js_shape_ordinary_inline_slot_for_key` for each
tracked property's inline slot in that shape. Both are loop-invariant, so the
read stays one bare offset load. The repair now precedes the brand (the refresh
is safe on an unbranded value by construction: it resolves through
`clean_arr_ptr`, which returns null for every tracked non-array). The residual
mask drops the typed-layout conjunct and the loaded word is tag-tested as a
Number instead, side-exiting to the slow clone when it is not one — the same
"this slot holds a raw double" claim, established from the value rather than
from a layout declaration.

`rows[k]` and `const d = i % m; rows[d]` each carry their own preheader bounds
obligation (`length > k`, `1 <= m <= length`), so the clone still pays no
per-read bounds test. The derived binding is virtual inside the clone: its
`Let` emits one `srem i32`, because the generic `%` lowering is a runtime call
and a call inside this clone deletes it rather than slowing it (#7690). The
matcher admits exactly one index form per loop and the fact lookup re-checks
the spelling, so a fact can never serve a read whose obligation was not
discharged.

A constant-index loop deliberately does not require the counter's canonical i32
slot: `stmt/let_stmt.rs` mints one only for an index-used or i32-bounded local,
and the `repeat` shape's counter is neither. The matcher, the fact lookup and
the field lowering all ask the same `needs_counter_i32_slot()` question.

The revocation argument is unchanged — it never mentioned classes, and
call-free remains the whole admission test. `fast_clone_slice` was widened to
own the new `element_shape.number` blocks, or every negative assertion against
it would have been partly vacuous.

(cherry picked from commit a02e11f)
The shape-keyed clone (#10123) was entered by a `cond_br` the IR census could
see and never once at run time on its own benchmark. The preheader still
emitted the counter arm's `length >= bound`, so
`for (i = 0; i < 1000000; i++) sum += rows[7].id` over a 7,600-element array
asked it to prove `length >= 1000000` — false — and a million iterations ran
the slow clone. Measured: 5.73 ns/iter before, 5.43 after, with the whole
optimization inert.

The counter is not an index in the `Constant` and `DerivedMod` forms, so the
verified prefix has nothing to say about the trip count; each form already
carries its own obligation (`length > k`, `m <= length`). The `arr.length`
trip-count arm keeps its i32-fits check for every index form, because the
emitted trip test is signed whatever the index is.

Both shape-keyed index tests now assert the ABSENCE of the counter arm's
comparison, which is the assertion that would have caught this: the derived
case pins the count at exactly one `icmp uge`, and the counter case asserts the
obligation is still there, so neither can drift into the other.

`element_shape_loop_tests.rs` crosses the 2000-line cap with those cases, so
the shape-keyed half moves to `element_shape_shape_keyed_tests.rs` — a CHILD
module, because every helper it uses is private to the parent and duplicating
an IR census is how two of them drift apart. `let_stmt.rs` crossed the cap too;
its two virtual-binding arms (#7771's element binding, #10123's derived index)
are now one call into `element_shape_loop::lower_virtual_clone_binding`, which
is where their soundness arguments belong anyway.

(cherry picked from commit f603e0d)
… read

The local binding type-proof audit requires every local_type_hint read to
carry a classification. array_is_untyped reads the declared type only to
decline the shape-keyed element-shape loop clone for a receiver with any
layout claim; admission still goes through the preheader's runtime proof
and the per-element residual check.

(cherry picked from commit 2b77e7d)
A full collection unregistered each dead old object from the page index on its
own: two `Vec` allocations, a deferral-buffer flush, a promoted-run
materialization, both table borrows, and a linear `position` over the page's
object list before `swap_remove`. Freeing every object on a page that way is
quadratic in objects per page, and a full frees whole pages of small objects.
On records_array_8m:scan (~720k dead records per full)
`invalidate_dead_old_arena_header` was 15.8% of ALL samples -- the single
largest cost of a full collection.

The registration side already batches for exactly this reason
(`flush_deferred_old_page_registrations_batch`, #7624). This is its mirror:
`ArenaSweepObjectsState` still invalidates each dead header's fields
immediately, so no walker can read it as live, but queues the page-index
removal and flushes once per sweep step (and every 4096 headers, so an
unbudgeted sweep never stages an unbounded buffer). The batch does one flush,
one run materialization per touched page, one `retain` per page against that
page's sorted dead headers, and one page-meta update per page -- allocated
bytes and object counts only fall here, so applying a page's decrements
together and resetting/refreshing once is the same state the per-object path
reaches. Nothing inside a sweep step reads page-index membership, and the
queue is empty at every step boundary.

Four fulls on records_array_8m:scan: 241 ms -> 187 ms (-22%). The batched
flush is 6.3% of samples where the per-object remover was 15.8%.

`every_page_object_reader_expands_promoted_runs` covers the new remover (it
touches OLD_GEN_PAGE_OBJECTS and expands first). A new test drives both
removers over the same population in one arena -- page-spanning objects, a
fully emptied page, partial pages -- and requires identical page membership
and metadata; it was sabotage-tested (dropping the object-count decrement
fails it with "page metadata diverged").

(cherry picked from commit 7f55a7d)
…raversed

A top-level JSON array parses onto a validating tape and materializes its
elements on demand. That wins whenever the caller touches a few elements --
`parse` and `sparse` run at ~0.4x the better of Node and Bun -- but a
program that then walks every element pays for two tokenizations: the tape
build and the per-record reparse the scan flip hands to the direct parser.
Profiled on records_array_1m:scan: 26.7% tape build, 43.1% record reparse,
1.11x the better engine. The same inputs parsed eagerly run at parity.

No size threshold can choose between the two, because the direct parser is
also the wrong choice for some arrays nobody traverses: heterogeneous_1m's 32
record shapes cost it 2.89x CPU and 3.31x RSS where the tape takes 0.35x and
0.66x. Raising the lazy window's lower bound would have passed the matrix
cells and regressed small heterogeneous parses ~4x, so the decision follows
behaviour instead.

`json::traversal_feedback` keeps a per-thread score: every lazy array created
costs a point, every traversal earns two. Once traversal is the norm, eligible
parses go eagerly, and one in 16 still takes the tape so a program that stops
traversing drifts back. Parse-only and sparse programs never earn evidence,
so they never leave the tape. The gate applies only in `PERRY_JSON_TAPE`'s
auto mode.

Two traversal signals, both from `lazy_get_rooted`'s cold read and folded into
one call so json_tape.rs stays under its line cap:
  - the existing adaptive flip (cumulative walk or scan streak), and
  - an in-order read of the LAST element. The flip deliberately never fires
    for an array too small for its streak to be proportional evidence -- a
    120-row array reaches the 64-read streak with over half of it cached --
    so without this, records_array_16k:scan never produced any evidence.
Stringify, revivers, array methods and mutation also force materialization,
but none of them is evidence that a scan would have been cheaper eagerly.

Measured across all 50 matrix cells in one binary against the tape-only route:

  records_array_16k:scan   1.23x -> 0.78x   (RSS 51 -> 34 MiB)
  records_array_1m:scan    1.11x -> 0.98x
  records_array_8m:scan    CPU 0.93x -> 0.77x, RSS 190 -> 163 MiB

No other cell moved outside noise.

(cherry picked from commit 549a7c4)
The brand test is the first thing every indexed read on an unknown receiver
executes, and it computed the entire guard set before finding out the
receiver was not a typed array: the element kind load, its range test, both
index range checks and three ANDs. A JSON.parse array -- and any ordinary
Array behind an erased receiver -- is not a typed array, so it paid all of
that on every element read, forever, to reach a branch it was always going
to take.

Decide on the tag alone and leave. The kind and index guards only mean
anything once the tag says typed array, so they move behind it into
tav.get.kind_guard; the typed-array fast path reaches the same guard set by
the same AND-reduction and is unchanged.

Retired instructions per read, measured against the pre-#10114 compiler on
the JSON access fixtures: -2.4% to -3.5% on all twelve rows. That also
erases #10114's one disclosed cost -- the 20 MiB rows, which are ordinary
Arrays above the lazy admission bound, go from +0.7..+1.8% against that
reference to -0.4..-2.6%, i.e. below it.

(cherry picked from commit 2adaac2)
A lazy cluster born old is never swept by a minor, so a DEAD one holds its
whole element graph live through the remembered set until a full collection.
On records_array_16k:scan that full collection never arrives -- the arena
rebaselines its own trigger 134M->268M->536M->1073M while old_in_use climbs
past 48 MB -- and every minor reports survival_permille=996,
copied_objects=0, freed_bytes=0. That is 205 MiB peak RSS against Node's
62 MiB.

GC_TYPE_LAZY_ARRAY was pinned for two reasons, both now removed the way
GC_TYPE_REGEXP removed its own:

  - json_tape_store keys a tape by its owner's address, so a moved header
    orphaned it. Added json_tape_store::owner_moved plus
    GcMoveHookKind::LazyArrayTape, mirroring GcMoveHookKind::RegExpSideTables.
  - the copying minor's flip runs no per-object finalize hook, so a header
    dying young leaked its tape. Added
    finalize_dead_copied_minor_from_space_lazy_tapes, the twin of the
    sweep-entry collect_owners pass, wired into the flip beside
    map/set/errors/regex and reported in the diag line.

With both present the type is movable and the cluster's generation is
decided by cache size -- decided ONCE, so #7546's rule that header, cache
and bitmap share a generation still holds. Large clusters stay old exactly
as before.

Four tests that asserted immovability were retargeted: the large-cluster one
still pins old-gen residency on size, and the handle tests now assert the
stronger property -- that the rooted handle resolves to wherever the
collector left the header, and that alloc_lazy_array returns the refreshed
address rather than the stale one. One test was itself holding a raw header
across a forced evacuation and faulting on the 0xDEADBEEFBAADF0DE poison
fill; it is rooted now.

Dead-owner tape release for a NURSERY header is not handled here -- a dead
nursery header can carry a stale GC_FLAG_MARKED from an earlier cycle, so
the full trace's dead-owner predicate, which assumed old-gen residency,
never reports it dead. That is mark-bit semantics rather than movability,
so it is the next commit rather than this one.

(cherry picked from commit e199635)
Completes the movability change. Two tape-release tests asserted that a full
mark-sweep reclaims a dead owner, which was true only while every owner was
old-gen: the old-gen sweep finalizes an unmarked payload directly. A
nursery-resident owner is reclaimed by a MINOR instead, through the new
finalize_dead_copied_minor_from_space_lazy_tapes, exactly as Map/Set/Error/
RegExp reclaim theirs. A full sweep leaves nursery mark bits to the minor, so
asserting on it was asserting against the wrong pass -- measured directly: six
consecutive full sweeps released nothing, and the first minor released exactly
the dead owner's bytes.

The same probe confirmed GcMoveHookKind::LazyArrayTape works: across that
minor the surviving owner relocated and the registry followed it to the new
address.

Both tests also held raw headers across collections, which only became visible
once headers could move. They re-read through the roots they already had --
the shadow slots, which the collector rewrites -- rather than keeping the
address owned_small returned. A RuntimeHandleScope is NOT a root in this file
unless register_runtime_handle_root_scanner_for_tests ran, which these two do
not call, so the shadow slot is the correct root to read back from.

3611 runtime tests pass, 0 fail.

(cherry picked from commit ac9aa77)
arena_alloc_gc keeps two large-object lines apart, and its comment says why:
tenuring a POINTER-BEARING object does not cost its own bytes, it costs
"every object it can reach, held live through the remembered set by a
container nothing refers to any more". The sparse cache is a block of
JSValues, so it is exactly that container, and a lazy array is the case the
distinction was drawn for. Use the 128 KB line rather than the flat 16 KB
one -- V8's kMaxRegularHeapObjectSize, inside the copier's own ceilings, so a
cluster admitted by it is always movable.

Peak RSS on records_array_16k:scan, against main: 205 MiB -> 51 MiB, where
Node is 62 MiB and Bun 77 MiB.

test_json_tape_lazy_get_records_its_cache_store_as_an_external_edge needed a
bigger fixture to keep its premise: it wants a born-old cluster, which used
to be free because every lazy header was born old unconditionally. At 4096
elements its cache is 32 KB and would now be nursery-resident, so the test
would still pass its later assertions while exercising none of the
containment branch it exists for. 20 000 elements is ~156 KB, over the line.

3611 runtime tests pass, 0 fail.

(cherry picked from commit 1da6aec)
`json_owned_tape_*`'s post-collection check dereferenced a raw
`get_raw_mut_ptr` across the collection it had just forced, which is one new
raw-handle debt site in a module with no ceiling (#7341) and fails the
per-module ratchet. Read `materialized` inside `with_mut_ptr` instead: the
header may have moved, and the scoped read is the protocol that says so.

This fix was made and verified on this branch before it was first pushed, then
lost to an uncommitted-tree reset while checking an unrelated gate, so the
pushed head still carried the raw site. Ratchet: 944 (baseline 944), exit 0.

(cherry picked from commit 718239b)
Repeated `JSON.parse` of a 50,000-field document held 220 MiB peak RSS
against a live set of ~0. Node holds 136 MiB on the same workload, Bun 71.

`arena/allocators.rs` already names the failure mode: a large pointer-bearing
object is stamped GC_FLAG_TENURED, and a minor never sweeps old-gen, so its
cost "is not its own bytes, it is every object it can reach, held live through
the remembered set by a container nothing refers to any more".

A wide document's property storage and its shape-keys array are exactly that
container. Above 16,384 fields each crosses the 128 KB pointer-bearing
threshold, is born tenured, and then holds its whole field or key set live
long after the document is dead. Measured to the byte with PERRY_GC_CENSUS at
64 parses -- 16,300 fields: 0 retained, 29 MiB; 16,500: 30 retained, 98 MiB;
50,000: 15 retained, 34.3 MB live, 177 MiB. The step lands exactly on the
constant, and the census names the retainer: 15 shape-keys arrays holding
750,000 live strings. The same binary under PERRY_GEN_GC=0 reports 336 bytes
live, which is the truth about the workload.

Admit that storage into the nursery past the threshold, for as long as the
copier can still move it (512 KB -- half a nursery block, inside
copying::MAX_YOUNG_MOVE_BYTES). The scope is read only from the cold
large-object branch of arena_alloc_gc, behind a short-circuiting `&&`, so no
allocation hot path gains work.

wide_1m:parse 220 MiB -> 40 MiB, 0.20s -> 0.15s. Below both engines on RSS
and faster than before on CPU.

SCOPED AND TYPE-MASKED ON PURPOSE. Raising the constant globally reaches the
same 40 MiB but also moves ordinary ARRAY element storage into the nursery,
which other rows neither need nor can afford: records_array_8m:scan 643 ->
710 MiB, 0.91s -> 1.20s. Only a document's own object storage and its keys
array are admitted; array element storage keeps the flat threshold. Measured
in a SINGLE binary (one build, three arms by env) the scoped change is
byte-identical to baseline on records_array_8m:scan, records_array_20m:parse,
records_array_1m:scan, records_array_16k:scan and heterogeneous_1m:parse.

Three tests hardcoded a field count and then asserted pointer_in_old_gen, so
they silently depended on the threshold being 128 KB and failed on their
PREMISE rather than their subject. They now derive their width from the
governing ceiling and keep covering the old-gen path at any value.

perry-runtime: 3627 passed, 0 failed (three consecutive runs).
(cherry picked from commit bb698ff)
#10123)

The direct parser pre-sizes `[{...}]` arrays from `remaining_bytes / 96`,
but clamped that estimate at 16,384 slots: a 131,088-byte allocation, 16
bytes over the 131,072-byte pointer-bearing birth threshold. So every large
record array was born OLD on its first allocation and then doubled twice more
in old-gen (131 -> 262 -> 524 KB for a 59,000-row document). An old array of
young records keeps them alive through the remembered set after the document
dies: on records_object_8m:parse `remembered_set/array` was the origin of 98%
of minor survivors, across three minors and zero fulls.

Use the estimate as-is. One allocation, admitted into the nursery through
`JsonWideBirthScope::arrays()` when it fits the JSON young-birth ceiling, and
a single old allocation past it rather than a chain of four.

The ceiling rises 512 KB -> 768 KB (three quarters of a nursery block, still
inside `arena::BLOCK_SIZE` and `copying::MAX_YOUNG_MOVE_BYTES`) because a
7.1 MB document's estimate is 593 KB. No fixture has object storage between
the two values, so the wide-object path from the previous commit is unchanged.

An earlier attempt admitted every JSON array young, including the doubling
chain's intermediates. It regressed eight cells (records_array_20m:* CPU
+45%) because each abandoned young intermediate still cost a copy. Sizing
once is what removes that cost; this version was measured against the old
clamp in a single binary across all 50 matrix cells:

  records_object_8m:parse           187 -> 118 MiB   (1.68x -> 1.05x best)
  records_array_20m:parse/scan/sparse  256 -> 240 MiB
  records_object_20m:parse          256 -> 240 MiB

records_object_8m:parse CPU 167 -> 206 ms, still 0.88x the better engine.
No other cell moved outside noise.

A unit test pins both outcomes: an under-ceiling estimate is one young
allocation that does not regrow for the rows it was sized for, and a
past-ceiling estimate keeps its old-gen birth.

(cherry picked from commit 0c2876b)
@coderabbitai

coderabbitai Bot commented Sep 13, 2026

Copy link
Copy Markdown

Review Change StackReview Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: d26119a3-fa3b-427f-be51-6cb24ae54138

📥 Commits

Reviewing files that changed from the base of the PR and between 6874a9e and 1718250.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (52)
  • CLAUDE.md
  • Cargo.toml
  • changelog.d/10098-json-lazy-array-movable-and-brand.md
  • changelog.d/10123-json-wide-object-birth-generation.md
  • changelog.d/10147-batch-old-page-unregistration.md
  • changelog.d/10150-json-traversal-feedback.md
  • changelog.d/10168-json-parse-depth-in-descent.md
  • changelog.d/10171-json-record-loop-clone.md
  • crates/perry-codegen/src/expr/element_shape_guard.rs
  • crates/perry-codegen/src/expr/index_get/inline_dyn_typed_array.rs
  • crates/perry-codegen/src/expr/mod.rs
  • crates/perry-codegen/src/expr/property_get/helpers.rs
  • crates/perry-codegen/src/expr/shadow_slot.rs
  • crates/perry-codegen/src/runtime_decls/arrays.rs
  • crates/perry-codegen/src/runtime_decls/strings.rs
  • crates/perry-codegen/src/stmt/element_shape_loop.rs
  • crates/perry-codegen/src/stmt/element_shape_loop_tests.rs
  • crates/perry-codegen/src/stmt/element_shape_shape_keyed_tests.rs
  • crates/perry-codegen/src/stmt/let_stmt.rs
  • crates/perry-codegen/src/type_analysis/numeric.rs
  • crates/perry-runtime/src/arena/allocators.rs
  • crates/perry-runtime/src/arena/mod.rs
  • crates/perry-runtime/src/arena/page_meta/mod.rs
  • crates/perry-runtime/src/arena/tests.rs
  • crates/perry-runtime/src/arena/tests_batch_unregister.rs
  • crates/perry-runtime/src/array/element_shape.rs
  • crates/perry-runtime/src/array/element_shape_tests.rs
  • crates/perry-runtime/src/gc/copying_phase.rs
  • crates/perry-runtime/src/gc/oldgen.rs
  • crates/perry-runtime/src/gc/oldgen/sweep_batch.rs
  • crates/perry-runtime/src/gc/tests/alloc.rs
  • crates/perry-runtime/src/gc/tests/helper_stores.rs
  • crates/perry-runtime/src/gc/tests/lazy_tape_side_alloc.rs
  • crates/perry-runtime/src/gc/tests/runtime_roots/callback_scanners.rs
  • crates/perry-runtime/src/gc/tests/runtime_roots/json_construction.rs
  • crates/perry-runtime/src/gc/tests/runtime_roots/json_key_lifetime.rs
  • crates/perry-runtime/src/gc/tests/runtime_roots/json_tape_owned.rs
  • crates/perry-runtime/src/gc/types.rs
  • crates/perry-runtime/src/json/construction_array.rs
  • crates/perry-runtime/src/json/mod.rs
  • crates/perry-runtime/src/json/parse_api.rs
  • crates/perry-runtime/src/json/parse_reuse.rs
  • crates/perry-runtime/src/json/parser.rs
  • crates/perry-runtime/src/json/traversal_feedback.rs
  • crates/perry-runtime/src/json_tape.rs
  • crates/perry-runtime/src/json_tape_store.rs
  • crates/perry-runtime/src/object/json_construction.rs
  • crates/perry-runtime/src/object/shapes.rs
  • crates/perry-runtime/src/object/shapes_tests.rs
  • scripts/gc_runtime_root_holders.json
  • scripts/local_binding_type_allowlist.json
  • test-files/test_gap_json_record_loop_clone.ts

📝 Walkthrough

Walkthrough

The PR adds shape-keyed optimization for JSON record-array loops, changes JSON depth handling and allocation behavior, makes lazy arrays movable, batches old-page cleanup, adds runtime and integration tests, and updates version and changelog files.

Changes

JSON record-loop specialization

Layer / File(s) Summary
Shape-keyed loop code generation
crates/perry-codegen/src/expr/*, crates/perry-codegen/src/stmt/*
The compiler now supports ordinary ShapeId proofs, runtime property slots, constant indexes, and derived modulo indexes for JSON record arrays.
Runtime shape support
crates/perry-runtime/src/array/*, crates/perry-runtime/src/object/*
The runtime records exact shapes for class-zero records and exposes ordinary-shape inline-slot lookup.
Loop and runtime validation
crates/perry-codegen/src/stmt/*, test-files/*
Tests cover accepted forms, rejected forms, bounds, revocation, non-numeric values, and non-record receivers.

JSON and GC runtime changes

Layer / File(s) Summary
JSON allocation and lazy arrays
crates/perry-runtime/src/gc/*, crates/perry-runtime/src/json/*, crates/perry-runtime/src/json_tape*
Wide JSON storage can be born young within scoped ceilings. Lazy-array clusters use size-based generation, relocation hooks, and dead-tape finalization.
JSON parsing and traversal feedback
crates/perry-runtime/src/json/parse_*, crates/perry-runtime/src/json/parser.rs, crates/perry-runtime/src/json/traversal_feedback.rs
The direct parser tracks depth during descent. Deep failures use the heap-stack parser. Thread-local traversal feedback selects eager parsing with periodic lazy resampling.
Batched old-page cleanup
crates/perry-runtime/src/arena/*, crates/perry-runtime/src/gc/oldgen/*
Dead old-generation headers are queued and removed from page indexes in batches. Tests compare batched and per-object results.

Estimated code review effort: 5 (Critical) | ~120 minutes

Change: Feature · Severity of issue fixed: Medium

Sequence Diagram(s)

sequenceDiagram
  participant JSONParse
  participant DirectParser
  participant TraversalFeedback
  participant JsonTape
  JSONParse->>TraversalFeedback: request eager or lazy route
  JSONParse->>DirectParser: parse and track nesting during descent
  DirectParser-->>JSONParse: parsed result or depth-exceeded status
  JSONParse->>JsonTape: create lazy array when selected
  JsonTape->>TraversalFeedback: report lazy creation and traversal
Loading

Possibly related PRs

  • PerryTS/perry#10022: Adds the parse-reuse depth-validation state that this PR removes while moving depth decisions into direct parsing.
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch train179

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant