Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 3 additions & 9 deletions TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -700,11 +700,6 @@ bounded and consumer-driven as planned below.
marking are now young and reclaimed at remark, so a long cycle costs memory
held until then rather than promoted garbage; find what bounds progress per
step.
- [ ] Speculate int32 loop state that enters as a double. A direct entry with a
Number parameter keeps `seed = (Math.imul(seed, a) + b) | 0` in a double
phi, so every iteration converts to int32 and back: `serial-compute` runs
995 ms against Node's 204. Versioning the loop on an int32 entry value would
keep the chain in integers; a boxed seed already decodes int-first.
- [ ] Keep int32 elements integral in dense-array loop regions. The region types
every element as a double, so `(sum + values[i]) | 0` over small integers
costs 5 ns per element against Node's 0.5. Restarting into the generic loop
Expand Down Expand Up @@ -1166,10 +1161,9 @@ only reproducible input.
opaque callee that returns its strict `arguments` costs 340 ns per call against
Node's 6.6 (`call-arguments-escape`). Dense storage needs define, delete,
freeze, and own-key order to understand it, and a move to the table once an
index gets non-default attributes. In V8's EarleyBoyer, creating arguments
objects takes 19% of the main thread's samples: its `var a = arguments`
loops read `a[i]` with a dynamic index, which stays virtual only under
locked primordials, and the suite needs mutable ones.
index gets non-default attributes. Unmapped in-range numeric reads can use
the incoming slice under either primordial policy; escaping or materialized
arguments objects still need dense indexed storage.

- [ ] Deterministically release nonescaping RegExp and ICU handles at compiled scope
end once ownership analysis proves their lifetime. Retain finalization for every
Expand Down
19 changes: 19 additions & 0 deletions docs/decisions/runtime-region-contracts.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,13 @@ the same materialization contract. The cached heap object becomes authoritative
before invoking either observer, including when the builtin changes descriptors
or returns the same object. Callee and receiver positions require separate proof.

An unmapped arguments binding can read an in-range numeric index from its incoming
slice while the binding remains unmaterialized. The invocation retains and roots
the entire slice, including extra arguments, through its last read. Other keys and
materialized bindings use the arguments object's property semantics. This read
belongs to its activation, so transformations cannot move it into another function
or use an entry that retains only declared parameters.

Fixed TypedArray regions preserve memory-operation order for overlapping views.
Read-only reductions retain the same storage guards without requiring writable
buffers. Every view contributes its complete visited index window, including all
Expand All @@ -123,6 +130,18 @@ store. Collecting initializers retain the owner as an ordinary rooted value and
use per-store barriers; no backing pointer survives their calls. A handler that
could observe partially changed Array length prevents that admission.

Safe-integer regions also retain certified `Math.imul` and `Math.clz32` calls in
integer locals. Entry guards reject non-integer inputs and negative zero before
the original loop performs any coercion. Integer arithmetic failure restores the
iteration's starting values, while polling publishes completed iterations.

Dense Array loops may overwrite a present own element when the storage has uniform
writable descriptors. An overwrite must be the iteration's sole store, followed
only by local work that cannot restart it. Independent indices check each access
before the store commits; holes, accessors, coercions and unsupported descriptors
resume the original iteration. Replacing a heap value preserves the deletion
barrier. Borrowed storage ends at a poll, after publishing the completed iteration.

DataView accessor calls join fixed storage regions as byte views. Admission uses the
conditions that make `byteLength` the intrinsic getter: the watched-prototype
protector, the intrinsic prototype, an empty own layout, and a fixed view over an
Expand Down
1 change: 1 addition & 0 deletions runtime/src/generated/bytecode_operations.inc
Original file line number Diff line number Diff line change
Expand Up @@ -127,3 +127,4 @@ BYTECODE_OPERATION(PREPARED_STRING_COMPARE)
BYTECODE_OPERATION(PRECISE_NUMBER_SUM)
BYTECODE_OPERATION(STRING_CONCAT_LENGTH)
BYTECODE_OPERATION(CONCAT_STRINGS)
BYTECODE_OPERATION(LOAD_DYNAMIC_ARGUMENT)
5 changes: 3 additions & 2 deletions runtime/src/perf_stats.c
Original file line number Diff line number Diff line change
Expand Up @@ -1046,8 +1046,9 @@ static void mal_perf_stats_print(void) {
(unsigned long long) mal_perf_stats.fixed_typed_array_region_views[3],
(unsigned long long) mal_perf_stats.fixed_typed_array_region_views[4],
(unsigned long long) mal_perf_stats.fixed_typed_array_affine_regions);
fprintf(stderr, "[perf-dense-array-region-stats] admissions=%llu\n",
(unsigned long long) mal_perf_stats.dense_array_number_region_admissions);
fprintf(stderr, "[perf-dense-array-region-stats] admissions=%llu element-stores=%llu\n",
(unsigned long long) mal_perf_stats.dense_array_number_region_admissions,
(unsigned long long) mal_perf_stats.dense_array_number_region_stores);
fprintf(stderr, "[perf-typed-array-stride-stats] admissions=%llu\n",
(unsigned long long) mal_perf_stats.fixed_typed_array_stride_regions);
fprintf(stderr, "[perf-data-view-region-stats] views=%llu\n",
Expand Down
1 change: 1 addition & 0 deletions runtime/src/perf_stats.h
Original file line number Diff line number Diff line change
Expand Up @@ -241,6 +241,7 @@ typedef struct MalPerfStats {
u64 fixed_typed_array_stride_regions;
u64 fixed_data_view_region_views;
u64 dense_array_number_region_admissions;
u64 dense_array_number_region_stores;
u64 record_iteration_region_admissions;
u64 record_columns_admissions;
u64 record_columns_flushes;
Expand Down
6 changes: 6 additions & 0 deletions runtime/src/vm.c
Original file line number Diff line number Diff line change
Expand Up @@ -2739,6 +2739,12 @@ static void mal_vm_run_until_frame_count(
MAL_VM_INTERPRETER_DIRECT_LEAF();
continue;
}
MAL_VM_CASE(LOAD_DYNAMIC_ARGUMENT)
registers[instruction->as.load_dynamic_argument.dst] =
mal_vm_load_dynamic_argument(frame->arguments, frame->argument_count,
registers[instruction->as.load_dynamic_argument.key]);
MAL_VM_INTERPRETER_DIRECT_LEAF();
continue;
MAL_VM_CASE(LOAD_STATIC_ARGUMENT) {
i32 index = instruction->as.load_static_argument.index;
if (index < frame->argument_count) {
Expand Down
4 changes: 4 additions & 0 deletions runtime/src/vm.h
Original file line number Diff line number Diff line change
Expand Up @@ -358,6 +358,10 @@ typedef struct MalInstruction {
i32 dst, index;
} load_argument;

struct {
i32 dst, key;
} load_dynamic_argument;

struct {
i32 dst, direct, fallback, index;
} load_static_argument;
Expand Down
16 changes: 14 additions & 2 deletions runtime/src/vm_load.c
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
*/

#define WIRE_MAGIC 0x574c414du // "MALW" little-endian
#define WIRE_VERSION 84u
#define WIRE_VERSION 85u
#define WIRE_FLAG_HAS_DEBUG 1u

typedef enum WireOp {
Expand Down Expand Up @@ -777,6 +777,11 @@ static void rd_instruction_operands(Rd *r, MalInstruction *o, I32Builder *side_d
r->ok = false;
}
return;
case WIRE_LOAD_DYNAMIC_ARGUMENT:
o->opcode = MAL_OP_LOAD_DYNAMIC_ARGUMENT;
o->as.load_dynamic_argument.dst = rd_i32(r);
o->as.load_dynamic_argument.key = rd_i32(r);
return;
case WIRE_LOAD_STATIC_ARGUMENT:
o->opcode = MAL_OP_LOAD_STATIC_ARGUMENT;
o->as.load_static_argument.dst = rd_i32(r);
Expand Down Expand Up @@ -1628,6 +1633,7 @@ static bool mal_loaded_instruction_writes_register(
MAL_WRITES_DST(MAL_OP_CREATE_ARGUMENTS_OBJECT, create_arguments_object);
MAL_WRITES_DST(MAL_OP_LOAD_ARGUMENT_COUNT, load_argument_count);
MAL_WRITES_DST(MAL_OP_LOAD_ARGUMENT, load_argument);
MAL_WRITES_DST(MAL_OP_LOAD_DYNAMIC_ARGUMENT, load_dynamic_argument);
case MAL_OP_LOAD_STATIC_ARGUMENT:
return instruction->as.load_static_argument.dst == target_register ||
instruction->as.load_static_argument.fallback == target_register;
Expand Down Expand Up @@ -1791,7 +1797,8 @@ static i32 argument_retention_limit(const MalFunction *fn) {
if (instruction->opcode == MAL_OP_CREATE_ARGUMENTS_OBJECT ||
instruction->opcode == MAL_OP_CREATE_REST_ARGUMENTS ||
instruction->opcode == MAL_OP_CALL_REST_ARGUMENTS ||
instruction->opcode == MAL_OP_LOAD_ARGUMENT) {
instruction->opcode == MAL_OP_LOAD_ARGUMENT ||
instruction->opcode == MAL_OP_LOAD_DYNAMIC_ARGUMENT) {
return INT32_MAX;
}
if (instruction->opcode == MAL_OP_LOAD_STATIC_ARGUMENT &&
Expand Down Expand Up @@ -2924,6 +2931,11 @@ MalLoadedRuntimeImage *mal_runtime_image_load_with_host_resolver(
r.ok = false;
}
}
} else if (instruction->opcode == MAL_OP_LOAD_DYNAMIC_ARGUMENT) {
i32 dst = instruction->as.load_dynamic_argument.dst;
i32 key = instruction->as.load_dynamic_argument.key;
if (dst < 0 || dst >= fn->register_count ||
key < 0 || key >= fn->register_count) r.ok = false;
} else if (instruction->opcode == MAL_OP_GUARD_SERVICE_EPOCH) {
if (instruction->as.guard_service_epoch.dst < 0 || instruction->as.guard_service_epoch.dst >= fn->register_count || instruction->as.guard_service_epoch.epoch < 0 || instruction->as.guard_service_epoch.epoch >= fn->register_count) r.ok = false;
} else if (instruction->opcode == MAL_OP_READ_SERVICE_EPOCH) {
Expand Down
37 changes: 32 additions & 5 deletions runtime/src/vm_ops.h
Original file line number Diff line number Diff line change
Expand Up @@ -500,6 +500,17 @@ void mal_op_load_argument_count(MalCallable *callable, const MalInstruction *ins
void mal_op_load_argument(MalCallable *callable, const MalInstruction *instruction);
void mal_op_load_static_argument(MalCallable *callable, const MalInstruction *instruction);

/** Read an incoming value without observing an arguments object's properties. */
static inline MalValue mal_vm_load_dynamic_argument(
const MalValue *arguments, i32 count, MalValue key
) {
if (!mal_ops_is_number(key)) return MAL_VALUE_UNDEFINED;
f64 number = mal_ops_number_as_f64(key);
if (!(number >= 0.0 && number < (f64) count)) return MAL_VALUE_UNDEFINED;
u32 index = (u32) number;
return number == (f64) index ? arguments[index] : MAL_VALUE_UNDEFINED;
}

// Build an arguments object over `args`; a non-empty map aliases indexed
// properties to captured formal bindings. Shared by both execution backends.
MalValue mal_create_arguments_object(
Expand Down Expand Up @@ -2723,19 +2734,21 @@ static inline bool mal_vm_fixed_numeric_storage(
}

/**
* A dense Array's elements below `count`, borrowed until a poll like typed-array
* bytes. A hole or non-Number element is not an own Number, so its read declines.
* Dense Array storage borrowed until a poll. Reads establish their own value kind;
* writable borrows overwrite only present elements and preserve the deletion barrier.
*/
typedef struct MalDenseNumberSpan {
const MalValue *elements;
MalValue *elements;
u32 length;
u32 count;
} MalDenseNumberSpan;

static inline bool mal_vm_dense_number_storage(MalValue receiver, MalDenseNumberSpan *span) {
static inline bool mal_vm_dense_number_storage(
MalValue receiver, bool writable, MalDenseNumberSpan *span
) {
if (!mal_value_is_heap_type(receiver, MAL_HEAP_ARRAY_OBJECT)) return false;
const MalArrayObject *array = (const MalArrayObject *) mal_value_to_heap(receiver);
if (array->dense_deopted) return false;
if (array->dense_deopted || (writable && !array->dense_elements_writable)) return false;
*span = (MalDenseNumberSpan) {
.elements = array->elements,
.length = array->length,
Expand All @@ -2745,6 +2758,20 @@ static inline bool mal_vm_dense_number_storage(MalValue receiver, MalDenseNumber
return true;
}

static inline bool mal_vm_dense_number_store(
const MalDenseNumberSpan *span, u32 index, f64 value
) {
if (index >= span->count) return false;
MalValue *slot = &span->elements[index];
MalValue previous = *slot;
if (mal_value_is_array_hole(previous)) return false;
// Number stores introduce no young edge, but replaced heap values still need SATB.
mal_gc_write_barrier(previous);
*slot = mal_ops_number_value(value);
MAL_PERF_COUNT(dense_array_number_region_stores);
return true;
}

// Each width scales the index by a constant, which keeps a multiply by the span's
// element size out of every element address.
static inline f64 mal_vm_fixed_numeric_load(const MalTypedArraySpan *span, u32 index) {
Expand Down
2 changes: 2 additions & 0 deletions src/compiler/core/core-cross-call-transforms.ts
Original file line number Diff line number Diff line change
Expand Up @@ -332,6 +332,7 @@ const INLINE_ENVIRONMENT_OPCODES: ReadonlySet<string> = new Set([
]);

const INLINE_UNSUPPORTED_OPCODES = new Set([
"loadDynamicArgument",
"createArgumentsObject",
"createRestArguments",
"createRestArgumentsState",
Expand Down Expand Up @@ -2244,6 +2245,7 @@ function canBridgeInlineResult(
}

const CALLBACK_SPECIALIZATION_EXCLUDED_OPCODES: ReadonlySet<string> = new Set([
"loadDynamicArgument",
"loadCallee",
"loadArgument",
"loadArgumentCount",
Expand Down
3 changes: 3 additions & 0 deletions src/compiler/core/core-ir-opcodes.ts
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,7 @@ export const CORE_OPCODES = [
"iteratorStep",
"loadArgument",
"loadArgumentCount",
"loadDynamicArgument",
"loadCallee",
"loadCaptured",
"loadGlobalIndex",
Expand Down Expand Up @@ -194,6 +195,7 @@ const TWO_OUTPUTS = new Set<CoreOpcode>([
]);

const GC_FREE = new Set<CoreOpcode>([
"loadDynamicArgument",
"createRestArgumentsState",
"guardServiceEpoch",
"readServiceEpoch",
Expand Down Expand Up @@ -714,6 +716,7 @@ const INPUT_ARITIES = {
iteratorNext: [2, 2],
iteratorStep: [2, 2],
loadArgument: [0, 0],
loadDynamicArgument: [1, 1],
loadArgumentCount: [0, 0],
loadCallee: [0, 0],
loadCaptured: [0, 0],
Expand Down
1 change: 1 addition & 0 deletions src/compiler/core/core-local-capture-arguments.ts
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ function helperBody(
opcode === "loadCallee" ||
opcode === "loadArgumentCount" ||
opcode === "loadArgument" ||
opcode === "loadDynamicArgument" ||
opcode === "loadStaticArgument" ||
opcode === "createFunction" ||
opcode === "createArgumentsObject" ||
Expand Down
6 changes: 5 additions & 1 deletion src/compiler/core/core-native-entry-analysis.ts
Original file line number Diff line number Diff line change
Expand Up @@ -533,7 +533,11 @@ export function coreArgumentObservation(fn: CoreFunctionStore): CoreArgumentObse
const header = fn.instructionHeader(instruction);
if (typeof header === "string") continue;
const opcode = header.opcode;
if (opcode === "createArgumentsObject" || opcode === "callRestArguments")
if (
opcode === "createArgumentsObject" ||
opcode === "callRestArguments" ||
opcode === "loadDynamicArgument"
)
return { kind: "general" };
if (opcode === "createRestArguments") {
const startIndex = fn.instructionAttributes(instruction).startIndex;
Expand Down
1 change: 1 addition & 0 deletions src/compiler/core/core-native-field-analysis.ts
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ const argumentProofs = new WeakMap<

// These read the parameter through the activation rather than as an operand.
const PARAMETER_ALIAS_OPCODES = new Set([
"loadDynamicArgument",
"loadArgument",
"loadStaticArgument",
"createArgumentsObject",
Expand Down
1 change: 1 addition & 0 deletions src/compiler/core/core-private-field-elements.ts
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ const CALLBACK_METHODS: ReadonlyMap<string, number> = new Map([

// Operations through which a function can reach arguments it declares no parameter for.
const ARGUMENT_READERS: ReadonlySet<string> = new Set([
"loadDynamicArgument",
"callRestArguments",
"createArgumentsObject",
"createRestArguments",
Expand Down
1 change: 1 addition & 0 deletions src/compiler/core/core-static-value-calls.ts
Original file line number Diff line number Diff line change
Expand Up @@ -166,6 +166,7 @@ function staticParameterPlan(
"loadCallee",
"loadNewTarget",
"loadArgument",
"loadDynamicArgument",
"loadArgumentCount",
"loadStaticArgument",
"createArgumentsObject",
Expand Down
17 changes: 15 additions & 2 deletions src/compiler/core/semantic-lowering.ts
Original file line number Diff line number Diff line change
Expand Up @@ -522,6 +522,7 @@ interface CoreFrontendFunction {
staticArgumentsFallbackRegister?: number;
/** Incoming arguments as a packed rest view for in-range element reads. */
argumentsElementRegister?: number;
directArgumentElements?: true;
/**
* Locked primordials pin the builtin slice, so this function's `arguments` copies
* read the frame and its reads use `argumentsCopyAccesses`; no object exists.
Expand Down Expand Up @@ -5767,6 +5768,14 @@ function compileFunctionParams(
});
}
(fn.staticArgumentsRegisters ??= new Map()).set(usage, argumentCountRegister);
if (
!fn.isGenerator &&
!fn.isAsync &&
!fn.mappedArgumentSlots?.some((slot) => slot >= 0)
) {
fn.directArgumentElements = true;
continue;
}
// Without these facts the view could not stay virtual and would allocate per call.
if (
fn.argumentsElementRegister === undefined &&
Expand Down Expand Up @@ -13438,7 +13447,7 @@ function compileArgumentsElementRead(
const count = fn.staticArgumentsRegisters?.get(usage);
const objectJumps: Array<Extract<CompilerInstruction, { type: "jump" }>> = [];
let viewDone: Extract<CompilerInstruction, { type: "jump" }> | undefined;
if (view !== undefined && count !== undefined) {
if ((fn.directArgumentElements || view !== undefined) && count !== undefined) {
const current = loadRegisterFromLocation(
fn,
cursor.block,
Expand Down Expand Up @@ -13492,7 +13501,11 @@ function compileArgumentsElementRead(
operator: "<",
});
guard(inRange);
cursor.block.emitter.emit({ type: "loadProperty", registers: [result, view, index] });
cursor.block.emitter.emit(
fn.directArgumentElements
? { type: "loadDynamicArgument", registers: [result, index] }
: { type: "loadProperty", registers: [result, view!, index] },
);
viewDone = { type: "jump", blocks: [-1] };
cursor.block.emitter.emit(viewDone);
const objectIndex = fn.blocks.push({ emitter: unboundCoreEmitter }) - 1;
Expand Down
5 changes: 5 additions & 0 deletions src/compiler/shared/compiler-instruction.ts
Original file line number Diff line number Diff line change
Expand Up @@ -347,6 +347,11 @@ type CompilerOperation =
registers: [number];
index: number;
}
| {
/** Read the retained incoming slice; invalid indices produce undefined without coercion. */
type: "loadDynamicArgument";
registers: [number, number];
}
| {
// Direct static index read. Supplied indexes use the optional live mapped
// value; missing indexes lazily materialize and cache the arguments object.
Expand Down
1 change: 1 addition & 0 deletions src/compiler/target/bytecode-operation-spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,7 @@ export const BYTECODE_OPERATIONS = [
"PRECISE_NUMBER_SUM",
"STRING_CONCAT_LENGTH",
"CONCAT_STRINGS",
"LOAD_DYNAMIC_ARGUMENT",
] as const;

export type BytecodeOperation = (typeof BYTECODE_OPERATIONS)[number];
Expand Down
2 changes: 1 addition & 1 deletion src/compiler/target/compiler-artifact-codec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ import type {
/** Host-compiler cache format. This metadata never reaches the VM loader. */
export const COMPILER_ARTIFACT_MAGIC = 0x434c414d; // "MALC" little-endian
// Internal artifacts are hard cut-overs: stale cache entries rebuild.
export const COMPILER_ARTIFACT_VERSION = 370;
export const COMPILER_ARTIFACT_VERSION = 371;

/** Direct entries of C library bindings, as `binding-unit.ts` names them. */
const FOREIGN_CALL_SYMBOL = /^mal_ffi_direct\d+_\w+$/;
Expand Down
2 changes: 2 additions & 0 deletions src/compiler/target/emit-program-image.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3275,6 +3275,8 @@ function emitInstruction(instruction: BytecodeInstruction, dataOffset?: number)
return `{ .opcode = MAL_OP_LOAD_ARGUMENT_COUNT, .as.load_argument_count = { .dst = ${instruction.dst} } }`;
case "LOAD_ARGUMENT":
return `{ .opcode = MAL_OP_LOAD_ARGUMENT, .as.load_argument = { .dst = ${instruction.dst}, .index = ${instruction.index} } }`;
case "LOAD_DYNAMIC_ARGUMENT":
return `{ .opcode = MAL_OP_LOAD_DYNAMIC_ARGUMENT, .as.load_dynamic_argument = { .dst = ${instruction.dst}, .key = ${instruction.key} } }`;
case "LOAD_STATIC_ARGUMENT":
return `{ .opcode = MAL_OP_LOAD_STATIC_ARGUMENT, .as.load_static_argument = { .dst = ${instruction.dst}, .direct = ${instruction.direct}, .fallback = ${instruction.fallback}, .index = ${instruction.index} } }`;
case "LOAD_THIS":
Expand Down
Loading
Loading