Skip to content

Compare arm64 and x86_64 at every memory size - #15

Merged
NimmLor merged 19 commits into
mainfrom
ln/elegant-ritchie-mw7eia
Nov 1, 2026
Merged

NimmLor merged 19 commits into
mainfrom
ln/elegant-ritchie-mw7eia

Conversation

@NimmLor

@NimmLor NimmLor commented Nov 1, 2026 •

Copy link
Copy Markdown
Owner

A new arm64 vs x86_64 section between the Event Functions and the Web Functions. It answers one question: arm64 or x86_64, and does the answer change with memory, runtime or metric? Anything that repeated the section above has moved there or been removed. The result list at the bottom also gets a First answer column, and every column can sort it.

The section

arm64 vs x86_64
At 256 MB arm64 cold-starts 1.46× as fast and costs 31% less per cold start; warm, x86_64 is
up to 1.37× faster below 512 MB. Memory barely moves cold start: by 3% at most.

┌ Advantage by memory ──────────────────── Bun  LLRT  Node.js  All             ┐
│                                          ▒ arm64 faster  ▒ x86_64 faster     │
│  Cold start  │  First answer  │  Warm start  │  Cost per cold start          │
└──────────────────────────────────────────────────────────────────────────────┘
┌ Per function at 256 MB ── [Cold start|First answer|Warm start] [128 256 512 1024 1769] ┐
│ Bun                                         1.43×   1.14×   0.80×   −31%              │
│   runtime        ●━━━━━━━━┃   162  231      1.43×   1.15×   0.76×   −33%              │
│   …                                                                                 │
└──────────────────────────────────────────────────────────────────────────────────────┘
  • Headline. It's computed from the same geometric means as the panels:

    • all Event Functions' cold start and cost per cold start at the picked size;
    • the sizes where x86_64's warm start is more than 5% faster, and by how much at most;
    • how much memory moves cold start: the largest spread across sizes of any family's (or All's) geometric-mean cold start, on either architecture.

    Cost is stated as a result, not as following from the GB-second price; the 20% discount stays in the note at the bottom. With no warm start measured in view, the headline leaves the warm clause out.

  • Advantage by memory. Four small multiples on one shared log axis, symmetric around "even", so the panels compare directly: cold start, first answer, warm start and cost per cold start. They sit in one row on wide screens, 2×2 on medium ones, and stack on a phone. They're computed once per render and shared with the headline.

    • A family's value is the geometric mean of x86_64 ÷ arm64 over its Event Functions that have both at that size. The groups are fixed (Bun, LLRT, Node.js, All) and ignore the Functions filter. Web Functions and MicroVMs are left out.
    • Family lines are solid and told apart by colour alone, using their variants' hue (--color-line-BunCompileAotTrim, -Llrt, -Node24). All is thick and neutral, drawn on top. Dashes no longer mean anything in this section.
    • Halves say who wins: the plots are tinted faint green above even, where arm64 is faster, and faint red below, where x86_64 is, matching the ratio chips. Ticks give only the margin ("2×", "1.5×", "1.25×", "even"), and a legend in the card header names the two halves once for all four panels. On wide screens the panels share one column of tick labels.
    • Clicking a column picks that size for the whole page. Hovering shows each group's verdict ("1.62× arm64"), highest first.
  • Per function. One full-width table at the picked size, with its own size picker:

    • Rows sit under Bun, LLRT and Node.js headings, labelled without the family name (e.g. "AOT executable, trimmed", "24"). The labels come from VARIANTS via shortName, with short: 'no SDK' for Llrt.
    • The dumbbells show cold start, first answer or warm start, picked by a switch beside the size picker and kept for the next visit (useStored('archTableMetric')). The axis is fitted to the picked metric, and the column heading, the card's icon and the values' decimals follow it. The ratio columns keep all four metrics whichever is picked.
    • Markers: arm64 is a green 14 px circle and x86_64 an upright red bar, so the two differ in shape as well as colour. The colours (--color-arch-arm, --color-arch-x86) are the hues of the ok and bad tones the ratio chips use. A thick neutral bar spans the gap between them, and the legend draws the same marks. The values beside the track take the same colours, so the distinction holds on a phone, which hides the track. The runtime's own colour stays on the dot beside its name.
    • Four ratio columns: cold, first answer and warm are green above 1× and red below, so x86_64's warm win at 128 MB reads as a loss. Cost is shown as a difference (−31%).
    • Each family row holds its geometric means. Hovering a row brings that function forward in the charts above.
    • On a phone the dumbbell track is hidden, the ratio headers are one word, and the table scrolls sideways behind sticky names.

Elsewhere on the page

  • At a glance. The arm64 vs. x86_64 row's two tiles were the cold start over every function and over Node.js 24 alone. They are now Cold starts and Warm starts, both over every function: today arm64 is 1.46× faster to cold-start and 1.09× slower once warm. The row links to the section (#arm64-vs-x86_64), on a line of its own.
  • Event Functions section. The absolute first answer, over time and by memory, now has a row there for the toolbar's architecture. Bun's executables are up in 120–170 ms but take 400–670 ms to answer their first invocation at 256 MB, against 200–290 ms for LLRT and Node.js, on both architectures. The over-time chart has data only from the last day of runs, when round trips started being recorded, and fills in as runs accrue.
  • Result list.
    • New column: First answer, after cold start, with the fastest tinted like the leaders in the other columns.
    • Sorting: every figure's heading (Cold start, First answer, Warm start, Memory used, Duration, Size) sorts the list lowest first, a second click reverses it, and the rank follows the order. Functions without a value go last; older runs lack the first answer.
    • Below xl: there's no heading row, so a "Sort by" picker and a reverse button do the same.
    • Layout: the list's column layout now starts at xl (1280 px) rather than lg. At 1024 px an eighth column left the name a few characters wide, and the card layout fits there.

Also

  • Fix: niceMax returned NaN for values just under a power of ten (e.g. 961: 961 × 1.05 > 10 × 100). The first answer over time hit this and drew nothing. It now takes the magnitude of the padded value. Covered by web/src/data.test.ts.
  • Readout's placement moved into Floating (Chart.tsx), which the section's readouts share. Versus's colours moved into tone(), which the table's chips share.
  • data.ts adds ARCHS, PRICES (eu-central-1's first tier from the AWS Price List API: arm64 $0.0000133334, x86_64 $0.0000166667 per GB-second, $0.20 per million requests), coldStartCost, geomean (shared by the section and the glance), FAMILY_NAMES and shortName.
  • tokens.css adds --color-arch-arm and --color-arch-x86, the two architectures' colours, plus the icon colours below. No new per-variant colours.
  • New chart icons: arrows for the first answer (--color-icon-trip) and a coin for cost (--color-icon-cost).

Commits

  1. Keep chart axes finite for values just under a power of ten
  2. Show the first answer in the Event Functions section
  3. Compare the architectures in four small multiples by family
  4. Rebuild the per-function table under family headings
  5. Answer arm64 or x86_64 in a headline above the panels
  6. Add the first answer to the table and sort it by any column, then take it back out: this was meant for the result list, not the architecture table
  7. Show the first answer in the result list and sort it by any column
  8. Three passes at the comparison table's markers: colour per architecture, then green and red circles, then arm64's circle beside x86_64's upright bar
  9. Tint the advantage chart's halves instead of naming each tick
  10. Switch the comparison table's dumbbells between three metrics
  11. Compare cold and warm starts in the glance's arm64 row, and give its link a line of its own
  12. Update Architectures.tsx (a shorter methodology note)
  13. Tidy the arm64 section after review

(The two commits below them added the section's first version.)

Checked

  • bun run typecheck, bun lint --max-warnings 0 (canary), bun fmt --check, bun test (26 pass)
  • Rendered against the production /runs/history at 1440, 1280, 1024, 900 and 390 px wide: no horizontal page scroll and no console errors. Readouts, row focus and size picking work, and the headline updates with the picked size. The list sorts and reverses by first answer from the headings at xl and from the picker below it. The comparison table's dumbbells switch between all three metrics, and the glance's tiles read cold and warm, at 1440 and 390 px.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DdeTv6cEp6YBuNxEGc9Mum

claude and others added 19 commits November 1, 2026 13:57
A section below the Event Functions draws each shown runtime on both
architectures across every memory size, arm64 solid and x86_64 dotted
in the runtime's colour: cold start, warm start, the cold invocation's
round trip, and what a million cold starts cost at eu-central-1's
first-tier prices, now that Lambda bills the init on every runtime.
Beside the cold start, a table of both architectures at the picked
size with how many times as fast arm64 is; hovering a row brings its
lines forward. The readout's placement moves into Floating so the
section's two-column readout can share it, and the glance's arm64
tiles link to the section.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DdeTv6cEp6YBuNxEGc9Mum
A full-width card at the top of the arm64 vs x86_64 section plots
x86_64's value over arm64's for Bun, LLRT, Node.js 24 and every Event
Function together, each the geometric mean over its variants measured
on both architectures at that size. The log axis is symmetric around a
1x line, as far as the largest advantage either way with room for the
labels of its halves. A switcher shows the cold or warm start and is
kept for the next visit; clicking a size picks it for the whole page.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DdeTv6cEp6YBuNxEGc9Mum
niceMax took the magnitude of the value rather than of the value with
its 5% headroom, so for 960, whose padded 1009 is past 10 × 100, no
step matched and the axis came out NaN. The first answer over time
reaches 961 ms and drew no lines at all.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DdeTv6cEp6YBuNxEGc9Mum
The cold invocation's round trip, over time and by memory, for the
toolbar's architecture, in a row of its own under the warm start: Bun's
executables are up in 120-170 ms but answer their first invocation in
500-900 ms, against about 250 ms for LLRT and Node.js, on both
architectures. The arm64 vs x86_64 section drops its absolute copy and
will show first answer only as a ratio.

MemoryChart takes its height from the metric, which equals what its
two charts had, so the wide first answer row can be shorter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DdeTv6cEp6YBuNxEGc9Mum
The advantage chart shows cold start, first answer, warm start and cost
per cold start side by side on one log axis, so arm64's cold start win
and x86_64's warm start win at small sizes show at once rather than
behind a toggle. Each family is a solid line in its variants' hue, and
All is thick and neutral; ticks name the winner (1.5x arm64, even,
1.5x x86_64). The three per-variant charts and the solid/dotted legend
go: 26 lines each repeated the section above, and their dashes meant
the architecture where everywhere else they mean the family.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DdeTv6cEp6YBuNxEGc9Mum
Each shown Event Function at the picked size, under Bun, LLRT and
Node.js headings, by its name without the family's (VARIANTS' short for
LLRT's no-SDK build), so none is cut off. The cold start is a dumbbell
on one ms axis, arm64 filled and x86_64 hollow, with both values, and
four columns give arm64's advantage in cold start, first answer, warm
start and cost: green above 1x, red below, so x86_64's warm start win
at small sizes reads as a loss, and cost as a difference. A family's
row holds its geometric means. The table takes the page width, picks a
size like the toolbar, and on a phone drops the track and scrolls
sideways beside its names.

Versus's colours move into tone(), which the chips share.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DdeTv6cEp6YBuNxEGc9Mum
A sentence from the same geometric means as the panels replaces both
the introduction and the table's footer: all Event Functions' cold
start and cost per cold start at the picked size, where x86_64's warm
start is more than 5% faster and by how much, and how little memory
moves cold start, as the largest spread across sizes of a family's mean
cold start on either architecture, which keeps the one fact of the
removed cold start chart. Cost reads as a result, not as following from
the GB-second price, which the note at the bottom keeps.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DdeTv6cEp6YBuNxEGc9Mum
A second dumbbell beside the cold start shows each function's first
answer on both architectures, on an axis of its own. Every column
heading sorts the table: the dumbbells by arm64's value, fastest first,
the ratios by arm64's advantage, largest first; a second click reverses
it and a third brings the families back. Sorted, the table lists every
function by its full name without the family headings, so a sort ranks
the runtimes against each other: by first answer, LLRT and Node.js all
come before every Bun build.

fullName gives LLRT's no-SDK build its "no SDK" there too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DdeTv6cEp6YBuNxEGc9Mum
They belong in the list at the bottom of the page, which ranks every
function; this reverts "Add the first answer to the table and sort it
by any column".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DdeTv6cEp6YBuNxEGc9Mum
The list at the bottom gets a First answer column, the cold invocation's
round trip, beside the cold start, with the leader tinted like the
others. Every figure's heading now sorts the list, lowest first, and a
second click reverses it; the rank follows the order. Below xl, which
has no heading row, a Sort by picker and a button to reverse do the
same. By first answer, LLRT and Node.js come before every Bun build.

The column layout starts at xl rather than lg: at 1024 px an eighth
column left the name a few characters wide, and the cards fit there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DdeTv6cEp6YBuNxEGc9Mum
A filled and a hollow circle in the runtime's colour told arm64 from
x86_64 by shape alone. Each value is now a slim upright pill in its
architecture's colour, the same on every row: a clear cyan for arm64,
the one the section measures, and a quiet slate for x86_64, the
baseline. The bar between them fades from one to the other, so the gap
reads as which way it goes. The values take the same colours, so a
phone, which shows no track, keeps the distinction, and the runtime's
colour stays on the dot by its name.

The values' heading splits arm64 and x86_64 into flex items: as one
run, the font's contextual alternates turned "4 x8" into "4 ×8".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DdeTv6cEp6YBuNxEGc9Mum
The slim pills were too faint to find. Each value is now a solid
14 px circle, arm64 green and x86_64 red, the hues of the ok and bad
tones the ratio chips use, ringed off a thick neutral bar that spans
the gap between them. The legend and the values take the same colours.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DdeTv6cEp6YBuNxEGc9Mum
arm64 keeps its green circle; x86_64's value is an upright red bar, so
the two differ in shape as well as colour, and the legend draws the
same marks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DdeTv6cEp6YBuNxEGc9Mum
The ticks named the winner on every line, so each architecture showed
three times down the axis. They now give the margin alone (2x, 1.5x,
1.25x, even), and the plots' halves say who wins: a faint green above
even, where arm64 is faster, and a faint red below, where x86_64 is, as
the ratio chips colour them. A legend in the card's header names the
two halves once for all four panels, and the axis column narrows to fit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DdeTv6cEp6YBuNxEGc9Mum
A Cold start / First answer / Warm start switch beside the table's
size picker sets what the dumbbells show, on an axis fitted to it, with
the column's heading, the card's icon and the values' decimals to
match. The choice is kept for the next visit like the toolbar's. The
ratio columns keep all four metrics whichever is picked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DdeTv6cEp6YBuNxEGc9Mum
The arm64 vs. x86_64 tiles showed two cold start ratios, over every
function and over Node.js 24 alone. They now show the cold start and
the warm start over every function, which tells the section's main
result at a glance: arm64 is about 1.46x faster to cold-start and
about 1.09x slower once warm.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DdeTv6cEp6YBuNxEGc9Mum
At the end of the description, "Each size and runtime" broke across
two lines at most widths. It now sits under the paragraph, at the foot
of the tile when the row is wide.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DdeTv6cEp6YBuNxEGc9Mum
- The headline no longer says "warm, arm64 is faster too" when no run
  in view measured a warm start: every() on no values was true. It
  leaves the warm clause out instead.
- The advantage panels are computed once per render, for the chart and
  the headline both, rather than three of them again for the headline.
- One index of the Event Functions serves advantage() and coldSpread(),
  which loops over ARCHS instead of its own list.
- geomean moves to data.ts, and the glance's arm64 tiles use it rather
  than averaging the logs themselves.
- The table's ratio cells follow the order of the record they're built
  from, the headings' order, instead of a list of their own.
- The result list's layout comment is wrapped at 100 columns.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DdeTv6cEp6YBuNxEGc9Mum
@NimmLor
NimmLor marked this pull request as ready for review November 1, 2026 19:57
@NimmLor
NimmLor merged commit 338d468 into main Nov 1, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants