199. The final benchmark matrix

Three questions, one run

By the end of Part XI, three comparisons were each measured on a different day, in a different way: Viscosity on Node, the TypeScript engine against the Rust one, and the three Drips. A benchmark matrix runs them all from one script, on one machine, in one sitting, so they can sit side by side.

The script is engine/ts/packages/bench/src/matrix.ts. It writes results/matrix-2026-10-31.json and generates the .md next to it from that file, with a header saying not to edit it by hand. Every figure on this page, in the chart and in the text, is read from that JSON.

How each cell was measured

The machine is a shared cloud VM with 4 vCPUs. Other tenants use the same CPUs, so the file itself warns that timings vary by up to about 2× between runs, and that differences under about 10% are noise. Against that, the end-to-end section runs each program in 9 fresh processes per version per engine, original and compiled interleaved, after one discarded run of each (chapter 24’s rules). It reports the best time, the most stable figure, and the median beside it. Every compiled program printed the same checksum as its original, on both engines, and the Rust CLI’s compiled code was byte-identical to the TypeScript engine’s for all five.

Bun 1.4.4 is the second engine; it runs JavaScriptCore, the engine of chapter 69. One detail shows why the file records its settings: the VM’s shell set BUN_OPTIONS=--smol, a memory-saving mode, and the script removed it so that Bun ran with its defaults.

The results

On Node, Viscosity speeds up every program, with a geometric mean of 2.54×. On Bun the mean is 1.74×, and heat goes from 4.11× on V8 to 1.02× on JSC: no gain at all. nbody comes out at 0.97× on Bun, also within the noise. The same compiled code, checked to behave the same, pays off on one engine and not on the other.

The third step shows where the difference is: in the originals. Bun runs the unchanged array-of-objects programs 1.71× faster than Node, and the compiled ones only 1.17× faster. Heat’s original takes 352.4 ms on Node and 96.4 ms on Bun. After Viscosity, the two engines are close. JSC was already fast on objects; V8 was not.

A likely cause, not yet confirmed

FINDINGS.md in the bench package, section 7, offers an explanation and labels it carefully as a likely cause, “not yet confirmed with engine traces”. Here it is, with the sources for the facts it rests on.

The facts are documented. In V8, a field that holds doubles gets the Double representation, and its value sits in a separate box on the heap that only that object points to^[1] (chapter 22). JavaScriptCore stores a double in the value itself: its JSValue encodes doubles by adding 2^49 to their bits, so a double needs no box^[2] (chapter 69). A Float64Array holds raw doubles on both.

The hypothesis joins them. If every c[i].h on V8 is one more pointer to follow, then SoA on V8 fixes two things, the storage and the locality, and on JSC only the locality. Heat reads four neighbours of every cell in every step, so the extra pointer would hurt it most. Section 5 of the same file fits: on V8, packing the fields into one Float64Array recovered much of SoA’s gain without changing the layout. What would confirm it is a trace of what V8 actually does with heat’s objects, and nobody has taken one yet.

The lesson for the cost model holds either way. Its rules about size and access patterns still predict whether a layout helps on these programs, but not how much, because that depends on an engine the compiler never sees. That is why explain says why a collection was transformed, and never promises a speed-up.

An older file, and the compiler’s speed

Track A’s chapters 111 and 120 (and the repository’s README) quote an earlier run, results/e2e-node-2026-10-31.json. It is Node only, a median of three runs, and it was made before the fuzzer fixes of Part XI. One of those fixes changed heat’s compiled code: a field write whose right-hand side may throw now stores late, through a small helper (__vsc_set), so that an error comes out in the original’s order. The matrix measured the code as it is now. The two files used different methods too, so their figures shouldn’t be compared cell by cell.

The compile-speed section (chapter 192 goes deeper) has two numbers that are easy to mix up. Run as command-line tools, one process per file over the 40 corpus programs, the Rust CLI is 10.9× faster. But the TypeScript CLI’s total of 4764.6 ms is mostly Node starting and loading a 1 MB bundle: inside one warm Node process, the TypeScript engine compiled all of them in about 15 ms in total. There, the Rust engine running as WebAssembly in the same Node is 5.26× faster. Same engines, different questions. That second figure carries a note, added to the results file afterwards: the WebAssembly side’s timed region didn’t copy the output strings out of linear memory, which the TypeScript side never has to do. matrix.ts now does, but the timing hasn’t been re-run, so treat it as an upper bound.

Your turn

// One program in two implementations:
// milliseconds, or None if it failed.
struct Row {
    old: Option<f64>,
    new: Option<f64>,
    // runs for a fixed time: no speed
    fixed: bool,
}

// The geometric mean of old / new over
// the rows that have a speed-up at all.
fn geomean(rs: &[Row]) -> Option<f64> {
    todo!()
}

What you learned

• A benchmark matrix runs every comparison with one method on one machine, and records the versions and the noise with the numbers.
• Each end-to-end cell is the best of 9 fresh processes, interleaved, with the checksum checked on both engines.
• Viscosity gains on V8 across the board and much less on JavaScriptCore, where heat gains nothing. The originals explain it: JSC was already fast on them.
• The heap-number explanation is a hypothesis, built on documented facts about both engines, and not yet confirmed.
• The CLI compile-speed ratio mostly measures start-up; the in-process ratio measures the engines.