101. The cost model

Safe is not the same as worth it

Here is one of the test programs in Viscosity’s corpus, small_auto.js:

// viscosity: pts=aos
const pts = [{ x: 1, y: 2 },
  { x: 3, y: 4 }];
let sx = 0;
for (let i = 0; i < pts.length; i++)
  sx += pts[i].x;
console.log(sx);

The first line is the corpus header from chapter 88: the test expects pts to stay an array of objects. Yet every check from the last four chapters passes. The records have one stable shape, both fields are numbers, the loop reads x, and nothing can observe the layout. Viscosity could turn pts into two Float64Array columns. It doesn’t:

$ viscosity explain small_auto.js
pts (line 2) {x, y} → kept as array
  of objects
  · about 2 elements: below the
    64-element threshold, where the
    benefit is too small

(Explain output is wrapped to fit the page throughout this chapter.)

A layout change has costs: a generated class, rewritten accesses, and a program that is harder to read in a debugger. For two points the loop takes nanoseconds either way, so nothing pays those costs back.

The ladder

The planner in core/src/plan.ts decides one collection at a time. It asks a fixed series of questions, and the first one answered “yes” decides. Some of the questions are about permission, and the rest are the cost model. Step through them in order:

Here is the same ladder as code, simplified from plan() (the reason strings are shortened here):

if (c.bailouts.length)
  return keep('unsafe: …');
if (ann) return ann.layout;
if (opts.mode === 'opt-in')
  return keep('opt-in mode: …');
if (python && !opts.pythonAuto)
  return keep('CPython: …');
if (c.numericFields.size === 0)
  return keep('no field is …');
if (!c.loops.some((l) => l.depth > 0))
  return keep('no loop reads …');
const measured =
  opts.profile?.[c.name]?.maxLength;
const size = measured ?? c.sizeEstimate;
const about = measured === undefined
  && c.sizeIsUpperBound
  ? 'at most' : 'about';
if (size !== null && size < opts.minSize)
  return keep(`${about} ${size} …`);
return 'soa';

The order matters. Safety comes before everything, so not even an annotation can force an unsafe change. An annotation comes before the cost rules, because it is the programmer overruling the guess.

Three rules, and where they came from

Once permission is settled, the cost model is just three questions.

Is any field provably numeric? If not, every column would be a plain JavaScript array of values. Part of SoA’s measured advantage comes from typed storage: in this book’s benchmark run, a packed Float64Array that keeps the records together already captures a large share of the win (see FINDINGS.md, section 5). Plain-array columns get none of that, so the rule says no.

Does a loop read or write a field? A layout only pays off through work that is repeated. A collection whose fields are touched once, or never, is kept.

Is it big enough? Here is the measured basis. The same x += vx loop, at two sizes:

In this run SoA was about 1.7× faster at 1,000 elements and 3.8× faster at 100,000 (results/baseline-node-2026-10-31.md). The win grows with size because the cache stops hiding the waste in each object.

Be clear about what that does and doesn’t prove. The benchmark never measured fewer than 1,000 elements, so the default threshold of 64 is not read off a chart. It is a conservative judgment. Sixty-four records of two doubles are 1 KB, sixteen 64-byte cache lines, and a loop over them is over too quickly for any layout to matter. You can change it with --min-size.

Guessing the size

The planner needs a size before the program runs. The analysis counts the literal elements, then adds each push, multiplied by the trip count of every loop around it, when the loop is a simple for (let i = 0; i < 1000; i++) with a constant bound. A push under an if may not run every time, so the count is a maximum, and explain says “at most 40 elements” rather than “about”. A push inside a function makes the size unknown, because the analysis doesn’t know how often the function is called. Call this program spawn.js:

const ps = [];
function spawn(k) {
  for (let i = 0; i < k; i++)
    ps.push({ x: i, v: 1 });
}
spawn(10);
for (let i = 0; i < ps.length; i++)
  ps[i].x += ps[i].v;
console.log(ps[3].x);

ps (line 1) {x, v} → struct of arrays
  · loop at line 7 in <module>()
    touches 2 of 2 fields (v, x)
  · typed Float64Array columns for
    all 2 fields
  · size unknown statically
  · 2 of 3 element accesses proven
    in bounds

An unknown size passes the threshold. The planner is optimistic: when it can’t tell, it assumes the collection may be large. Here it is wrong, because ps holds ten points.

Measuring fixes that. viscosity profile runs the original program with a little tracking added, and writes down how long each collection was at the end. Chapter 109 builds it. Fed back in, the measured size replaces the estimate:

$ viscosity profile spawn.js \
    -o prof.json
$ cat prof.json
{
  "ps": {
    "maxLength": 10
  }
}
$ viscosity explain spawn.js \
    --profile prof.json
ps (line 1) {x, v} → kept as array
  of objects
  · about 10 elements (measured):
    below the 64-element threshold,
    where the benefit is too small

Try it

This program transforms ws. Then try two changes, one at a time. Make n a string (n: 'x' + i), so that no field is numeric. Or delete the second loop, and print ws[9].w instead. Each change stops the transform at a different rung, with its own reason.

What the model leaves out

It’s worth being blunt: this is a simple model. It doesn’t look at:

• How hot a loop is. Any loop counts, even one that runs once, or lives in a function nobody calls. The analysis records trip counts, but the planner doesn’t use them.
• How many fields a loop touches. Explain prints “touches 1 of 2 fields”. SoA gains most when a loop uses few of many fields, but the planner doesn’t weigh that.
• Costs SoA adds. Bounds checks, the generated code, and rebuilding objects (materialization) don’t enter the model. Today the safety analysis refuses any collection whose elements would need rebuilding, so that last cost never arises.
• AoSoA. The planner never chooses it alone; only an annotation can.
• The engine. The thresholds come from one run of Node on one machine.

The end-to-end results show why that matters. All five benchmark programs were transformed, and in this book’s run each got faster, by anything from 1.09× (nbody) to 5.11× (ledger) (results/e2e-node-2026-10-31.md). The model had no way to predict which would gain 9% and which fivefold.

Your turn

function planFor(c, opts) {
  // Check the rules in order.
  // The first one that matches wins.
  return ['soa', 'soa'];
}

function estimate(lits, pushes) {
  return lits + pushes.length;
}

What you learned

• A cost model estimates whether a safe change will pay off. Safe and profitable are separate questions.
• Viscosity’s planner checks rules in order, and the first “yes” decides: unsafe, annotated, opt-in, Python, no numeric field, no loop, too small. Otherwise it chooses soa.
• The size rule rests on this book’s benchmark results: SoA’s lead grows with size. The threshold of 64 is a conservative judgment, not a measured crossover.
• The static size estimate counts literals and pushes in loops with constant bounds. An unknown size is treated as large, and a profile replaces the guess with a measurement.
• The model ignores how hot a loop is, how many fields it uses, and the costs SoA adds. Simple, and explainable, on purpose.