67. Escape analysis in V8

The puzzle from chapter 23

Two of this book’s benchmark suites rebuild an object per element from struct-of-arrays columns. materialize keeps each one in an array:

for (let i = 0; i < n; i++)
  out[i] = { x: x[i], y: y[i], /* … */ };

get-objects hands each one to a small function and drops it:

for (let i = 0; i < n; i++)
  t += consume({ x: x[i], /* … */ });

Same allocation, same fields. Here is what this book’s benchmark run measured:

The repo’s findings put it in one sentence: V8 inlined the consumer and removed the temporary object. This chapter takes that apart.

Step one: see every use

In the loop as written, the object is passed to a call. To the optimizing compiler, a call is a door: whatever goes through might be stored anywhere. So the first move is to copy the small function’s body into the loop, which compilers call inlining (chapter 82 builds it). Then every use of the object is in view.

You can watch V8 do it. A loop like get-objects, with two fields:

function len2(p) {
  return p.x * p.x + p.y * p.y;
}
function run() {
  let t = 0;
  for (let i = 0; i < n; i++)
    t += len2({ x: xs[i], y: ys[i] });
  return t;
}

Real output, Node v22.23.4 (V8 12.4), addresses shortened:

$ node --trace-turbo-inlining temp.js
…
Inlining 0x… {0x… <SharedFunctionInfo len2>}
  into 0x… {0x… <SharedFunctionInfo run>}

Step two: the object that isn’t

Now TurboFan’s escape analysis looks at each allocation. Its source describes how. Each allocation becomes a virtual object that tracks a variable for each field. A store to a field of a virtual object just sets that variable, and a load is replaced by the value last stored. Anything the analysis doesn’t understand, it treats as an escape^[1]. If, at the end, nothing needed the object to really exist, the allocation is removed. Step through it:

function len2(p) {
  return p.x * p.x + p.y * p.y;
}
for (let i = 0; i < n; i++)
  t += len2({ x: xs[i], y: ys[i] });

The proof is in the garbage

If the objects really are gone, the garbage collector should notice. New objects are allocated in the young generation^[4], and when it fills up V8 collects it with a scavenge (next chapter). So run the len2 loop 50 times over a million elements and count scavenges, with the optimization on, and with each step switched off. Real output, Node v22.23.4:

$ node --trace-gc temp.js \
    | grep -c Scavenge
3
$ node --trace-gc --no-turbo-escape \
    temp.js | grep -c Scavenge
1710
$ node --trace-gc --no-turbo-inlining \
    temp.js | grep -c Scavenge
4257

The count wobbles a little from run to run, but the story doesn’t. With both steps, the hot loop allocates nothing. Turn off escape analysis and every temporary is real. Turn off inlining and even more is allocated, for reasons this chapter doesn’t chase.

When a guard fails

The optimized loop still has guards (chapter 66). If one fails halfway through, the interpreter takes over, and its bytecode still says len2(p). It needs a real p.

So the object isn’t forgotten, only postponed. When TurboFan writes the deopt information, it describes each virtual object that is still live by its fields, as an ObjectState^[1]. The deoptimizer can then build the object for real. V8’s concurrent marking post lists “materialization during deoptimization of a function” among the operations that create objects^[3]. That is the sunk allocation: it happens only on the path that needs it.

materialize has no such way out. out[i] = p stores the object where code outside the loop can find it and compare identities. It escapes, so every allocation stays.

Your turn

function scalarReplace(code) {
  // 1. find objects that never escape
  // 2. rewrite new / get / set on them
  return code;
}

function materialize(desc, env) {
  return {};
}

What you learned

• get-objects didn’t lose for SoA because V8 inlined the consumer and then removed the temporary object. materialize lost because its objects escape into an array.
• TurboFan treats each allocation as a virtual object with a variable per field. If it doesn’t escape, its loads become the stored values: scalar replacement.
• The allocation is sunk to the deopt path: the frame state describes the object by its fields, and the deoptimizer builds it only if a guard fails.
• On this machine, turning off escape analysis took the len2 loop from a handful of scavenges to about 1700.
• Viscosity has no deoptimizer, so it must prove an object doesn’t escape before it changes the layout.