108. Hot/cold splitting

The fields nobody reads

A game keeps its entities in an array of objects:

const es = [];
es.push({
  x: 0, y: 0, vx: 1, vy: 0,
  health: 100, name: 'goblin',
});

Every frame a loop moves every entity, using x, y, vx and vy. The name is printed when an entity dies. Yet V8 keeps an object’s fields together, right after its header^[2], and a cache works in whole 64-byte lines^[1]. So the movement loop drags health and a pointer to name through the cache for every entity, and never looks at them.

Splitting hot from cold

Chapter 23 named the fix: a hot/cold split. Put the busy fields in one place and the rest in another, both indexed by the same i:

const hot = {
  x: [0], y: [0], vx: [1], vy: [0],
};
const cold = [
  { health: 100, name: 'goblin' },
];
// the movement loop:
for (let i = 0; i < n; i++)
  hot.x[i] += hot.vx[i];

The movement loop now streams only hot data. The price is on the cold side: code that wants a whole entity has to look in two places.

Which fields go together?

“Hot” is a matter of counting. Count, for each field, how many times the program touches it. Then count, for each pair of fields, how often a loop uses both. Weight each loop by how many times it runs:

The run counts matter. Counted by loop alone, report would look as important as move. Weighted by runs, name scores 1 and x scores 1,100.

What leaving fields out is worth

The sum-one-field suite reads one of six particle fields:

Packed AoS already drops the headers and pointers, but its cache lines still carry five unused fields. SoA carries none. That is the hot/cold argument taken to its limit.

What Viscosity does

Viscosity doesn’t build a hot struct and a cold struct. It gives every field its own column, which is the finest split there is: each field is its own group, and each loop loads exactly the columns it touches. A move loop over Viscosity’s columns never reads name at all, whatever its affinity.

What Viscosity does decide per field is the representation. The corpus program mixed_columns.js shows it:

for (let i = 0; i < 300; i++)
  es.push({
    id: 'e' + i, hp: 100, x: i,
    tag: i % 3 ? null : { boss: true },
  });
for (let r = 0; r < 3; r++) {
  for (let i = 0; i < es.length; i++) {
    es[i].x += 1;
    if (es[i].tag !== null) es[i].hp -= 10;
  }
}
es[4].id = 42;

(Long lines are wrapped here; the declaration of es and the printing at the end are left out.) viscosity explain says:

es (line 2) {id, hp, x, tag} → struct of
  arrays
  · loop at line 5 in <module>() touches
    3 of 4 fields (hp, tag, x)
  · loop at line 12 in <module>() touches
    2 of 4 fields (hp, x)
  · typed Float64Array columns for hp, x;
    plain array columns for id, tag (may
    hold non-numbers)

And the generated constructor, reformatted:

constructor(cap = 16) {
  this.length = 0;
  this.cap = cap;
  this.hp = new Float64Array(cap);
  this.x = new Float64Array(cap);
  this.id = [];
  this.tag = [];
}

The type analysis proved hp and x are always numbers, so they get Float64Array columns of raw doubles^[3]. id holds strings and, after es[4].id = 42, a number; tag holds null or an object. Those get plain array columns, which store any value.

Notice that this split is by type, not by heat. tag is read in the hottest loop, yet it is a plain column, because a Float64Array can’t hold an object. And a numeric field that is almost never read still gets a typed column. The per-loop lines in the explanation (“touches 3 of 4 fields (hp, tag, x)”) are exactly the raw data an affinity analysis needs, but Viscosity doesn’t group fields with them today. Try the policy: make id a number, and it gets a typed column too.

Your turn

function coAccess(loops) {
  const aff = {};
  // your code here
  return aff;
}

const loops = [
  { runs: 1000,
    fields: ['x', 'y', 'vx', 'vy'] },
  { runs: 100,
    fields: ['x', 'health'] },
  { runs: 1,
    fields: ['health', 'name'] },
];
console.log(coAccess(loops));

function groupFields(fields, aff, t) {
  // your code here
  return [fields];
}

What you learned

• A hot/cold split moves rarely used fields out of the hot loop’s way. Code that wants a whole record pays by looking in two places.
• Field affinity measures which fields are used together: for each pair, add the runs of every loop using both. Hotness is the same count per field.
• SoA is the finest split: each field is its own column, and a loop loads only what it touches.
• Viscosity’s real per-field decision is typed or plain: Float64Array for fields proven numeric, plain arrays for the rest. It doesn’t group fields by affinity.