21. SIMD: one instruction, many numbers

The same sum, a million times

Here is the loop at the heart of every particle game:

for (let i = 0; i < n; i++) {
  x[i] = x[i] + vx[i];
}

Every iteration does exactly the same thing: load two numbers, add them, store one. Only the index changes. The CPU decodes the same instructions over and over, a million times, to do a million tiny additions that don’t depend on each other at all.

Surely a machine could just do several at once?

One instruction, four answers

That glued stamp exists in real CPUs. Besides the ordinary registers that hold one number, many CPUs have wide registers that hold several numbers at once, and instructions that work on all of them together. Step through an addition done both ways:

How many lanes you get depends on the register’s width and the size of each number. A 128-bit register has room for four 32-bit floats, or two 64-bit doubles. JavaScript numbers are 64-bit doubles, so at that width they get two lanes. WebAssembly’s SIMD names spell this out: f32x4 means “four 32-bit float lanes”, f64x2 means “two 64-bit lanes”^[4]. Some CPUs have wider registers still: the ISPC compiler’s guide describes 256-bit and 512-bit registers on x86 CPUs with AVX-512^[3].

You can feel the idea in plain JavaScript by writing the loop the way SIMD hardware would see it: four lanes per step.

const a = [1, 2, 3, 4, 5, 6];
const b = [10, 20, 30, 40, 50, 60];
const c = [];
const W = 4; // lanes per step
let i = 0;
// full steps of W lanes
for (; i + W <= a.length; i += W) {
  for (let l = 0; l < W; l++) {
    c[i + l] = a[i + l] + b[i + l];
  }
}
// the tail that doesn't fill W lanes
for (; i < a.length; i++) {
  c[i] = a[i] + b[i];
}
console.log(c);

That code isn’t faster (it’s still one add at a time underneath). It only shows the shape of the work: chunks of four, plus a tail when the length isn’t a multiple of four.

Vectorization: letting the compiler do it

You rarely write SIMD by hand. Instead, an optimizing compiler looks at a loop like ours and rewrites it to process several iterations per step. LLVM, the compiler framework behind the clang C compiler, has a loop vectorizer that does this, and it is enabled by default^[1].

The compiler can only do it when the rewrite can’t change the answer. Two things commonly get in the way^[1]:

1. Overlap. If x and vx might be the same memory, writing four x values before reading the next vx values could change what is read. LLVM handles this by checking for overlap while the program runs and falling back to the plain loop.
2. Order. Floating-point addition isn’t associative: grouping the additions differently can round differently. Summing in four lanes and adding the lanes at the end changes the grouping, so for floats LLVM only does it when you explicitly allow reordering.

console.log((0.1 + 0.2) + 0.3);
// 0.6000000000000001
console.log(0.1 + (0.2 + 0.3));
// 0.6

Same three numbers, different grouping, different answer. A compiler that silently changed one into the other would be breaking your program.

Why SIMD wants columns

A vector load is fast because it copies one contiguous run of memory straight into a register: four neighbours in, four lanes full. So ask: in our particle data, are the four x values neighbours?

In an array of structs, the x values are spread out with y, vx, vy, mass and health in between. The hardware has to gather them one at a time. The ISPC performance guide says exactly this: contiguous accesses can use efficient vector loads and stores, while the AoS layout needs gathers and runs slower^[2]. LLVM’s own documentation notes that its cost model often judges strided gathers not worth vectorizing at all^[1].

In a struct of arrays, every column is already lined up for the stamp. That’s the deeper reason SoA keeps appearing in high-performance code: it is the layout that both cache lines and SIMD lanes want.

Your turn

The core exercise is the shape of every vectorized sum: one accumulator per lane, then a final step that adds the lanes together.

function sumLanes(xs, W) {
  const lanes = [];
  // your code here
}

const r = sumLanes([1, 2, 3, 4, 5], 2);
console.log(r);

function column(packed, nf, f) {
  // your code here
}

const p = [1, 2, 3, 4, 5, 6];
console.log(column(p, 3, 1));

What you learned

• SIMD means one instruction works on several values at once, held in a wide vector register.
• Each slot of that register is a lane. Lanes work side by side and never mix; a 128-bit register holds two 64-bit doubles.
• Vectorization is a compiler rewriting a loop into SIMD steps. It must not change the answer, so overlapping memory and floating-point reordering can block it.
• SIMD loads want contiguous values. AoS spreads a field out with a stride, so it needs gathers; SoA columns load straight into lanes.
• JavaScript has no SIMD API (SIMD.js was withdrawn). WebAssembly has fixed-width 128-bit SIMD.

Next: what JavaScript numbers and objects actually look like in memory, and why one innocent-looking TypeScript line can make a loop five times slower.