46. Chunks and a disassembler

Eleven bytes

Here is a complete Drip program as the VM sees it:

0, 0, 0, 1, 18, 24, 0, 2, 24, 1, 33

With chapter 45’s opcode table you could decode it by hand. 0 is CONSTANT, so the next 0 is its operand… constant number 0. But which value is that? And if this program crashes, which line of the source should the error point to? The bytes alone can’t say. They need company.

A chunk

Drip’s VM keeps bytecode in a Chunk. This is the whole class, from companion/ts/src/vm/chunk.ts:

export class Chunk {
  code: number[] = [];
  lines: number[] = [];
  constants: Value[] = [];

  write(byte: number, line: number) {
    this.code.push(byte);
    this.lines.push(line);
  }

  addConstant(v: Value): number {
    this.constants.push(v);
    return this.constants.length - 1;
  }
}

Every byte is written together with the source line it came from, so code and lines always have the same length: lines[i] is the line of code[i]. Step through the chunk for those eleven bytes:

print 1.5 + 2;
print "hi";

The constant pool

A byte holds 0 to 255. The number 1.5 is a 64-bit float, eight bytes, and a string can be any length. Instead of squeezing values into the instruction stream, the compiler puts each one in the chunk’s constants array and writes only its index. OP_CONSTANT 2 means “push constants[2]”.

That index is one byte, so a chunk can name at most 256 constants. The compiler checks:

function makeConstant(v: Value): number {
  const c = chunk().addConstant(v);
  if (c > 255) {
    error(
      'Too many constants in one chunk.');
    return 0;
  }
  return c;
}

addConstant never looks for an existing copy, so print 7; print 7; puts two 7s in the pool. Later, global variable names go in the pool, once per mention. That’s how the string_equality benchmark, with only eight variables, ran out of room.

A disassembler

Reading bytes by hand gets old fast. The companion’s disassembler, in companion/ts/src/vm/debug.ts, prints them as text. This is its real output for print 1.5 + 2; and print "hi";:

== script ==
0000    1 OP_CONSTANT         0 '1.5'
0002    | OP_CONSTANT         1 '2'
0004    | OP_ADD
0005    | OP_PRINT
0006    2 OP_CONSTANT         2 'hi'
0008    | OP_PRINT
0009    | OP_NIL
0010    | OP_RETURN

Each line is one instruction. First comes its offset, the index of its opcode in code. Then the source line, or | when it’s the same as the line before. Then the instruction’s name, and for OP_CONSTANT the operand and the value it points to. The offsets jump by 2 after each constant: the operand byte is inside the instruction.

That is the one hard part. Instructions have different lengths, so the disassembler can’t just loop over bytes. It decodes one instruction, which tells it where the next one starts:

// debug.ts, simplified
let offset = 0;
while (offset < chunk.code.length) {
  const [text, next] =
    disassembleInstruction(chunk, offset);
  lines.push(text);
  offset = next;
}

function disassembleInstruction(
  chunk: Chunk, offset: number,
): [string, number] {
  const op = chunk.code[offset];
  const name = OP_NAMES[op];
  switch (op) {
    case OP.CONSTANT: {
      const c = chunk.code[offset + 1];
      const v = chunk.constants[c];
      return [`${name} ${c} '${v}'`,
        offset + 2];
    }
    // …other operand shapes…
    default:
      return [name, offset + 1];
  }
}

The real version pads the columns and handles a few more operand shapes that later chapters add, such as jumps.

The lines array at work

Why carry a line number for every byte? Because the VM only ever knows where in the code it is. When an instruction fails, it looks up the line of the byte it just ran. This program:

print 1;
print 2 +
  -"a";

prints 1, then stops with:

Operand must be a number.
[line 3] in script

The NEGATE that failed was written while the compiler was reading line 3, so lines says 3. One number per byte is simple and wasteful: most neighbours repeat. A more compact encoding is a nice exercise, once everything else works.

Your turn

const NAMES = ['CONSTANT', 'ADD',
  'PRINT'];

function disassemble(chunk) {
  const out = [];
  let off = 0;
  while (off < chunk.code.length) {
    const op = chunk.code[off];
    const at = `000${off}`.slice(-4);
    out.push(`${at} ${NAMES[op]}`);
    off += 1;
  }
  return out;
}

function addConstant(chunk, v) {
  chunk.constants.push(v);
  return chunk.constants.length - 1;
}

What you learned

• A chunk holds three arrays: code (the bytes), lines (one source line per byte) and constants.
• The constant pool keeps values that don’t fit in a byte. Instructions refer to them by a one-byte index, so a chunk can hold 256 at most.
• A disassembler prints each instruction’s offset, line, name and operands, and must decode from the start, because instructions vary in length.
• Runtime errors find their line by looking up the failing byte’s offset in lines.

Next: the loop that actually runs a chunk.