174. Emitting JavaScript from Rust

Change only what you must

Chapter 94 had a choice: print the whole program from the IR, or edit the original text. Viscosity edits. Here is one of the corpus programs before and after (the container class above it is left out):

const ps = [];
for (let i = 0; i < 100; i++)
  ps.push({ x: i, y: (i, i + 1) });

const ps = new __Vsc_ps();
for (let i = 0; i < 100; i++)
  ps.push(i, (i, i + 1));

(The loop header is wrapped to fit here; the file has it on one line.) Everything not rewritten is copied byte for byte: comments, spacing, odd formatting. That makes the output easy to read and review, and it is what lets a test demand byte-identical output from two engines (chapter 175).

A list of edits

emit.rs walks the plan, as emit.ts does: first the collection’s creation, then each push, then element reads, then field accesses. Each step records edits, and nothing is changed yet:

// Replace start..end with text
// (an insertion when they are equal).
struct Edit {
    start: u32,
    end: u32,
    text: String,
    rank: u32, // orders insertions
}

Then it applies them all (adapted from the end of emit_js):

edits.sort_by_key(|e| {
    let back = Reverse(e.start);
    (back, Reverse(e.end), e.rank)
});
let mut out = source.to_string();
let mut last_start = u32::MAX;
for e in &edits {
    if e.end > last_start {
        return Err(overlap(e));
    }
    let r =
        e.start as usize..e.end as usize;
    out.replace_range(r, &e.text);
    last_start = e.start;
}

The diagram simplifies one thing. The real emitter doesn’t replace { x: 1 } in one piece: it deletes the text around each value, so a value that itself needs rewriting (say ps[j].x) gets its own edit. Edits then meet at the same byte, and some are pure insertions. That is what the rest of the sort key is for: at one position, the longer replacement goes first, and insertions go by rank.

The TS version sorts on the same three keys and rebuilds the string for each edit: out = out.slice(0, e.start) + e.text + out.slice(e.end). Rust’s String::replace_range edits one buffer in place, moving only the tail. Both sorts are stable, so edits with equal keys keep the order they were added in. Rust documents that for sort_by_key^[3], and ECMAScript has required it of Array.prototype.sort since 2019^[4]. Both engines must apply ties in the same order.

One more detail: p.x += 1 both reads and writes p.x, which would mean two edits for one range: an overlap. So the emitter keys field edits by their range and keeps one, chosen so that an out-of-bounds access throws the same error the original would.

Bytes, not code units

The offsets come from the parser. oxc’s spans count UTF-8 bytes. acorn’s count UTF-16 code units, which is what JavaScript strings index by. On rustc 1.97:

let src = "// é😀\nps[i].x";
let b = src.find("ps").unwrap();
let u = src[..b].encode_utf16().count();
println!("bytes {b}, utf-16 {u}");
// bytes 10, utf-16 7

In Node, src.indexOf("ps") is 7. The numbers differ, yet both point at the same p, because each engine slices its own kind of string with its own unit. So the port never converts. tests/extra/unicode_offsets.js puts accented letters, an emoji and U+2028 ahead of the rewritten code, and its output matches the TS engine’s.

A wrong offset can’t quietly corrupt the text, either. replace_range panics if a range doesn’t fall on char boundaries^[2]. Replacing 4..5 in that string (the middle of é) stops with start of range should be a character boundary.

Writing text with fmt::Write

The edits’ texts, and the container classes that go above the code, are built with format! and join, line for line like the TS template literals and join('\n'). format! is itself a writer: in the standard library it makes a String with a capacity estimate and calls write_fmt on it^[1].

You can call that directly and append to one growing String. The trait is std::fmt::Write, and the macros are write! and writeln!:

use std::fmt::Write;

let mut s = String::new();
writeln!(s, "class {cls} {{").unwrap();

Leave out the use and rustc 1.97 says (trimmed):

error[E0599]: cannot write into `String`
help: trait `Write` which provides
  `write_fmt` is implemented but not in
  scope; perhaps you want to import it
  |
1 + use std::fmt::Write;

write! returns a fmt::Result, because some writers can fail. String can’t: its implementation calls push_str and returns Ok(()), so .unwrap() is safe here^[1].

use std::fmt::Write;

fn class(cls: &str, cols: &[&str])
    -> String {
    let t = "Float64Array";
    let mut s = String::new();
    writeln!(s, "class {cls} {{")
        .unwrap();
    s += "  constructor(cap) {\n";
    for c in cols {
        s.push_str("    ");
        writeln!(
            s,
            "this.{c} = new {t}(cap);"
        )
        .unwrap();
    }
    s.push_str("  }\n}");
    s
}

fn main() {
    let cols = ["x", "vx"];
    let c = class("__Vsc_ps", &cols);
    println!("{c}");
    println!("{} bytes", c.len());
}

Your turn

use std::cmp::Reverse;

// Replace bytes start..end with text.
#[derive(Clone, Debug)]
struct Edit {
    start: usize,
    end: usize,
    text: String,
}

fn apply(src: &str, edits: &[Edit])
    -> Result<String, String> {
    todo!()
}

What you learned

• Viscosity emits by range edits on the original source. Everything outside them is copied byte for byte.
• Edits are sorted by start, last first, and applied with replace_range, so earlier offsets stay valid. An edit that ends past the previous start overlaps, and is an internal error.
• The sort is stable in both engines, so ties apply in the same order.
• oxc’s UTF-8 byte offsets in a Rust String and acorn’s UTF-16 offsets in a JS string name the same characters. A bad offset panics instead of corrupting text.
• fmt::Write (write!, writeln!) appends formatted text to a String. format! is the same machinery. Remember use std::fmt::Write;.