195. Compiling in parallel

Four cores, one file at a time

Compiling one test program takes the Rust engine about 0.1 ms. That sounds like the end of the speed story, until you hand it a whole project. viscosity build a.js b.js ... -o out/ takes many files, and they don’t depend on each other: the analyses never look across files. So the CLI compiles them at the same time, one thread per CPU by default, or --jobs N.

engine/rust/results/parallel-build-2026-10-31.md timed 396 files (the 66 JavaScript and Python test programs, copied six times) on a shared 4-vCPU cloud VM, best of 11 runs. Writing the output to tmpfs, one thread took 40.0 ms and four took 13.5 ms: 2.95×. Writing to the VM’s disk, 105.3 ms and 85.1 ms: 1.24×. Same compiler, same threads. Hold that thought.

Threads that may borrow

A thread is a second line of execution in the same program, sharing its memory; the operating system runs threads on different cores at once. std::thread::scope creates threads that may borrow the caller’s local variables, because the scope joins every thread before it returns^[1]. The borrow rules of chapter 125 still hold across threads: many threads may share &a, but nothing can be changed through a shared reference unless the type is built for it. Two such types do the work here.

The CLI’s helper, from crates/cli/src/main.rs (adapted):

fn for_each_parallel<T: Send>(
    n: usize,
    jobs: usize,
    f: impl Fn(usize) -> T + Sync,
) -> Vec<T> {
    let jobs = jobs.clamp(1, n.max(1));
    if jobs == 1 {
        return (0..n).map(f).collect();
    }
    let next = AtomicUsize::new(0);
    let slots: Mutex<Vec<Option<T>>> =
        Mutex::new(empty(n));
    thread::scope(|s| {
        for _ in 0..jobs {
            s.spawn(|| loop {
                let k = next.fetch_add(
                    1, Ordering::Relaxed);
                if k >= n {
                    break;
                }
                let r = f(k);
                slots.lock().expect("...")
                    [k] = Some(r);
            });
        }
    });
    let v = slots.into_inner()
        .expect("...");
    v.into_iter()
        .map(|r| r.expect("..."))
        .collect()
}

empty(n) stands for n Nones. T: Send says a result may be moved to another thread, and f: Sync that f may be called from several at once.

One counter, numbered slots

fetch_add(1) adds one and returns the previous value, as one indivisible step^[2], so two threads can never draw the same index. Relaxed is the weakest ordering, and enough here: the counter only hands out numbers. The results travel through the Mutex, which lets one thread at a time reach the vector ^[3], and the main thread reads them only after the scope has joined every thread.

Each result goes into slot k, so into_inner gives them back in input order whatever order they finished in. That makes the output deterministic: multi_file.rs checks that explain a.js b.js ... prints the same bytes with --jobs 1, 3 and 16, and the measurement checked --jobs 1 against --jobs 4 with diff -r.

When one file goes wrong

A file that fails to compile is just an Err in its slot. For a bad.js holding const ps = [;, the main loop prints viscosity: bad.js: Unexpected token (1:12), prints the others, and returns Ok(false), which main turns into exit status 1.

A panic is a bug in the compiler, and with 396 files it should cost one file, not the run. compile_file wraps the compiler in std::panic::catch_unwind, which turns an unwinding panic into an Err^[4]. The file is reported as internal error: the compiler panicked (see above); “above” is the panic message, which Rust’s panic hook has already printed to stderr. Because the panic is caught inside f, no thread ever panics while it holds the lock, so the expect on lock() can’t fire. Here is the whole pattern, with a stand-in compiler:

use std::panic::catch_unwind;
use std::sync::atomic::{
    AtomicUsize, Ordering,
};
use std::sync::Mutex;
use std::thread;

type Res = Result<usize, String>;

// Stands in for the real compiler
fn compile(file: &str) -> Res {
    match file {
        "bad.js" => {
            Err("SyntaxError".into())
        }
        "bug.js" => panic!("a bug"),
        _ => Ok(file.len() * 10),
    }
}

// A panic costs only its own file
fn guarded(file: &str) -> Res {
    catch_unwind(|| compile(file))
        .unwrap_or_else(|_| {
            Err("panicked".into())
        })
}

fn main() {
    let files = [
        "a.js", "bad.js", "bug.js",
        "b.js", "longer.js",
    ];
    let n = files.len();
    let next = AtomicUsize::new(0);
    let empty = (0..n).map(|_| None);
    let slots: Mutex<Vec<Option<Res>>> =
        Mutex::new(empty.collect());
    thread::scope(|s| {
        for _ in 0..2 {
            s.spawn(|| loop {
                let k = next.fetch_add(
                    1,
                    Ordering::Relaxed,
                );
                if k >= n {
                    break;
                }
                let r = guarded(
                    files[k]);
                let mut v = slots.lock()
                    .unwrap();
                v[k] = Some(r);
            });
        }
    });
    let v = slots.into_inner().unwrap();
    let mut code = 0;
    for (f, r) in files.iter().zip(v) {
        match r.unwrap() {
            Ok(n) => {
                println!("{f}: {n}")
            }
            Err(e) => {
                println!("{f}: {e}");
                code = 1;
            }
        }
    }
    println!("exit status: {code}");
}

Reading the numbers

From the results file, 396 files, best of 11 (median in parentheses), in milliseconds (wrapped):

                1 thread     4 threads
build, tmpfs    40.0 (45.2)  13.5 (14.9)
build, disk    105.3 (109.6) 85.1 (98.8)
explain         34.6 (36.5)  18.1 (20.9)

Three readings, all from the file:

• tmpfs: 2.95×. Not 4×, because the machine was busy (load average 2 to 3.5 on 4 CPUs), and because some of the run is serial: starting the process, reading the arguments, creating the directory, printing in order.
• disk: 1.24×. Creating 396 files costs about 65 ms whatever the thread count: the virtual disk serializes file creation. Threads shrink the rest, which is about the tmpfs time, but not those 65 ms.
• explain: 1.9×, and 2.15× at 3 threads. All of its output is printed by the main thread after the workers finish.

With 1,584 files on tmpfs, build peaked at 2.5× with 3 threads. For a handful of files none of this matters: the results file finds the default as fast as --jobs 1 there. The TypeScript CLI takes the same arguments and compiles one file at a time.

Your turn

use std::sync::atomic::{
    AtomicUsize, Ordering,
};
use std::sync::Mutex;
use std::thread;

fn par_map<T: Send>(
    n: usize,
    jobs: usize,
    f: impl Fn(usize) -> T + Sync,
) -> Vec<T> {
    todo!()
}

What you learned

• Separate files are data-parallel: same work, no shared state, one thread per core.
• thread::scope lets threads borrow; fetch_add hands out each index once; a Mutex guards the slots.
• Slots in input order make the output deterministic for any --jobs. A failing file is an Err; a panic costs one file.
• 2.95× to tmpfs, but 1.24× to disk: the serial part, there file creation, caps the speed-up.