190. The npm package

Two ways to ship Rust

Chapter 119 packed the TypeScript CLI for npm: one JavaScript bundle, dist/viscosity.mjs, that runs on any computer with Node 18. Easy, because JavaScript is the same everywhere. The Rust engine is faster (chapter 192), but a compiled Rust program is not the same everywhere. cargo build --release gave me a 2,785,400-byte ELF executable for x86-64 Linux, linked against glibc. A Mac can’t run it, and neither can an ARM laptop.

Tools written in Rust, Go or C++ solve this in one of two ways, and you have used both without noticing. Look in this book’s own node_modules. The bundler esbuild 0.27.7 lists 26 optionalDependencies, @esbuild/darwin-arm64 to @esbuild/win32-x64, and each of those packages declares the os and cpu it runs on. npm compares them with the machine and skips the rest. Here only @esbuild/linux-x64 was installed: one 11,120,788-byte binary^[1].

use std::env::consts::{ARCH, OS};

fn main() {
    // esbuild 0.27.7's platform
    // packages, named "<os>-<cpu>".
    let all = [
        "aix-ppc64", "android-arm",
        "android-arm64", "android-x64",
        "darwin-arm64", "darwin-x64",
        "freebsd-arm64", "freebsd-x64",
        "linux-arm", "linux-arm64",
        "linux-ia32", "linux-loong64",
        "linux-mips64el", "linux-ppc64",
        "linux-riscv64", "linux-s390x",
        "linux-x64", "netbsd-arm64",
        "netbsd-x64", "openbsd-arm64",
        "openbsd-x64",
        "openharmony-arm64",
        "sunos-x64", "win32-arm64",
        "win32-ia32", "win32-x64",
    ];
    // Rust and Node name things apart.
    let os = match OS {
        "macos" => "darwin",
        "windows" => "win32",
        o => o,
    };
    let cpu = match ARCH {
        "x86_64" => "x64",
        "aarch64" => "arm64",
        "x86" => "ia32",
        a => a,
    };
    let want = format!("{os}-{cpu}");
    println!("this machine: {want}");
    let mine: Vec<_> = all
        .iter()
        .filter(|p| **p == want)
        .collect();
    println!(
        "npm installs {:?} of {}",
        mine,
        all.len()
    );
}

The other way is to compile once to WebAssembly and let the JavaScript engine run it. wasm-bindgen’s guide offers its Node target “perhaps as an alternative to a native module”^[6].

One package, two kinds of glue

@viscosity/wasm takes the second road. Chapter 189’s build left two folders, dist/node (CommonJS glue that loads synchronously) and dist/web (an ES module with init()). The package.json decides who gets which, with an exports map. Trimmed:

"exports": {
  ".": {
    "node": {
      "types": "./dist/node/….d.ts",
      "default": "./dist/node/….js"
    },
    "browser": {
      "types": "./dist/web/….d.ts",
      "default": "./dist/web/….js"
    },
    "types": "./dist/web/….d.ts",
    "default": "./dist/web/….js"
  },
  "./web": …, "./node": …,
  "./viscosity_wasm_bg.wasm": …
}

Each key is a condition. Node sets "node" and either "import" or "require"; a bundler building for the web uses "browser". Key order is significant: the first key whose condition is active wins, "default" always matches and so comes last, and a nested object that matches nothing falls back to its parent’s next key^[2]. TypeScript reads the same map and always matches "types"^[3]. Step through five importers:

What is in the box

Packed at commit 288e497, the package is 13 files, 997 KiB, and 2,840 KiB once unpacked. Most of it is two copies of the same module: dist/node/viscosity_wasm_bg.wasm (1,373,454 bytes) and dist/web/viscosity_wasm_bg.wasm (1,373,390). Each glue file loads the module sitting next to it, so sharing one would mean editing generated code. The build doesn’t, and the price is 1.4 MB of disk.

There is no bin. @viscosity/cli installs a viscosity command; @viscosity/wasm is a library, for programs that want to call compile themselves: an editor plugin, a bundler plugin, this book.

The third big file is THIRD_PARTY_LICENSES.txt, 130,597 bytes. The TypeScript CLI has one too, because its bundle contains acorn and sucrase. The .wasm is the same situation, larger. It statically links 71 third-party Rust crates: oxc’s parser, indexmap, rustc-hash, the ICU4X Unicode crates, wasm-bindgen itself. Their code is inside the file you hand out, so their licence notices must go with it. build.mjs asks cargo tree which crates are linked for wasm32-unknown-unknown (proc macros, which run only at build time, are left out), finds each crate’s licence files through cargo metadata, and writes every distinct text once. One snag: oxc’s crates are published without their licence file. The package carries copies of oxc’s own files in licenses/, and the script uses them as a fallback for any oxc crate.

Installed like a stranger would

Chapter 119 found three bugs by installing the TypeScript CLI in an empty folder. smoke.mjs does that for this package, every time: npm pack, install the tarball into a fresh directory, then compile every corpus program through the installed package, 40 JavaScript and 20 Python, in three ways, and compare code and explain with what the TypeScript CLI prints. Finally it type-checks a small consumer under both the Node and the browser conditions. My run, at commit 288e497 (repeated with the same results after the version became 0.0.1), trimmed:

packed viscosity-wasm-0.0.1.tgz:
  13 files, 997 KiB (2840 KiB unpacked)
✓ nodejs target, require():
  {"programs":60,"mismatches":0}
✓ nodejs target, import:
  {"programs":60,"mismatches":0}
✓ web target, init(bytes):
  {"programs":60,"mismatches":0}
✓ types (node, nodenext)
✓ types (browser, bundler)

The web variant runs in Node too, by passing the .wasm bytes to init() instead of a URL. The package isn’t published: like the CLI, everything up to npm publish has been walked.

Your turn

// A value in an "exports" map: a file,
// or conditions tried in this order.
enum Target {
    File(&'static str),
    Conds(Vec<(&'static str, Target)>),
}
use Target::*;

// The file `t` gives an importer whose
// active conditions are `conds`.
fn resolve(
    t: &Target,
    conds: &[&str],
) -> Option<&'static str> {
    todo!()
}

What you learned

• Native builds are fastest but per platform. esbuild ships 26 packages, and npm installs the one whose os and cpu match.
• A .wasm is one file for every platform, run by the JS engine, without threads or files on wasm32-unknown-unknown.
• An exports map is tried in key order: first active condition wins, "default" last, nested misses fall through.
• A statically linked .wasm must carry its crates’ licences: 71 of them here.
• smoke.mjs installs the tarball and checks 60 programs three ways against the TS CLI.