135. Modules, crates and visibility

Nine files, one line each

Here is the top of drip-rs’s src/lib.rs, the root of the drip library crate:

pub mod chunk;
pub mod compiler;
pub mod debug;
pub mod gc;
pub mod object;
pub mod scanner;
pub mod table;
pub mod value;
pub mod vm;

pub use vm::{Options, Outcome, Vm};

In chapter 12 each JavaScript file was a module just by existing, and import { scan } from './scanner.js' found it by path. Rust works the other way round. A file is not compiled until a parent says it should be: mod scanner; tells the compiler “there is a module called scanner; its code is in scanner.rs next to me”. Delete that line and scanner.rs is ignored, however much code is in it.

The module tree

A module can also be written inline, with a body in braces. You have seen one in every Rust exercise: the tests are wrapped in mod tests { use super::*; ... }. Inline or in a file, the result is the same tree, rooted at crate:

crate                 src/lib.rs
├── chunk             src/chunk.rs
├── compiler          src/compiler.rs
├── scanner           src/scanner.rs
│   └── tests         mod tests { } inside
└── ...

Items are named by paths through the tree. crate::scanner::Token starts at the root; super:: means the parent module, which is how a tests module reaches the code above it; self:: is the current one. use brings a path into scope under a short name, like import. drip-rs’s compiler opens with:

use crate::chunk::{Chunk, OpCode};
use crate::gc::{Heap, Trace};
use crate::scanner::{Scanner, Token};

Private by default

In chapter 12, names were private unless exported. Rust’s default is the same, with one more level: a struct can be public while its fields stay private. Here is drip-rs’s hash table (table.rs, trimmed):

pub struct Table {
    count: usize,
    keys: Vec<Option<StrRef>>,
    values: Vec<Value>,
}

impl Table {
    pub fn new() -> Self { ... }
    pub fn get(&self, key: StrRef)
        -> Option<Value> { ... }
    // no pub: private
    fn grow(&mut self, capacity: usize) {
        ...
    }
}

Other modules may name Table and call new and get. They can’t touch count, keys or values, and they can’t call grow. Try both on a small copy of this table, from outside its module, and Rust 1.97 says (trimmed):

error[E0624]: method `grow` is private
 9 |  fn grow(&mut self, cap: usize) {}
   |  ------------------------------
   |  private method defined here
...
15 |     t.grow(16);

error[E0616]: field `count` of struct
  `Table` is private
16 | println!("{}", t.count);
   |                  ^^^^^ private field

A private free function or module named by a path, such as table::helper(), gives a third code, E0603 (“function helper is private”).

Why bother? count counts live entries and tombstones, and the two columns must always have the same length. If any module could write t.count = 0 or push onto keys alone, the table could be broken from anywhere. With private fields, only table.rs can break it, so only table.rs has to be read to trust it. Because outsiders can’t fill in private fields, they also can’t build a Table { ... } literal: they must go through new.

The exact rule, from the Rust Reference: a private item may be used by the module it is in and that module’s descendants; a public one by anyone who can reach all its ancestor modules^[2]. That second half is why a tests child module can test private functions, and why pub on an item inside a private module doesn’t make it visible outside the crate. Two exceptions: the variants of a pub enum are all public, and so are the items of a pub trait. For in-between cases there is pub(crate), visible anywhere in this crate but not outside; neither drip-rs nor engine/rust uses it.

The public face

The last line of lib.rs, pub use vm::{Options, Outcome, Vm};, is a re-export. The VM really lives at drip::vm::Vm, but users can write drip::Vm. drip-rs’s main.rs is a separate crate, the drip-rs binary (chapter 121), and it sees only what the library made pub:

use drip::compiler::compile;
use drip::gc::Heap;
use drip::{Options, Vm};

Like a package’s exports in npm, the set of pub paths is a crate’s public API, and changing it is what breaks other people’s code.

Workspaces

drip-rs is one package holding two crates. The Rust Viscosity is bigger: six packages, each with its own Cargo.toml, built together as one workspace. Its root Cargo.toml has no code, only this (trimmed):

[workspace]
members = ["crates/vir", "crates/core",
    "crates/frontend-js",
    "crates/frontend-python",
    "crates/cli", "crates/wasm"]

[workspace.dependencies]
viscosity-vir = { path = "crates/vir",
    version = "0.1.0" }
oxc_parser = "0.156"

(The real file keeps each entry on one line.) A member then writes viscosity-vir.workspace = true instead of a path or a version, so every crate agrees on one version of oxc. The diagram leaves out frontend-python, chapter 187’s Python plugin, which sits beside frontend-js; the five here are the ones Part IX uses. A package named viscosity-core is viscosity_core in Rust code, since - can’t appear in a path. Step through what depends on what:

Your turn

mod soa {
    // Invariant: xs and vxs always
    // have the same length.
    struct Particles {
        xs: Vec<f64>,
        vxs: Vec<f64>,
    }

    impl Particles {
        fn new() -> Self {
            Particles {
                xs: Vec::new(),
                vxs: Vec::new(),
            }
        }

        fn push(&mut self, x: f64,
            vx: f64) {
            self.xs.push(x);
            self.vxs.push(vx);
        }

        fn len(&self) -> usize {
            self.xs.len()
        }

        fn x(&self, i: usize)
            -> f64 {
            self.xs[i]
        }

        fn step(&mut self,
            dt: f64) {
            let vs = &self.vxs;
            let xs = self.xs.iter_mut();
            for (x, v) in xs.zip(vs) {
                *x += v;
            }
        }
    }
}

What you learned

• A crate is a module tree rooted at crate. mod name; pulls in name.rs; an unnamed file isn’t compiled.
• Paths name items: crate::, super::, self::. use shortens them.
• Everything is private by default, fields included. Private means “this module and its descendants”; pub opens it up, pub use re-exports.
• Private fields let a type guard its invariants, like drip-rs’s Table.
• A workspace builds several crates together, with one lock file and one target/. engine/rust’s six crates make the plugin boundary a compile-time fact.