136. Testing in Rust

You’ve been graded all along

Chapter 15’s tests looked like this:

test('max picks the largest', () => {
  expect(max([3, 7, 2])).toBe(7);
});

Every Rust exercise since chapter 121 has been graded by tests you could read but didn’t write, and todo!() made the starters fail on purpose. Here is a real one, from drip-rs’s scanner (scanner.rs, trimmed):

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn reports_errors_as_tokens() {
        let src = "\"open\n";
        let mut s = Scanner::new(src);
        let t = s.scan_token();
        assert_eq!(t.kind,
            TokenKind::Error);
        assert_eq!(t.line, 2);
    }
}

A test is a function with #[test] above it: no arguments, no return value. It passes if it returns and fails if it panics. That’s the whole contract, which is why todo!(), a panic, fails a test.

Where unit tests live

The tests module sits in the same file as the code it tests. Two details from the last two chapters make that work:

• #[cfg(test)] means “compile this only when building tests”, so normal builds don’t carry test code.
• tests is a child module, so use super::* brings in the parent’s items, and the privacy rule lets a child see private items. Unit tests can test is_alpha, which nothing outside scanner.rs can call.

cargo test builds the tests and runs them, in parallel threads by default^[1]. Each test is named by its module path. On drip-rs’s library (trimmed, long names wrapped):

running 12 tests
test chunk::tests::
  opcodes_round_trip_through_bytes ... ok
test scanner::tests::
  reports_errors_as_tokens ... ok
test object::tests::
  using_a_freed_handle_panics
  - should panic ... ok
...
test result: ok. 12 passed; 0 failed; ...

Any argument after cargo test filters by name: cargo test scanner runs only the tests whose path contains scanner.

The assert macros

Three macros do the checking:

• assert!(cond) panics if cond is false.
• assert_eq!(a, b) panics if a != b, and prints both.
• assert_ne!(a, b) panics if a == b.

drip-rs’s interning test uses two of them: the same text must give the same handle, and different text a different one:

let a = heap.intern("hello");
let b = heap.intern_owned(
    String::from("hel") + "lo");
assert_eq!(a, b);
assert_ne!(a, heap.intern("world"));

When assert_eq! fails, it shows both sides. Here is a test of chapter 23’s AoS slot formula, with the formula written wrong (rustc 1.97, thread id and backtrace removed):

---- tests::aos_slot stdout ----
thread 'tests::aos_slot' panicked at
  t.rs:11:9:
assertion `left == right` failed
  left: 5
 right: 7

To print the values, assert_eq! needs both sides to implement Debug; to compare them, PartialEq. That is why types get #[derive(Debug, PartialEq)] before they are tested^[1]. Every assert macro also takes a message, formatted like println!: assert!(n % 2 == 0, "n = {n} should be even") fails with n = 3 should be even.

Some code is supposed to panic. drip-rs’s arenas panic if you use a handle to an object that was freed, and a test checks that:

#[test]
#[should_panic(expected = "use after free")]
fn using_a_freed_handle_panics() {
    let mut arena = Arena::default();
    let a = arena.insert("x");
    arena.remove(a.index(), false);
    let _ = arena[a];
}

#[should_panic] flips the contract: the test passes only if the body panics, and expected requires the panic message to contain that text.

// AoS: element i, field f, nf fields.
fn slot(i: usize, f: usize, nf: usize)
    -> usize {
    f * nf + i // bug: i * nf + f
}

fn main() {
    assert_eq!(slot(0, 0, 2), 0);
    assert_ne!(slot(1, 0, 2), 0);
    println!("first two checks pass");
    assert_eq!(slot(3, 1, 2), 7,
        "element 3, field 1");
    println!("never printed");
}

Integration tests

Files in a crate’s tests/ folder are integration tests. Each one is compiled as a separate crate that depends on yours, so it sees only your pub API, exactly as a user would^[1]. drip-rs’s tests/vm.rs starts like a user’s program:

use drip::compiler::compile;
use drip::gc::Heap;
use drip::{Options, Vm};

Helpers shared between test files go in tests/common/mod.rs. A common.rs would be compiled as one more test crate; a folder with a mod.rs is just a module, which each test file pulls in with mod common;^[1]. All of it adds up to several test programs per cargo test:

Bringing your own harness

The VM’s corpus is 246 Drip programs whose expected output is written in their comments (chapter 88). Wrapping all of them in one #[test] would report a single ok or FAILED for the lot. drip-rs instead switches the standard harness off for that file, in Cargo.toml:

[[test]]
name = "corpus"
harness = false

Then tests/corpus.rs is an ordinary program with its own main, and cargo simply runs it and looks at its exit code (adapted, paths shortened; ..Options::default() fills in the other fields):

fn main() {
    let options = Options {
        debug_gc: true,
        ..Options::default()
    };
    run_suite("corpus",
        |t| !should_skip(t, SKIP_VM),
        options);
}

That closure is the impl Fn(&str) -> bool from chapter 134. run_suite lists the .drip files, runs each through the VM, checks stdout, errors and exit code, prints one line per program the way libtest would, and calls std::process::exit(1) if any failed. A filter still works:

$ cargo test --test corpus \
    -- closure/close_over
running 3 corpus tests
test closure/
  close_over_function_parameter.drip ... ok
test closure/
  close_over_later_variable.drip ... ok
test closure/
  close_over_method_parameter.drip ... ok

corpus result: ok. 3 passed; 0 failed

Your turn

fn first_diff(want: &[&str],
    got: &[&str]) -> Option<usize> {
    todo!()
}

fn check_columns(xs: &[f64],
    vxs: &[f64]) {
    todo!()
}

What you learned

• #[test] functions pass by returning and fail by panicking. #[cfg(test)] mod tests keeps them beside the code, with access to private items.
• cargo test builds a test program per crate and per file in tests/, runs them, and filters by name.
• assert!, assert_eq! and assert_ne! check, and the last two print both sides. #[should_panic(expected = ...)] tests that a panic happens.
• Integration tests in tests/ are separate crates that see only the pub API; shared helpers go in tests/common/mod.rs.
• harness = false lets a test file be its own program, like drip-rs’s corpus runner.