133. Lifetimes

The tick you skipped

Chapter 126 showed you drip-rs’s token type and asked you to ignore one piece of it:

pub struct Token<'src> {
    pub kind: TokenKind,
    pub lexeme: &'src str,
    pub line: usize,
}

A token’s lexeme is a &str, a borrowed slice of the program’s source text. It isn’t a copy. So what happens if you delete the 'src and write the struct the way you’d write a TypeScript interface? Rust 1.97 on this machine (trimmed):

error[E0106]: missing lifetime specifier
2 |     lexeme: &str,
  |             ^ expected named lifetime
  |               parameter
help: consider introducing a named
  lifetime parameter
1 ~ struct Token<'a> {
2 ~     lexeme: &'a str,

Your chapter 28 scanner gave each token source.slice(start, current), a new string kept alive by the garbage collector. A Rust token holds a pointer into someone else’s text, which nobody keeps alive for it. So the compiler asks: whose text, and for how long?

How long is a borrow valid?

Here is the smallest way to get it wrong, in the diagram below. The scanner is cut down to split on spaces, but its signatures are the real ones (adapted from companion/rust/src/scanner.rs). src owns the text and is dropped at the closing brace (chapter 124), and t is used after that. Step through what the compiler sees:

The real error, from compiling that program (trimmed):

error[E0597]: `src` does not live long
  enough
51 | let mut sc = Scanner::new(&src);
   |                  ^^^^ borrowed value
   |                  does not live long
   |                  enough
53 | }
   | - `src` dropped here while still
   |   borrowed
54 | println!("{}", t.lexeme);
   |                -------- borrow later
   |                         used here

Chapter 125’s three points again: where the borrow started, where the owner died, where the borrow was still needed. And note what is not an error: sc dies at the same brace, but t borrows the source, not the scanner.

Naming lifetimes

Inside a function body the compiler works lifetimes out by itself. It needs your help at the boundaries: a struct that holds a reference, and a signature that returns one. 'src is a lifetime parameter, a name for “however long the borrowed source is valid”, declared in angle brackets like a type parameter (chapter 132):

struct Scanner<'src> {
    src: &'src str,
    pos: usize,
}

impl<'src> Scanner<'src> {
    fn next(&mut self) -> Token<'src> {
        // ... skip spaces, find the word
        let s = &self.src[at..self.pos];
        Token { lexeme: s, line: 1 }
    }
}

Read next’s signature aloud: “borrow the scanner mutably for a moment, and hand back a token that is valid as long as the source is”. Same name, same span.

A plain function with two inputs, longer, returns whichever string is longer. Without annotations, Rust 1.97 asks which input the answer borrows from (trimmed):

error[E0106]: missing lifetime specifier
1 | fn longer(a: &str, b: &str) -> &str {
  |              ----     ----     ^
  = help: this function's return type
    contains a borrowed value, but the
    signature does not say whether it is
    borrowed from `a` or `b`

The fix it suggests is fn longer<'a>(a: &'a str, b: &'a str) -> &'a str: the result is valid only while both inputs are.

The rules that let you leave them out

Chapter 126’s exercise was fn take_ident(src: &str) -> &str, with no ticks at all. That works because of three elision rules, which fill in lifetimes when the answer is obvious^[2]:

1. Each reference parameter gets its own lifetime.
2. If there is exactly one input lifetime, every output reference gets it.
3. In a method, if one input is &self or &mut self, every output reference gets self’s lifetime.

If an output is still unnamed after that, you get E0106, as longer did. The compiler never looks at the body to decide^[1].

Rule 3 is where drip-rs has to be explicit. Take a word splitter shaped like the scanner, and write its method the short way: fn next_word(&mut self) -> Option<&str>. Rule 3 ties the result to the borrow of the splitter. Then holding two words at once, which the compiler does with previous and current all the time, is refused (from a test program, trimmed):

let a = w.next_word();
let b = w.next_word();
println!("{:?} {:?}", a, b);

error[E0499]: cannot borrow `w` as mutable
  more than once at a time
35 | let a = w.next_word();
   |         - first mutable borrow
36 | let b = w.next_word();
   |         ^ second mutable borrow

While a is alive, the splitter counts as mutably borrowed, so there can’t be a second call. drip-rs’s -> Token<'src> says the token borrows the source instead, so the scanner is free again as soon as the call returns.

The longest lifetime: ‘static

One lifetime is built in. 'static means “valid for the whole run of the program”. String literals have it, because their text is stored in the program’s binary, and a &'static str can be used wherever a shorter lifetime is wanted. drip-rs’s error tokens rely on that:

fn error(&self, message: &'static str)
    -> Token<'src> {
    Token {
        kind: TokenKind::Error,
        lexeme: message,
        line: self.line,
    }
}

The message outlives any source, so it fits in a Token<'src>. If a compiler hint suggests 'static to silence a borrow error, the Rust book advises asking first whether the reference really should live that long^[1].

#[derive(Clone, Copy, Debug)]
struct Token<'src> {
    lexeme: &'src str,
    line: usize,
}

struct Scanner<'src> {
    src: &'src str,
    pos: usize,
}

impl<'src> Scanner<'src> {
    fn new(src: &'src str) -> Self {
        Scanner { src, pos: 0 }
    }

    // The token borrows the source,
    // not the scanner.
    fn next(&mut self) -> Token<'src> {
        let b = self.src.as_bytes();
        while self.pos < b.len()
            && b[self.pos] == b' '
        {
            self.pos += 1;
        }
        let at = self.pos;
        while self.pos < b.len()
            && b[self.pos] != b' '
        {
            self.pos += 1;
        }
        if b.get(at) == Some(&b'@') {
            return self.error();
        }
        let s = &self.src[at..self.pos];
        Token { lexeme: s, line: 1 }
    }

    // A &'static str fits in a
    // Token<'src>: it lives longer.
    fn error(&self) -> Token<'src> {
        let m = "Unexpected character.";
        Token { lexeme: m, line: 1 }
    }
}

fn main() {
    let src = String::from("let x @ 1");
    let mut toks = Vec::new();
    {
        let mut sc = Scanner::new(&src);
        for _ in 0..4 {
            toks.push(sc.next());
        }
    } // sc is dropped here
    for t in &toks {
        println!("{:?}", t);
    }
}

Your turn

struct Words<'src> {
    src: &'src str,
    pos: usize,
}

impl<'src> Words<'src> {
    fn new(src: &'src str) -> Self {
        Words { src, pos: 0 }
    }

    fn next_word(&mut self)
        -> Option<&str> {
        todo!()
    }
}

struct Span {
    start: usize,
    end: usize,
}

impl Span {
    fn text<'s>(&self, src: &'s str)
        -> &str {
        todo!()
    }
}

fn longer(a: &str, b: &str) -> &str {
    todo!()
}

What you learned

• A lifetime is the span during which a reference must stay valid. The borrow checker makes it fit inside the life of the value borrowed, or reports E0597.
• Lifetime annotations like 'src name lifetimes on structs, impls and signatures. They describe relationships; they never extend a value’s life.
• Three elision rules fill in the obvious cases. When they can’t, E0106 asks you which input an output borrows from.
• Rule 3 ties a method’s result to self; drip-rs’s Token<'src> ties it to the source instead.
• 'static lasts the whole run; string literals have it.