171. The plugin trait

A cast nobody checks

Chapter 90’s interface gives emit the same LoweredModule that lower returned. But the JavaScript plugin needs more than the base type holds: the ESTree program and the scope resolution. So its lower returns a bigger object, and its emit casts it back:

emit(_source, lowered: LoweredModule,
     plan: Plan): string {
  const l = lowered as JsLowered;
  return emitJs(l.js, l.program,
                l.resolution, plan);
}

as is a promise, not a check. I lowered xs = [] with the Python plugin and handed the result to the JavaScript plugin’s emit:

const lowered = pythonPlugin().lower(py);
jsPlugin().emit(py, lowered, p);

tsc --strict accepted it. Run, it returned undefined: not an error, not a string. The two halves of a plugin belong together, and the type system can’t see it.

The trait

Here is the Rust version, from crates/core/src/plugin.rs (doc comments shortened):

pub trait LanguagePlugin {
    /// carried from lower to emit
    type Extra;

    fn id(&self) -> &'static str;
    fn extensions(&self)
        -> &'static [&'static str];
    fn lower(&self, source: &str) -> Result<
        LoweredModule<Self::Extra>,
        CompileError,
    >;
    fn emit(
        &self,
        source: &str,
        lowered:
            &LoweredModule<Self::Extra>,
        plan: &Plan,
    ) -> Result<String, CompileError>;
}

type Extra; is an associated type: a type that each implementation chooses, the way Iterator lets each iterator choose its Item (chapter 134). LoweredModule is generic over it:

pub struct LoweredModule<X> {
    pub module: Module,
    pub annotations:
        HashMap<BindingId, Annotation>,
    pub extra: X,
}

The module and the annotations are what the core reads. extra is the plugin’s private data, the part TypeScript smuggled through the cast.

Two types implement the trait. JsPlugin sets type Extra = JsExtra: facts about the syntax nodes behind instructions and the VIR binding of each JavaScript binding (chapter 172 says why the AST itself can’t be kept). PyPlugin sets type Extra = PyExtra: the lowered Python, or None plus the reason when the source is outside the subset, so that emit returns the file unchanged (chapter 187). Both are empty structs: neither has options.

Now the mix-up from above, in Rust, against the real crates (rustc 1.97, trimmed):

error[E0308]: mismatched types
  JsPlugin.emit(py, &l, &p);
                    ^^ expected
  `&LoweredModule<JsExtra>`, found
  `&LoweredModule<PyExtra>`

One copy per plugin

The pipeline is generic, as chapter 132 said:

pub fn compile<P: LanguagePlugin>(
    source: &str,
    plugin: &P,
    opts: CompileOptions,
) -> Result<CompileResult, CompileError> {
    let lowered = plugin.lower(source)?;
    let analysis = analyze(&lowered.module);
    // options: opts + the annotations
    let p = plan(&analysis, options);
    let code =
        plugin.emit(source, &lowered, &p)?;
    let explain = explain(&p);
    Ok(CompileResult { code, plan: p,
        analysis, explain })
}

(Adapted from crates/core/src/lib.rs.) Each plugin gets its own monomorphized copy, compile::<JsPlugin> and compile::<PyPlugin>, and inside each copy Extra is one known type. Look at what comes out: CompileResult has the code, the plan, the analysis and the explain text. None of them mentions Extra. It is born in lower, used in emit, and dropped before compile returns.

trait Plugin {
    type Extra;
    fn lower(&self, s: &str)
        -> Self::Extra;
    fn emit(&self, s: &str,
        x: &Self::Extra) -> String;
}

struct Js;
struct Py;

impl Plugin for Js {
    // the number of lines
    type Extra = usize;
    fn lower(&self, s: &str) -> usize {
        s.lines().count()
    }
    fn emit(&self, s: &str, n: &usize)
        -> String {
        format!("// {n} lines\n{s}")
    }
}

impl Plugin for Py {
    // the first line
    type Extra = String;
    fn lower(&self, s: &str) -> String {
        let f = s.lines().next();
        f.unwrap_or("").to_string()
    }
    fn emit(&self, s: &str, f: &String)
        -> String {
        format!("# starts {f:?}\n{s}")
    }
}

// one copy per P: compile::<Js>,
// compile::<Py>
fn compile<P: Plugin>(s: &str, p: &P)
    -> String {
    let x = p.lower(s);
    p.emit(s, &x)
}

fn main() {
    let src = "xs = []\nys = []";
    for f in ["a.js", "b.py"] {
        let py = f.ends_with(".py");
        // both arms return a String
        let out = if py {
            compile(src, &Py)
        } else {
            compile(src, &Js)
        };
        println!("{f}:\n{out}\n");
    }
}

Why not dyn?

Chapter 131 kept a Js and a Py in one Vec<&dyn Plugin>. Try that with the real trait, and step through what rustc says:

The trait itself is fine. The Rust Reference’s rules for a dyn-compatible trait forbid generic methods, associated constants and associated types with generics, and ask for a receiver like &self^[1]. LanguagePlugin breaks none of them. But a trait object has to name every associated type, which is what E0191 says^[2]. And once you name it, dyn LanguagePlugin<Extra = JsExtra> and dyn LanguagePlugin<Extra = PyExtra> are different types. That is not fussiness. Code that calls lower through a vtable^[3] gets the result back and has to know its type, so the type can’t be left open.

The port’s answer is to branch before the generic call. From run_language in crates/wasm/src/lib.rs, where compile is imported under another name:

let r = match language {
    "javascript" | "js" =>
        compile(source, &JsPlugin, opts),
    "python" | "py" =>
        compile(source, &PyPlugin, opts),
    _ => return Err(/* ... */),
};

Both arms have the type Result<CompileResult, CompileError>, so the match has one type. The CLI does the same with the file’s extension. Chapter 132 left this as an open question; the answer needed no change to the trait.

Your turn

If the port ever needs a registry of plugins chosen at run time, the usual move is a second trait without the associated type, implemented for every plugin at once. That is a design, not code in engine/rust, and it is the exercise.

trait Plugin {
    type Extra;
    fn exts(&self)
        -> &'static [&'static str];
    fn lower(&self, s: &str)
        -> Self::Extra;
    fn emit(&self, s: &str,
        x: &Self::Extra) -> String;
}

struct Js;
struct Py;

impl Plugin for Js {
    type Extra = usize;
    fn exts(&self)
        -> &'static [&'static str] {
        &[".js", ".mjs"]
    }
    fn lower(&self, s: &str) -> usize {
        s.lines().count()
    }
    fn emit(&self, _: &str, n: &usize)
        -> String {
        format!("js {n}")
    }
}

impl Plugin for Py {
    type Extra = String;
    fn exts(&self)
        -> &'static [&'static str] {
        &[".py"]
    }
    fn lower(&self, s: &str) -> String {
        s.to_uppercase()
    }
    fn emit(&self, _: &str, u: &String)
        -> String {
        format!("py {u}")
    }
}

// Plugin with Extra hidden. No
// associated type, so &dyn works.
trait AnyPlugin {
    fn handles(&self, file: &str)
        -> bool;
    fn compile(&self, src: &str)
        -> String;
}

impl<P: Plugin> AnyPlugin for P {
    fn handles(&self, file: &str)
        -> bool {
        todo!()
    }
    fn compile(&self, src: &str)
        -> String {
        todo!()
    }
}

fn pick(file: &str)
    -> Option<&'static dyn AnyPlugin> {
    todo!()
}

What you learned

• LanguagePlugin has id, extensions, lower and emit, and an associated type Extra that carries each plugin’s own data from lower to emit. Passing one plugin’s output to another is E0308.
• compile<P: LanguagePlugin> is monomorphized per plugin, and its result never mentions Extra.
• &dyn LanguagePlugin is E0191: a trait object must name Extra, and each choice is a different type (E0271 for the other plugin).
• To pick a plugin at run time, branch first and call compile in each arm, or erase Extra behind a second, dyn-friendly trait.