Post

Lukas Wirth
@lukaswirth.dev

super excited to see the proc-macro server RPC rewrite being worked on, once this here is finished it should allow us to implement all the remaining proc-macro features (except for `expand_expr`)! And afterwards we'll be tackling parallelization! github.com/rust-lang/ru...

2 reposts22 likes

Lukas Wirth
@lukaswirth.dev

For context as to why we still cannot implement `expand_expr`, that feature effectively turns all proc-macro into eager macro expansions which would basically prevent rust-analyzer from doing a lot of incremental caching, so we just cannot implement this as designed doc.rust-lang.org/stable/proc_...

doc.rust-lang.orgTokenStream in proc_macro - RustThe main type provided by this crate, representing an abstract stream of tokens, or, more specifically, a sequence of token trees. The type provides interfaces for iterating over those token trees and...
Lukas Wirth
@lukaswirth.dev

And some additional notes for parallel proc-macro expansion, today r-a spawns a single proc-macro server that does expansion one at a time, meaning whenever r-a wants to expand a proc-macro, it takes a global lock on the proc-macro server!

Lukas Wirth
@lukaswirth.dev

The annoying part, you can't trivially parallelize this on the proc-macro server because proc-macros execute in (and are in control 😵‍💫 of) the process environment, so technically running multiple at once can cause issues. But even ignoring that they can change the environment,

Lukas Wirth
@lukaswirth.dev

we'd still only be able to execute proc-macros in parallel if they are run within the same environment (working dir, env vars) which only happens within the same crate really. So the alternative we are likely looking at is to just spawn multiple servers and handle them like a process pool.