Post

Lukas Wirth
@lukaswirth.dev

Turns out the TokenStream implementation of rust-analyzer's proc-macro server is super slow when it comes to one of the most common proc-macro operations: concatenation. I noticed that in Zed's codebase, there were ~4 derives that just straight up together took 8 seconds to expand in rust-analyzer.

Lukas Wirth
@lukaswirth.dev

All that proc-macro was doing though is `quote!`ing a bunch, not much more. So I spent the day rewriting the implementation of TokenStream used by the server to make use of an immutable shared tree instead to get super cheap concatenations.

Lukas Wirth
@lukaswirth.dev

And sure enough, this changed the 8 second expansion time into a 100ms! A bit more info can be found in the PR description here github.com/rust-lang/ru.... This should improve general speed of rust-analyzer in the majority of projects given how pervasive these kinds of macros are in Rust.

github.comproc-macro-srv: Reimplement token trees via immutable trees by Veykril · Pull Request #21097 · rust-lang/rust-analyzerFor macros, rust has the concept of TokenTrees and TokenStreams which are basically a tree-like form of the underlying tokenized input with (){}[] delimiters forming Groups (internal nodes) and all...

6 reposts74 likes

Lukas Wirth
@lukaswirth.dev

Oh almost forgot! For the folks using RustRover, you will benefit from this too as RustRover uses the proc-macro server for its proc-macro expansion as well!