Post

Today I shipped a huge change to syntax highlighting in Discord: code blocks and file previews on web are now being highlighted by Tree-sitter, instead of highlight.js.

Why, you might ask? highlight.js is a genuinely great project, but it suffers from one great weakness: regular expressions. This comes with 2 main downsides: RegEx is often vulnerable to denial-of-service, and it has no semantic understanding of the code.

highlight.js is fairly resilient to ReDoS, but even so we'd occasionally have to field some security reports about it. Tree-sitter is not immune to DoS either, but we can tune it to protect against DoS in ways that aren't possible with RegEx.

More importantly, however, tree-sitter is semantically aware, meaning that it can highlight code in context rather than based on keywords and pattern matching. This means that your code blocks in Discord are now as accurate as the best code editors. For example, None is not a keyword anymore.

Shout-out to the tree-sitter contributors and the hundreds of independent grammar authors for creating such a strong ecosystem, and shout-out @fasterthanli.me for building Arborium (the basis for our new syntax highlighter).

If you want to check out the source code, it's available at github.com/discord/arbo.... Your favorite language may not be supported, but please feel free to submit a patch with new grammars: they need only be permissively licensed.

this broke the css classes going from `{alias}` to something like `language-{main}`. example being `hbs` => `language-html`. they have different use cases. please, at the very least revert that change. the `language-` prefix doesn't matter here. keep the aliases separate.

I don't follow, is this some issue with CSS highlighting or with some missing languages? It's true that some languages got aliased to a close equivalent, which I figured was better than completely dropping support for the language.

the aliasing itself in terms of highlighting isn't the issue, a lack of what alias is being used being exposed in the css is the issue

I'm unsure how exposing the alias in the CSS is causing an issue. Is it not highlighting properly?

some screen readers may use the HTML element classes to know what the language of the codeblock is. `less` for whatever reason is now aliased with `css` despite having a slightly different use case. `svg` is now registered as `xml` even tho the spec is basically xml with flair. what's with `yuri`?

As for the aliases, again I figured that having some approximate highlighting from analogous languages would be better than none at all. Is the behavior significantly regressed?

```xml <!-- This is in XML -->``````svg <!-- This is for SVG -->``` A third party CSS-only theme could distinguish the differences between the two codeblocks above. Looking at the code, SVG is an alias for XML. SVG syntax highlighting should be XML, yes. It should also state that it's SVG.

The alias used will also show up if you right click the message and copy text as that uses the raw input of the message. IMO it's a slight regression. I'm aware that third-party themes and plugins are not supported. It's that the themes is the only use case that I can think of for showing the lang