Steven Levithan

Steven Levithan

@slev.life

Creator → Regex+, Oniguruma-To-ES, xregexp.com, regex.cool Coauthor → Regular Expressions Cookbook, High Performance JavaScript slev.life

216 followers16 following60 posts

Oniguruma-To-ES now has 12M dl/m. Most of that comes from Shiki, which uses it to make VSCode-style syntax highlighting possible in native JS (no large WASM). I like to think it's contributing to Shiki's rapid growth :) Recent example is in Vercel's streamdown.ai

Neither @antfu.me nor I thought this was possible when we started down this road. But it's finally done 😁

🚢 Released emoji-regex-xs v2.0. Now matches all underqualified emoji included in Unicode's emoji-test.txt list, to better match @mths.be's excellent emoji-regex and real-world emoji use. Lib size reduced from 0.3 to 0.2 kB in the process :)

oniguruma-to-es v4.2.0 shipped! 🚢 Continues improving by far the most sophisticated regex translator in the world. JS coders who've never used Oniguruma might still find it useful if they want much more feature-rich regexes (with native perf!) github.com/slevithan/on...

github.comGitHub - slevithan/oniguruma-to-es: Convert patterns from Oniguruma (the regex engine used by Ruby, TextMate grammars, etc.) to native JavaScript RegExpConvert patterns from Oniguruma (the regex engine used by Ruby, TextMate grammars, etc.) to native JavaScript RegExp - slevithan/oniguruma-to-es

Migrated a bunch of tests from Jasmine to Vitest so I could use TypeScript more easily. Super nice! ⚡ Will be trying to use Vitest everywhere from now on.

Over the last week and with major help from RedCMD, we converted all of oniguruma-parser from JS to TS. 🎉 Surprised that all of this work uncovered 0 bugs (despite the high complexity of the migrated code), which I think speaks to the pre-existing high quality of the lib. github.com/slevithan/on...

Just shipped version 0.6 of oniguruma-parser, which includes a new optimization that is able to automatically identify and fix some ReDoS vulnerabilities without changing what the regex matches. It already identified one such regex in the Haskell grammar used by @shiki.style

Although it's not in the release notes, @shiki.style 3.2.0 includes slight-to-major performance improvements for many languages (for both the Oniguruma and JS engines), thanks to the new regex optimizer from `oniguruma-parser`. Worth upgrading!

More oniguruma-parser updates! An example of what it's optimizer/minifier can do, in the image. Some transforms can significantly improve perf for complex regexes 🚀

New launch! `oniguruma-parser` extracts the parser/traverser from my Oniguruma-To-ES transpiler, makes a ton of improvements, and adds new code generator and optimizer modules: github.com/slevithan/on... Great for TextMate grammar authors and anyone who works with Oniguruma regexes

github.comoniguruma-parser/src/optimizer at main · slevithan/oniguruma-parserParse Oniguruma regular expressions. Contribute to slevithan/oniguruma-parser development by creating an account on GitHub.

Oniguruma-To-ES v3.1.0 includes a new feature I think is new in JS: Lazy construction of RegExp objects, deferred until first use in a search (with nothing observably different before/after). Also allows lazy-compile based on pattern length, since only very long patterns are slow to construct in V8

JavaScript regexes got most things right (and are better with flag u/v), but there were a handful of deep mistakes in behavior/API (some inherited but mostly breaks from prior art in Perl, etc.). After deep-diving into Oniguruma edge cases… good god, it could have been much worse

The biggest improvement most people can make to their mastery of regex is usually not learning more advanced features, but truly understanding *backtracking* This short article of mine from 2007 illustrates in depth the mechanics of regex backtracking: blog.stevenlevithan.com/archives/gre...

blog.stevenlevithan.comPerformance of Greedy vs. Lazy Regex QuantifiersA common misconception about regular expression performance is that lazy quantifiers (also called non-greedy, reluctant, minimal, or ungreedy) are faster than their greedy equivalents. That's generall...

Just shipped 3 library updates. 2024 saw a lot of OSS 🚢ing: New libs Regex+ (plus related babel-plugin-transform-regex, regex-recursion, regex-utilities), Oniguruma-To-ES, Awesome Regex, and emoji-regex-xs, plus new major versions of parseUri and Regex Colorizer. Can you guess the theme? 🤔

.@shiki.style's JS engine has continually improved since being introduced in v1.15. As of the just-launched v1.24.4, it's no longer experimental! 🎉 The JS engine allows dropping the large Oniguruma WASM dependency while still sharing the same syntax highlighting grammars used by VS Code, GitHub, etc

oniguruma-to-es v0.8 is now the best regex flavor→flavor translator in the world. Tell me if you disagree! Accurately translates 99.9% of real-world Oniguruma regexes, and it's hard to find any 2 regex flavors with more complex and nuanced differences than Oniguruma & JS. Will continue improving!

🎉 oniguruma-to-es 0.8.0 released, adding another large chunk of hard-fought grammars to @shiki.style's supported list when using its lightweight JS engine (which is < 4% of the size of the WASM regex engine). 95% of all grammars (> 99.9% of all regexes) now supported. github.com/slevithan/on...

github.comGitHub - slevithan/oniguruma-to-es: Convert patterns from Oniguruma (the regex engine used by Ruby, TextMate grammars, etc.) to native JavaScript RegExpConvert patterns from Oniguruma (the regex engine used by Ruby, TextMate grammars, etc.) to native JavaScript RegExp - slevithan/oniguruma-to-es

Thanks for the sponsorship! 🙏

Anthony Fu@antfu.me

💖 For this November, we redistributed our funds to @slev.life, Color.js @lea.verou.me, MSW @kettanaito.com, @esm.sh, @pnpm.bsky.social, Iconify @cyberalien.dev for their amazing work on Open Source! 🙏 Thanks to all the sponsors for making this possible! opencollective.com/antfu/update...

Since individual grammars are composed of anywhere from dozens to thousands of regexes, supporting 92% of grammars means it's successfully translating the syntax and behavior of more than 99% of Oniguruma regexes to JS RegExp.

Steven Levithan reposted

Oniguruma-To-ES v0.3.0 now supports the vast majority of ways that recursion is used, including indirect recursion and multiple recursions in the same pattern — further increasing the number of supported grammars when used with Shiki syntax highlighter. github.com/slevithan/on... cc @antfu.me

github.comGitHub - slevithan/oniguruma-to-es: Convert patterns from Oniguruma (the regex flavor used in Ruby, TextMate grammars, etc.) to native JavaScript RegExpConvert patterns from Oniguruma (the regex flavor used in Ruby, TextMate grammars, etc.) to native JavaScript RegExp - slevithan/oniguruma-to-es

Regex+ (formerly `regex`) recently crossed 500 ⭐s, 500K mo. dls, and 5 mo. since initial release (19 releases). 🎉 This is the regex library I've wanted since ES6 added tagged templates. Try it out -- I think you'll find the "regexes are unreadable" reputation is outdated. github.com/slevithan/re...

github.comGitHub - slevithan/regex: JS regexes + future. A template tag for readable, high-performance, native JS regexes, with context-aware interpolation and always-on best practices.JS regexes + future. A template tag for readable, high-performance, native JS regexes, with context-aware interpolation and always-on best practices. - slevithan/regex