Post

As a side-side-project I'm working on a classifier for my emails (given that 99.9% of all emails are junk but have a DMARC policy, the anti-spam filters are not doing much anymore). Long story short my whole Gmail is in a multi-gigabyte SQLite and I can find any email in milliseconds

Strip the quoted reply chains before indexing with FTS5, or every search hit is the same paragraph quoted twelve times down a thread.

Parsing is the cheap part. Most of a mail client's CPU goes to rendering HTML bodies and keeping its search index in sync, which a one shot script never has to do.

That's what I believed, but how exactly? Ingesting and indexing is not even a blip on the radar. A couple of µs/mail on my 10-years-old-never-emptied inbox. Once indexed, search is FAST. And displaying HTML... If they weren't so busy rolling custom utterly stupid renderers maybe I guess?