Post

Pangram's new model was only just released 7/29/2026. There basically hasn't been any opportunity yet for the failure modes of this AI detector to present themselves. "By all accounts" is an irresponsible claim when there are no real accounts available.

The thing I worry about here is, it’s an arms race: as the LLMs get better, Pangram will get worse, probably even as they retrain it.

> Obviously purveyors of AI writing don't want this. Strong disagree. I'm extremely excited about these technologies, and also think it's a good thing when their application is transparent and detectable. *Plagiarists* don't want this. It's completely orthogonal to supporting AI.

This is not true at all, it uses a transformer network like all LLMs (leaving aside for the moment the false distinction you set up)

No, this isn’t true.

Well, it could be if there just isn't much (or any) non-native English in the test sets...

My colleague @matteowong.bsky.social argued persuasively in May that Pangram's accuracy actually makes it more dangerous, because it will tempt folks to overrely on it: www.theatlantic.com/technology/2... Still, its accuracy has improved further since then. Is there a threshold of acceptability?

Their technical paper asserts they have no FPR bias in a large sample of non-native speaker (same sample as that used by Stanford 2023 study that raised spectre of bias)

It’s enough to justify it. There is no real trade off.

I hate AI slop, but I also tend to believe people are innocent until proven guilty. I can see this ruining some innocent person's career. How do you prove you didn't use AI?

I feel like a big reason that AI detectors got a bad rep is because they are a case of AI guys selling us a solution to a problem they caused.