Max Spero

Max Spero

@max.pangram.com

slop janitor @pangram.com

219 followers425 following79 posts

CALL FOR AI DETECTION BENCHMARKS I believe it is vital that third parties are able to audit AI detectors like Pangram. Our largest sample size benchmark, from UChicago, was run on Pangram 3 and is a year old now.

View full thread

What I value in a benchmark is extraordinarily clean data, frontier LLM data, and a high enough difficulty that the benchmark is not saturated by most detectors.

CALL FOR AI DETECTION BENCHMARKS I believe it is vital that third parties are able to audit AI detectors like Pangram. Our largest sample size benchmark, from UChicago, was run on Pangram 3 and is a year old now.

View full thread

- How well do current detectors do on mixed authorship text? Ideally, research in this area explores the full spectrum of AI assistance from light grammatical edits to full co-authorship.

- Any sort of sliced evals around language, background, or ethnicity. Null results that show no bias are still interesting to us.

CALL FOR AI DETECTION BENCHMARKS I believe it is vital that third parties are able to audit AI detectors like Pangram. Our largest sample size benchmark, from UChicago, was run on Pangram 3 and is a year old now.

View full thread

I am putting out a call for new third-party evaluations that attempt to answer one or more of the following questions:

- How well do current detectors catch text from the latest frontier AI models? What about in high context settings, like deep research or an agent harness?

CALL FOR AI DETECTION BENCHMARKS I believe it is vital that third parties are able to audit AI detectors like Pangram. Our largest sample size benchmark, from UChicago, was run on Pangram 3 and is a year old now.

New LLMs are being released all the time, and the binary task doesn't capture the full spectrum of real world performance. A great benchmark will help shape how the world interacts with AI over the coming years.

I consider games I played in my teens and early 20s as more formative than my childhood games (Zoo Tycoon, RuneScape, Neopets etc.). And somehow I have probably sunk as many lifetime hours into League of Legends as the rest of this list combined.

Ryan Moulton@moultano.bsky.social

9 most formative games. I'm old

Max Spero reposted

I profiled the startup behind Pangram, the AI detector everyone’s talking about. It works really, really well. I wanted to understand how they did it. and also: why people still don’t trust them. (also, news: Penguin Random House is using it.) gift link! ✨

bloomberg.comThe World’s Most Reliable AI Detector Has a Human ProblemArmed with a near-perfect detection tool, Pangram is fighting to convince the public AI cheating can be caught.

Max Spero reposted

Pangram is surprisingly good. Teachers and graders should check it out. (And journal editors .) www.pangram.com

pangram.comAI Detector: Free AI Checker for ChatGPT, Claude & Gemini | PangramAI Detector for ChatGPT, Gemini & Claude — 99.98% Accurate. Detect AI-generated text in essays, articles & documents. Third-party verified results.
Will Oremus@willoremus.com

People have real qualms with AI detectors, and so do I, but the evidence is mounting that Pangram in particular very rarely gets it wrong (especially on longer chunks of text, but increasingly even on short ones). @brianx.bsky.social tested it and it didn't miss: www.nytimes.com/2026/08/13/t...

Max Spero reposted

The sudden release of an accurate AI-writing detector is going to have hilarious repercussions in politics and journalism, let me tell you. (It’ll also literally save higher education.)

Will Oremus@willoremus.com

People have real qualms with AI detectors, and so do I, but the evidence is mounting that Pangram in particular very rarely gets it wrong (especially on longer chunks of text, but increasingly even on short ones). @brianx.bsky.social tested it and it didn't miss: www.nytimes.com/2026/08/13/t...

Max Spero reposted

People have real qualms with AI detectors, and so do I, but the evidence is mounting that Pangram in particular very rarely gets it wrong (especially on longer chunks of text, but increasingly even on short ones). @brianx.bsky.social tested it and it didn't miss: www.nytimes.com/2026/08/13/t...

nytimes.comI Tested a Popular A.I. Slop Detector. It Felt Empowering.Pangram excels at distinguishing chatbot-generated words from human writing. But it’s not reliable for spotting artificial images.

Good question! Our technical report specifically looks into ESL, demonstrating no elevated false positive rate across any English language learner benchmarks

Will Oremus@willoremus.com

This is a great question. I'd love to see research on Pangram's accuracy for non-native speakers specifically. Logically its error rate couldn't be *that* high or else its overall error rate couldn't be so miniscule. But any amount of bias makes the tool's use problematic from a fairness standpoint.

Max Spero reposted

Comments are off so I have to quote, but Pangram says that ~50% of segments have signs of AI. I highly doubt this is a false positive. @masnick.com can you confirm no AI use, or do you believe the indicated degree of AI use should be lower? www.pangram.com/history/17a3...

Mike Masnick@masnick.com

Listening to today's episode (excellent as usual) and they talk about accusations of things written by AI and name a tool I'd not heard of (Pangram). They claim it's pretty reliable. I feed it my piece from @liberalism.org this week that I *know* is human written because *I* wrote it...

a wiki editor@aionwikipedia.bsky.social

the thing that this guy is leaving out of his post is that he is on record talking about how he uses AI in the article writing process, up to and including asking claude to rewrite stuff www.techdirt.com/2024/04/29/h...

Max Spero reposted

I talked to the head of the Commonwealth Foundation and to one of the accused writers in the Granta AI scandal — & also had the past 15 years of the prize run through Pangram, which flagged only four stories as containing AI: all from this year and last. [gift link!]

theatlantic.comThis Literary AI Scandal Changes EverythingA magazine’s response to accusations of publishing AI-generated fiction points to a new phase in the struggle to keep literature human.

Pretty cool analysis of AI use on Substack by @taylorlorenz.bsky.social

Taylor Lorenz@taylorlorenz.bsky.social

How Much of Substack Is Actually AI? I used @pangram.com (AI detection tool) to analyze thousands of posts from the top Substack newsletters across every category to find out. Here's what I discovered 👇🏻

Max Spero reposted

If you do this, or your company does this, the rest of society should treat you like someone who walked into Notre Dame and pooped on the altar.

Last year I met a writer who casually referenced "the AI-generated piece in the New York Times last week." I hadn't heard of it so asked for more info. She told me that she started reading it and immediately recognized the AI voice. "Of course that's chat. My bestie."

This post is unavailable.

She complained how she missed 4o but she was getting used 5, even if it was more difficult to talk to. And how she would recognize that voice anywhere.

I was a little incredulous but Pangram agreed with her. Today in retrospect I think a lot more people can tell. But I was amazed at the time. She clearly had a highly attuned internal AI sensor from all of her time talking with "Chat."

Max Spero reposted

Pangram is releasing a new model today! Pangram 3.2 is a significant improvement over 3.1 in several aspects - Improved performance on 'humanized' text - Improved performance on adversarial prompts - Minimum word reduced from 75 to 50 - Overall improvement on Claude 4.6 recall

Wow this is so cool

hikikomorphism@hikikomorphism.bsky.social

For any AI system, there is a set of euphemisms and dual use framings that will allow it to construct nearly any output. This jailbreak teaches Gemini 3 Pro to construct and step into such framings on the fly, and thus to route around its own safety infrastructure. recursion.wtf/posts/jit_on...

ai dot com purchased for $70m. METR evals going exponential. former crypto grifters signalling we're in a bubble big labs signalling recursive self improvement I wonder, if the bubble pops, will it even matter?

Max Spero reposted

I was going to write a whole effortpost about model evaluations but instead here, just take this chart

Digital ads are one of the greatest positive-sum technologies of the 21st century. Google Search has 4 billion users, and they make on average $40 per user per year. This type of business model couldn't exist without ads; Google cannot find 4 billion people who would pay $40 for ad-free search.

View full thread

Yes, there are lazy and bad ways to do ads. One of the worst things companies can do is to show ads to already-paying customers. This happens most often when consumers have no alternative to a monopoly.

But overall, I believe that an open Internet and free technologies like ChatGPT will bring so good into the world, and ads are the only way to make tech like this universally free.

Digital ads are one of the greatest positive-sum technologies of the 21st century. Google Search has 4 billion users, and they make on average $40 per user per year. This type of business model couldn't exist without ads; Google cannot find 4 billion people who would pay $40 for ad-free search.

Ads enable a powerful technology to become a global utility. They are the great equalizer: even the poorest user can pay for the service with their attention.

Digital ads have better targeting and attribution. Well-targeted ads cost more, meaning that more marketing dollars are exhausted on fewer ad impressions. A world where you see half as many ads because they are twice as relevant is a world in which everybody is happier.

Pretty cool project on /r/localllama - they take human written text and sloppify it 10x with 4o-mini, then train the model to de-slop by reversing the transformation

Max Spero reposted

🚨 New Study 🚨 @arxiv.bsky.social has recently decided to prohibit any 'position' paper from being submitted to its CS servers. Why? Because of the "AI slop", and allegedly higher ratios of LLM-generated content in review papers, compared to non-review papers.

You are up to date.