Post

AI RANT Following the cancellation of three major debut book deals over AI suspicions, I decided to put my own (needless to say, 100% human-written) prose to the test. My detector of choice was Originality AI, and the results proved to be jarring, to say the least. 🧵1/5

AI RANT (cont’d) The picture above isn’t a case of “false positives.” It’s a case of ALL sentences in a 300-word sample being classified as either ~100% AI- or ~100% human-written *depending on a single punctuation mark*. 🧵2/5

128 reposts1.1K likes

AI RANT (cont’d) The most troubling thing about this is, Originality AI is apparently used by agents & publishers alike. Dear agents and publishers, PLEASE DO NOT FORGET: AI “detectors” are slop machines same as the slop machines whose output they purport to analyze. 🧵4/5

Do you have specific examples of agents or publishers using it? I'd love to share with anyone in the industry my specific experience with informing Originality about serious and ridiculously broken false positives and them doubling down on them.

Building these on the gateway side, this tracks. Detectors read surface stats, never intent. So they flag a human who writes in fragments and pass a model parroting a clean template. The real lever isn't 'human vs AI', it's verifiability: sources + a trace you can audit.

AI RANT (cont’d) AI slop machines excel at pattern matching, but what they can’t do for you is analyze the author’s _intent_. You hit them with too many sentence fragments or em dashes, and they’ll claim you’re AI; they have zero idea whether you really _meant_ it or not. 🧵5/5

I've had the em dash beaten into me through high school, lots of older books (like Lem) and then additionally through my university where Chicago Style was law. seeing people assume that using the em dash is a surefire, 100%, never-miss way to identify AI writing kills me inside every time.

Per my other comment, Pangram was also the ONLY one mentioned in the two canceled book deals and the other major accusation. They're built in a completely different way, and I think it's really important to know of any examples of false positives there vs. the super explicitly scammy ones!

Oh originality is EXPLICITLY a massive scam/fraud. I told them they have basically built an em dash detector a year ago after a similar result (fully human-written chapter flagging as 90%+ AI, switching four natural em dashes to commas over 1,000+ words changed it by over 40%).

Despite being constantly called "intelligent", it is bitter LLMs don't go much further than keyword searches. Also, old MSWord had this "AutoSummary", basically just selecting a number of complete sentences from original. Worked poorly to fairly well (like LLMs), probably with 0.001% consumption.

I think something that needs said as well is that they're slop machines that are guided initially by human intent. A HUMAN undoubtedly set up the training to make it highly suspicious of things like em dashes, because right now it's trendy to point to them as a signature of AI.

Plenty of people who are otherwise aware of just how fallible generative AI is just completely forget that when they're asking generative AI if something is generative AI. -.-

A ship's bell AND diver's lantern is terrible. Maybe 'or', but not 'and'. But also just clunky suggestions. Like it has to say something so will say any old thing.