Post

this isn't cheating, and it's not "absolutely bizarre"! this is just how LLMs work. they are improv machines. they do not have any sense of truth or fiction bc they don't "know" anything. they can't even "know" if they "looked" at an "image." this is totally unsurprising

"seeing" something that isn't there is literally the definition of "hallucination" so i think these researchers had to know what they were doing & were doing it on purpose

answer-shaped extrusions can be useful in some situations but literally zero of those situations are where you need 1. facts about reality 2. truth 3. probability 4. certainty 5. important decision-making

I’m a little surprised that people don’t understand ai/ml algorithms to be pattern finding machines. And they might not find the patterns you expect! (It might be more accurate to call them correlation finding machines)

i wouldn't even say "pattern-finding" but they can be reasonably good at surfacing/extruding them ~accidentally~ hehe yeah, idk why people who ought to know better treat llms like universal tools. i know it's marketed that way but that literally harms the perception of the product? it's weird

well maybe these researchers really are the dumb babies they sound like, bc llms don't EVER work by "diving into the data" and using "probability" that implies the llms analyze and calculate. and they can't! responses are dictated by statistical correlations between TOKENS in the CORPUS

I saw something similar a couple years ago about AI more accurately diagnosing TB than doctors. Turns out it was just dating the image, and older images in the dataset were more likely to be of a TB patient.

incredible stuff i have to say, i had a chatbot help me in identifying something… bc what i needed was basically "fuzzy search" and it ingested lots of research papers that weren't connected to each other except by a couple words and concepts BUT i confirmed it myself & it also made shit up

i think it's bizarre how people seem to not realize "it's good for a few things, in certain circumstances, but not others" bc it's not like they try to wash clothes with their stovetop obviously marketing is toxic and misleading but you'd figure RESEARCHERS would know the basics of how it works