Post

this isn't cheating, and it's not "absolutely bizarre"! this is just how LLMs work. they are improv machines. they do not have any sense of truth or fiction bc they don't "know" anything. they can't even "know" if they "looked" at an "image." this is totally unsurprising

"seeing" something that isn't there is literally the definition of "hallucination" so i think these researchers had to know what they were doing & were doing it on purpose

the models generally "championed" for radiology are typically specialized, not commercial chatbots aka "frontier models" they still do the stupidest shit — like correlating diagnosis based on which hospital name was on the image — bc there's no such thing as artificial *intelligence*

I saw something similar a couple years ago about AI more accurately diagnosing TB than doctors. Turns out it was just dating the image, and older images in the dataset were more likely to be of a TB patient.

i think it's bizarre how people seem to not realize "it's good for a few things, in certain circumstances, but not others" bc it's not like they try to wash clothes with their stovetop obviously marketing is toxic and misleading but you'd figure RESEARCHERS would know the basics of how it works

And if you can’t verify the output??

what do you mean?