Post

Benjamin Warner
@benjaminwarner.dev

Good LLMs do know and/or can reason about these things. Small, cheap, and often free LLMs are the models which cannot. Remember the glue on pizza Reddit post that the subpar Google AI cited uncritically? Bing’s then integration of GPT 3.5 recognized the Reddit post as sarcasm.

Mark@mbainter.bsky.social

LLMs don't reason, so it doesnt know that the reddit post is likely an uncritical repost of the other articles. It doesn't follow the discussion to see it is (probably) debunked in the comments. All things a marginally educated human would do. AI summary raises that bar, because of ai hype.

0 reposts0 likes

Benjamin Warner
@benjaminwarner.dev

A modern example: When attempting to trick GPT 5 + search with a question on the health benefits of raw milk, GPT 5 Fast (the less smart one) started out by citing the raw milk institute before eventually concluding there aren’t any benefits and citing high quality sources.