Post

Companies have a strong incentive to make their LLMs look better on benchmarks. Sometimes, how models are trained or evaluated can make those scores look better than what you see in real-world use. A benchmark shows how well a model performs on a test, not how capable it is in the real world.