Post

Large Language Models (LLMs) with "reasoning" capabilities are super hyped, but new research from Apple shows their "thinking" is often an illusion. Beyond a certain complexity, they fail completely. Key points in the 🧵.

The paper identifies three distinct "reasoning regimes" in puzzles like Tower of Hanoi and River Crossing. Frontier LRMs (Claude 3.7 Thinking, DeepSeek-R1) outperform standard LLMs only in the medium complexity zone. Beyond this, their accuracy collapses regardless of compute.

Worst of all, LRMs reduce their reasoning effort as problems get harder. Despite having spare token budgets, they underthink when complexity peaks, like a student giving up on an exam’s toughest question—even with extra time left 😅

Researchers even handed models step-by-step algorithms (e.g., for Tower of Hanoi). The result: identical failure points. LRMs couldn’t reliably execute prescribed logic—suggesting their "reasoning" is often pattern-matching, not general intelligence.