Post

@johnstewart.dev

🧵 Extracting important bits from Dario's essay on DeepSeek https://darioamodei.com/on-deepseek-and-export-controls

darioamodei.comDario Amodei — On DeepSeek and Export ControlsOn DeepSeek and Export Controls

@johnstewart.dev

The 3 dynamics of AI development - Scaling laws - Shifting the curve - Shifting the paradigm

@johnstewart.dev

When it comes to shifting the paradigm he discusses that in 2024 the real shift has been adding in reinforcement learning (RL) on top of pre-trained models.

@johnstewart.dev

DeepSeek V3 was the real innovation (a month ago). Specifically they innovated on an area called the KV cache using a method called "mixture of experts".

@johnstewart.dev

DeepSeek did not train as good a model for $6M. Also we got some insight into the true cost of Claude 3.5 Sonnet at "a few $10Ms"

@johnstewart.dev

All of this is expected progress on the same curve. What's different is the fact that China produced this.

29/1/25, 10:51 pm0 reposts0 likes

@johnstewart.dev

DeepSeek's cost for the ~50k Hopper chips is ~$1B, which isn't far off from US companies.

@johnstewart.dev

R1 isn't as interesting as V3. It's using RL like o1

@johnstewart.dev

Export controls are still the best tool we have when it comes to preventing China from winning the AI race.

@johnstewart.dev

If you liked this consider giving a follow and checkout my newsletter below. 🖖 www.breakfast.dev

breakfast.devThe Breakfast Dev | John Stewart | SubstackJohn Stewart's newsletter on software engineering, technology, AI, the future, and more! Click to read The Breakfast Dev, by John Stewart, a Substack publication. Launched 4 months ago.