Post

🧵 Extracting important bits from Dario's essay on DeepSeek https://darioamodei.com/on-deepseek-and-export-controls

The 3 dynamics of AI development - Scaling laws - Shifting the curve - Shifting the paradigm

DeepSeek V3 was the real innovation (a month ago). Specifically they innovated on an area called the KV cache using a method called "mixture of experts".

DeepSeek did not train as good a model for $6M. Also we got some insight into the true cost of Claude 3.5 Sonnet at "a few $10Ms"

All of this is expected progress on the same curve. What's different is the fact that China produced this.