Post

Every now and then I throw some rocks at the latest AI shit just to know my enemy. This "agentic" shit is wild. Less than useless with open weight models. Barely cobbled together harnesses. The amount of tokens, hardware, and energy it takes to puke out useless slop is astounding.

Unless we just see entirely new hardware approaches with orders of magnitude levels of improvement, there's no way in hell the unsubsidized cost of this isn't fucking ridiculous.

The margins on frontier model inference actually do seem to be positive, from what I've read. I imagine they benefit massively from economies of scale, specialized models, aggressive shared KV caches, and the like.

Reuters reported 80% gross margin from Anthropic based on its leaked IPO documents

IDK if that number is correct, or if it accounts for things other than inference, but there's a nice discussion here bsky.app/profile/simo...

Like, "oh yeah, this new car burns 1 gallon of gas for every 10 feet but it sounds cool as hell while it does it!"

yeah, I don't think anybody believes the training is netting any cash. nevertheless, these are the numbers we have.

well that isn't really what I'm saying. Training is obviously shoveling money into the fire. They're saying they're making money on inference, but they aren't saying if they're making enough money on inference to recover the money burnt on training, and if they can do so before the model is obsolete