Post

Every now and then I throw some rocks at the latest AI shit just to know my enemy. This "agentic" shit is wild. Less than useless with open weight models. Barely cobbled together harnesses. The amount of tokens, hardware, and energy it takes to puke out useless slop is astounding.

Unless we just see entirely new hardware approaches with orders of magnitude levels of improvement, there's no way in hell the unsubsidized cost of this isn't fucking ridiculous.

The margins on frontier model inference actually do seem to be positive, from what I've read. I imagine they benefit massively from economies of scale, specialized models, aggressive shared KV caches, and the like.

Reuters reported 80% gross margin from Anthropic based on its leaked IPO documents

IDK if that number is correct, or if it accounts for things other than inference, but there's a nice discussion here bsky.app/profile/simo...

I'm just so unbelievably skeptical of any numbers these clowns put out. Even if inference is, in isolation, profitable, does it actually recoup the cost of training before becoming obsolete? It's also so funny that agentic shit burning through tokens is how they're selling more.

yeah, I don't think anybody believes the training is netting any cash. nevertheless, these are the numbers we have.

well that isn't really what I'm saying. Training is obviously shoveling money into the fire. They're saying they're making money on inference, but they aren't saying if they're making enough money on inference to recover the money burnt on training, and if they can do so before the model is obsolete

if the leaks are true, they're turning a profit now. but we can only speculate, and the real S-1 will tell.