Post

Every now and then I throw some rocks at the latest AI shit just to know my enemy. This "agentic" shit is wild. Less than useless with open weight models. Barely cobbled together harnesses. The amount of tokens, hardware, and energy it takes to puke out useless slop is astounding.

Unless we just see entirely new hardware approaches with orders of magnitude levels of improvement, there's no way in hell the unsubsidized cost of this isn't fucking ridiculous.

The margins on frontier model inference actually do seem to be positive, from what I've read. I imagine they benefit massively from economies of scale, specialized models, aggressive shared KV caches, and the like.

Reuters reported 80% gross margin from Anthropic based on its leaked IPO documents

IDK if that number is correct, or if it accounts for things other than inference, but there's a nice discussion here bsky.app/profile/simo...

I'm just so unbelievably skeptical of any numbers these clowns put out. Even if inference is, in isolation, profitable, does it actually recoup the cost of training before becoming obsolete? It's also so funny that agentic shit burning through tokens is how they're selling more.

Like, "oh yeah, this new car burns 1 gallon of gas for every 10 feet but it sounds cool as hell while it does it!"

yeah, I don't think anybody believes the training is netting any cash. nevertheless, these are the numbers we have.

if the leaks are true, they're turning a profit now. but we can only speculate, and the real S-1 will tell.

you could be right! I appreciate you staying skeptical.

They can also lie and mislead on the S-1. There is immense pressure for these things to at least *appear* profitable in the long term. They will need to be making an unfathomable amount of money to justify the buildout, and good lord the depreciation on all that hardware. Yikes.