Post

MrChris
@mrchris2000.github.io

Put aside the quality question; the biggest issue I have with local llms are the inference speeds. I can load everything into my 16gb 9070xt and get 40tps, but that limits model size and then youre back to the quality question really quickly. At some point that dgx spark looks mighty attractive.

MrChris
@mrchris2000.github.io

It's funny because nvidia win either way. Frontier labs paying for data centre capacity, or local users paying for local tokens, everyone is paying nvidia. Turns out selling shovels is an ever green business opportunity.

0 reposts0 likes

No replies yet.