Post

for all talks of local-first AI, given controversies over datacenters, I don't think people are willing to hear that local LLM inference is *probably* less environmentally friendly than cloud inference the majority of the time.

im going to be put on a list called "datacenter defenders" or something but like. if you think the nearest datacenter is ruining the environment then your local power sources are almost definitely worse lol

i think that depends a lot on the hardware though i.e. a gaming laptop that's built on components that optimize performance per watt feels like it should still be more efficient even taking all the overhead into account, just because DC hardware is optimized for different variables

datacenters are definitely concerned with performance per watt! it's pretty key to profitability, they're not *trying* to maximize power usage. the way that gaming laptops achieve portability/any battery life at all is by sacrificing a great deal of performance and fidelity

(e.g. runs slower, generates less pleasing results). they're not even much more efficient than desktops, and they run on lithium ion batteries, which introduce inefficiencies of their own.