Post

"[...] the average [ChatGPT] query uses about 0.34 watt-hours, about what an oven would use in a little over one second, or a high-efficiency lightbulb would use in a couple of minutes. It also uses about 0.000085 gallons of water" - Sam Altman in blog.samaltman.com/the-gentle-s...

This blog post reads as delusional.

I wonder what they consider a "prompt". Text generation with transformers is autoregressive, and computationally N-squared on prompt length. In some cases the system prompt is already tonnes of tokens.

Prompt processing has never been the computationally heavy part. Also OpenAI, Anthropic, etc. have been caching prompts for a while now, amortizing the system prompt to nothing.

But given the median chatgpt user is asking a fairly basic question, it’s reasonable to assume a typical prompt is under 25 words. This doesn’t factor in multimodal inputs, but anecdotally, those are in the minority

built this over the weekend — these estimates are very very different than what folks have assumed… think we can trust them?

would also be interested if you had any feedback on this tool in general, i’m not nearly as well educated on this topic as you are

What figures are you using for that? I have found it infuriatingly difficult to get numbers that feel credible to me