Post

Working on a new blog post about local LLMs today! While running one of the tasks against llama 3.1 on my blazing fast M1 Max, I took a mactop snapshot. It's interesting to see the energy and GPU usage.

Chrissy LeMaire

@funbucket.dev

29 Mar 2025

Loaded Ollama and RAM jumped from 13.8 GB to 17.8 GB. GPU hit 97%, power spiked 27×. This is the before:

Chrissy LeMaire

@funbucket.dev

Interesting difference. The table below shows idle vs. maxed out doing work/inference, not just a loaded model. Before inference, only memory spiked as the model was loaded into memory. Not GPU or CPU. As an infrastructure person and toolmaker newish to AI, I find this all fascinating!

29/3/25, 9:27 am1 repost6 likes

No replies yet.