Post

I can now run a GPT-4 class model on my laptop (The exact same laptop that could just about run a GPT-3 class model 20 months ago) The new Llama 3.3 70B is a striking example of the huge efficiency gains we've seen in the last two years simonwillison.net/2024/Dec/9/l...

simonwillison.netI can now run a GPT-4 class model on my laptopMeta’s new Llama 3.3 70B is a genuinely GPT-4 class Large Language Model that runs on my laptop. Just 20 months ago I was amazed to see something that felt …

Hillel

@hillelwayne.com

11 Dec 2024

This finally inspired me to try downloading and running an llm locally. Jesus does this run slowly. Is the only way to get it faster to get a better GPU?

Simon Willison

@simonwillison.net

Yeah, to get good performance you need enough GPU memory to fit the whole model - which is about 40GB for this one, so best off on a high-end Mac laptop or with multiple expensive NVIDIA graphics cards

11/12/24, 5:59 pm0 reposts0 likes

Simon Willison

@simonwillison.net

11 Dec 2024

The Llama 3.2 1B and 3B models run with much less resources and are still really fun to try out though

Hillel

@hillelwayne.com

11 Dec 2024

looks like llama3.2 passes the Hillel "gives hilarious answers to dumb riddles" test

Simon Willison

@simonwillison.net

11 Dec 2024

It's a really fun model, I love playing with the small ones because they can hallucinate all kinds of wild answers You can even run llama 3.2 on an iPhone apps.apple.com/us/app/mlc-c...

apps.apple.com‎MLC Chat‎MLC Chat lets users chat with open language models locally on ipads and iphones. After a model is downloaded to the app, everything runs locally without server support, and it works without internet ...