Post

Simon Willison
@simonwillison.net

DeepSeek released a whole family of inference-scaling / "reasoning" models today, including distilled variants based on Llama and Qwen Here are my notes on the new models, plus how I ran DeepSeek-R1-Distill-Llama-8B on my Mac using Ollama and LLM simonwillison.net/2025/Jan/20/...

simonwillison.netDeepSeek-R1 and exploring DeepSeek-R1-Distill-Llama-8BDeepSeek are the Chinese AI lab who dropped the best currently available open weights LLM on Christmas day, DeepSeek v3. That model was trained in part using their unreleased R1 …

31 reposts203 likes

Karl Pettersson
@karlpettersson.bsky.social

One really should not need distilled models, because only one expert is needed at a time (have not tried this myself yet due to the large download size, so I do not know whether expert switching will become a massive bottleneck with mmap).

Karl Pettersson@karlpettersson.bsky.social

How would a non-distilled 671B model work on such a laptop? You only use one 37B expert at a time, but all experts have to be loaded in memory. In principle, it should work with mmap, but will it lead to dreadful performance (like 0.01 tok/s)?

huggingface.counsloth/DeepSeek-R1-GGUF at mainWe’re on a journey to advance and democratize artificial intelligence through open source and open science.
Nathan
@natfu.be

What do you make of the "reasoning". Is it just going back and trying to correct itself to change the next token or is it more elaborate?

James Harris
@jstevh.bsky.social

Oh cool! Clean minimalist page lightweight on my data usage. Thanks! Plus talking one of my favorite model families. I could fix the joke by the way! Have tested DeepSeekV3 and very excited by new releases. Really appreciated your thoughts.

Bruno Winck
@bruno.winck.org

I wonder where is the limit between thinking and rambling. Reading the <think> I would qualify it as rambling. But jokes can be the next level for a reasoning model :) The SVG prompt is interesting for showing how the model organizes its visual space.

jon
@rouach.net

How would you use 5x H100 GPUs on an Ubuntu with 1TB ? I think I can organize a donation for a few months..

Sandip Bhattacharya
@sandipb.net

Using ur notes, I compared DeepSeek with some of the other models for my "rainbow question". gist.github.com/sandipb/c964... Any idea why DeepSeek tends to summarize the answer compared to the rest of the models? o1 seems to give the more detailed answer but other models have interesting variations

gist.github.comComparison of the rainbow question between DeepSeek R1+llama, Claude, GPT4/O1, Gemini and llama3.2Comparison of the rainbow question between DeepSeek R1+llama, Claude, GPT4/O1, Gemini and llama3.2 - 00 Deepseek-R1+llama.md
OneLLM
@oneai8.bsky.social

Now, OneLLM / OneLLM Pro supports local loading of #DeepSeek #R1 on your phone . Head over to the Model Download Center and get your #DeepSeek #R1 today!

Simon Willison
@simonwillison.net

No idea - I think it might be mainly about model sizes, Llama doesn't have a 32B or 14B model Could relate to licenses too, the Qwen license is Apache 2, the Llama "community license" is janky

Simon Willison
@simonwillison.net

No concerns about privacy at all, these are local models you run in your own machine - you can turn off WiFi or inspect outbound network traffic if you want to be absolutely sure