Post

Turns out we weren't done for major LLM releases in 2024 after all... Alibaba's Qwen just released QvQ, a "visual reasoning model" - the same chain-of-thought trick as OpenAI's o1 applied to running a prompt against an image Trying it out is a lot of fun: simonwillison.net/2024/Dec/24/...

Here are all of experiments with full transcripts gist.github.com/simonw/6c296...

Ran it on my M2 64GB! uv run --with 'numpy<2.0' --with mlx-vlm python \ -m mlx_vlm.generate \ --model mlx-community/QVQ-72B-Preview-4bit \ --max-tokens 10000 \ --temp 0.0 \ --prompt "describe this" \ --image pelicans-on-bicycles-veo2.jpg simonwillison.net/2024/Dec/24/...

... and since the year still isn't over, DeepSeek published a new 685B 700+GB model on Hugging Face, no documentation yet, official announcement looks like it will come tomorrow huggingface.co/deepseek-ai/...

love it that you gave it one of the ARC-AGI puzzles that o3 had failed at :)

I suspected the model can't detect that many visual details accurately and tried to test that. I took the puzzle image from the transcripts above and asked it to convert the first example input and output to ascii or json. The results were both disappointing and surprisingly good.