Xuan Son Nguyen

llama.cpp now supports various small OCR models that can run on low-end devices. These models are small enough to run on GPU with 4GB VRAM, and some of them can even run on CPU with decent performance. In this post, I will show you how to use these OCR models with llama.cpp 👇

Very nice touch, Gmail 😅

Gemma 3n has arrived in llama.cpp 👨‍🍳 🍰 Comes in 2 flavors: E2B and E4B (E means "effective/active parameters")

See you this Sunday at AI Plumbers conference: 2nd edition! 📍 Where: GLS Event Campus Berlin, Kastanienallee 82 | 10435 Berlin 👉 Register here: lu.ma/vqx423ct

✨✨ AIFoundry is bringing you the AI Plumbers Conference: 2nd edition — an open source meetup for low-level AI builders to dive deep into "the plumbing" of modern AI 📍 Where: GLS Event Campus Berlin, Kastanienallee 82 | 10435 Berlin 📅 When: June 15, 2025 👉 Register now: lu.ma/vqx423ct

Hugging Face Inference Endpoints now officially support deploying **vision** models via llama.cpp 👀 👀 Try it now: endpoints.huggingface.co/catalog

Although we have A100, H200, M3 Ultra, etc Still can't match the power of that Casio FX 😆

llama.cpp vision support just got much better! 🚀 Traditionally, models with complicated chat template like MiniCPM-V or Gemma 3 requires a dedicated binary to run. Now, you can use all supported models via a "llama-mtmd-cli" 🔥 (Only Qwen2VL is not yet supported)

Someone at Google definitely had a lot of fun making this 😆 And if you don't know, it's available in "Starter apps" section on AI Studio. The app is called "Gemini 95"

Telling LLM memory requirement WITHOUT a calculator? Just use your good old human brain 🧠 😎 Check out my 3‑step estimation 🚀

Google having a quite good sense of humor 😂 Joke aside, 1B model quantized to Q4 without performance degrading is sweet 🤏

No vibe coding. Just code it ✅ Visit my website --> ngxson.com

Had a fantastic chat today with Georgi Gerganov, the brilliant mind behind ggml, llama.cpp, and whisper.cpp! We discussed about: 🚀 The integration of vision models into llama.cpp 🚀 The challenges of maintaining a smooth UX/DX 🚀 The exciting future of llama.cpp Big things ahead - stay tuned!

OK now you are the best, Gememe 2.0

Did you know? A number of 🤗 Hugging Face's blog posts now feature AI-created podcasts 🎙️ This offers an alternative way to absorb extensive and intricate articles 🔍

Qwen/QwQ-32B has just arrived on Hugging Chat! Try it now: huggingface.co/chat/models/...

CogView-4 is out 🔥🚀 The SoTa OPEN text to image model by ZhipuAI Demo: huggingface.co/spaces/THUDM... ✨ 6B with Apache2.0 ✨ Supports Chinese & English Prompts by ANY length ✨ Generate Chinese characters within images ✨ Creates images at any resolution within a given range

Wondering how much RAM is needed to run a given GGUF? Try: npx @huggingface/gguf [model].gguf This also work with remote file, for example: npx @huggingface/gguf https: //huggingface.co/bartowski/Qwen_QwQ-32B-GGUF/resolve/main/Qwen_QwQ-32B-Q4_K_M.gguf

Apple unveils the M3 Ultra chip, support up to 512GB unified money, oops sorry, unified memory. Perfect for pro workflows and AI development 👀 Read more: www.apple.com/newsroom/202...

Baby wake up! The Hugging Face Reasoning Course is out 🚀 Huge thanks to Maxime Labonne for building the first practical example in the reasoning course. Link: huggingface.co/reasoning-co...

With the new 🐸 JFrog 's model scanner on the 🤗 Hugging Face hub, we're making running AI models even more secured for everyone!

Disassemble Phi-4-multimodal-instruct: > Minimal Vision encoder: 440M/460M respectively > Projector: 2-layer MLP for both modalities > Language model: Phi-4-mini 3.3B parameters > LoRA adapters for Vision/Audio decoder, applied on top of Phi-4-mini