Post

Gemma 4 quantization-aware training (QAT) models are now available, bringing AI performance directly to edge devices and consumer GPUs. These checkpoints are optimized with quantization-aware training to dramatically reduce memory requirements and unlock high-speed local inference. 🧵

Instead of traditional post-training quantization, these models employ custom loss functions and targeted fine-tuning to minimize precision error. This approach delivers a massive reduction in disk and memory footprint while fully preserving the exceptional quality of Gemma 4.

Ecosystem integrations are live today across popular developer tools including Hugging Face, Llama.cpp, Ollama, MLX, LM Studio, NVIDIA, vLLM, Unsloth, and LiteRT-LM.