Unsloth Gemma‑4 12B QAT GGUF: Fast, Multimodal Any‑to‑Any Model

The **unsloth/gemma-4-12B-it-qat-GGUF** model is a Quantization‑Aware Training (QAT) checkpoint of Google DeepMind's Gemma‑4 12B instruction‑tuned model, packaged by Unsloth in the GGUF format. It retains the quality of the original bfloat16 weights while shrinking memory usage, making the 12‑billion‑parameter model runnable on consumer‑grade hardware. The repository ships both the main model and a Multi‑Token Prediction (MTP) drafter (`mtp-gemma-4-12B-it.gguf`), which enables near‑lossless speculative decoding for faster inference with llama.cpp‑compatible servers.

Built on the **transformers** library and tagged with `pipeline_tag: any-to-any`, the model can accept and generate across multiple modalities: text, images, video, and (for the 12B size) audio. It supports a context window of up to **256 K tokens**, native system‑prompt handling, built‑in reasoning (thinking) mode, function‑calling for tool use, and multilingual capabilities covering more than 140 languages. The README highlights integration with Unsloth Studio, MTP support, and extensive documentation for multimodal usage.

Gemma‑4’s architecture combines dense and Mixture‑of‑Experts designs, offering strong performance on reasoning, coding, and long‑context tasks. This GGUF release is trending because it blends state‑of‑the‑art multimodal abilities with a lightweight, deploy‑ready format, enabling developers to run high‑capacity models locally or in edge environments without sacrificing quality.

The model is licensed under Apache 2.0, originates from Google DeepMind, and is maintained by the Unsloth community, which provides benchmarks, Discord support, and detailed guides for running, fine‑tuning, and leveraging the MTP drafter.