DiffusionGemma 26B GGUF: Fast Multimodal Generation on Your GPU

DiffusionGemma-26B-A4B-it is a 26‑billion‑parameter multimodal model from Google DeepMind that generates text from interleaved text, image, and video inputs. The unsloth repository provides GGUF‑quantized checkpoints (Q8_0, Q4_K_M, etc.) that run on the DiffusionGemma branch of llama.cpp, enabling local inference on a single GPU with as little as 16 GB VRAM. The model uses a block‑diffusion architecture: it denoises 256‑token canvases in parallel, delivering much higher token‑per‑second throughput than traditional autoregressive LLMs.

The model supports up to a 256 K token context window, a sliding window of 1024 tokens, and a Mixture‑of‑Experts design with 8 active experts out of 128 total, keeping the active parameter count at 3.8 B. It includes built‑in reasoning ("thinking mode"), function‑calling support, and multilingual capabilities for 35+ languages. Vision features cover object detection, OCR, document parsing, UI understanding, and video frame analysis, making it suitable for a wide range of text‑to‑text and image‑to‑text tasks.

Unsloth’s GGUF quantization benchmarks and the integration with Unsloth Studio simplify fine‑tuning and inference: users can load the model with a single command, enable multi‑turn conversation mode, and even visualize the diffusion canvas live. The open‑weights Apache‑2.0 license, high download count (120 k) and recent updates for Studio compatibility have driven its trending status on Hugging Face.