Gemma‑4 26B A4B: Open‑Weight Multimodal MoE Model for Image‑Text Reasoning

Google DeepMind’s Gemma‑4 family expands with the 26B A4B mixture‑of‑experts (MoE) model, released under an Apache‑2.0 license. Identified on Hugging Face as `google/gemma-4-26B-A4B-it` and tagged for the `image-text-to-text` pipeline, this model accepts both visual and textual inputs and generates text responses. It is built with the Transformers library, distributed as safetensors, and has quickly attracted attention with over 24 000 downloads and 306 likes, reflecting its strong community interest.

The A4B variant combines a total of 25.2 B parameters with an active subset of 3.8 B during inference, delivering fast inference while preserving the capacity of a larger model. It supports a 256 K token context window, multilingual understanding in more than 140 languages, and native system‑prompt handling. Its multimodal capabilities cover image and video inputs (audio is not supported in this size), enabling tasks such as object detection, OCR, document parsing, chart comprehension, and video description. The model also includes built‑in reasoning ("thinking" mode), function‑calling support, and strong coding performance, making it suitable for agentic workflows.

Gemma‑4’s architecture blends local sliding‑window attention with global attention, using proportional RoPE and unified keys/values to keep memory usage low for long contexts. The README provides ready‑to‑use code snippets for loading the model with `AutoProcessor` and `AutoModelForMultimodalLM`, as well as best‑practice sampling settings (temperature 1.0, top_p 0.95, top_k 64). These resources make it straightforward for developers to integrate the model into applications that require sophisticated multimodal reasoning and generation.