Gemma4‑12B‑Coder GGUF: Tiny Local Python Coding Assistant

Gemma4-12B-Coder (GGUF) is a community‑fine‑tuned version of Google’s Gemma‑4 12B IT model, focused on verifiable Python code generation. The model was distilled from two chain‑of‑thought (CoT) sources—real CoT traces from Composer 2.5 and synthetic “second‑attempt” traces from Fable 5—where each solution was executed against deterministic tests and only passing examples were kept. This training pipeline yields a model that first reasons about a problem (edge cases, complexity, approach) and then emits clean, runnable Python code.

The model is distributed in GGUF quantized formats (Q2_K, Q4_K_M, Q6_K, Q8_0), ranging from a 4.5 GB footprint that fits on ~8 GB VRAM to near‑lossless 11.8 GB versions. It can be run locally with llama.cpp (or one‑click apps like LM Studio, Jan, Ollama) and supports up to 131 K tokens of context, with guidance on VRAM‑based context limits. Because the training data is task‑oriented and lacks safety‑aligned filtering, the model refuses less often than the base model, so users should add their own guardrails for production use.

Designed for algorithmic and function‑level Python tasks, Gemma4‑Coder excels at reasoning‑heavy coding prompts and can be used offline, without any API or cloud dependency. Its small memory requirements make it accessible to developers with modest GPUs or even integrated graphics, opening the door for private, on‑device coding assistants.

The repository notes a potential v2 that would lean more heavily on Composer 2.5 data and possibly incorporate GLM‑5.2 as an additional teacher, pending community interest. For now, the v1 release offers a practical, locally runnable coding model that blends reasoning and execution verification in a compact package.