Qwen3.5‑397B‑A17B: Ultra‑Large Multimodal Model Redefines Vision‑Language AI

Qwen3.5‑397B‑A17B is a next‑generation multimodal language model released by the Alibaba‑Qwen team. It is an image‑to‑text (image‑text‑to‑text) model built on a causal decoder architecture and equipped with a dedicated vision encoder. The model contains a total of 397 billion parameters, but thanks to a Mixture‑of‑Experts (MoE) design only 17 billion of them are active per inference step, allowing the system to deliver the performance of a massive LLM while keeping inference costs manageable. It is fully compatible with the HuggingFace Transformers library and can be served through vLLM, SGLang, KTransformers, and other high‑throughput inference stacks.

The Qwen3.5‑397B‑A17B model employs early‑fusion training, where visual tokens are injected into the language stream at the very first layer, enabling tight cross‑modal reasoning. It supports 201 languages, offers a native context window of 262 k tokens and can be extended to a staggering 1 million‑token context, making it suitable for long‑form tasks such as document analysis, code review, or research paper summarisation. Its architecture combines Gated‑Linear Units, Gated‑Attention Units, and a hierarchical MoE with 10 active experts per token, delivering strong performance across a wide range of benchmarks, including reasoning (LiveCodeBench v6 87.7 % accuracy), STEM (GPQA 88.4 %), long‑context (LongBench v2 63.2 %), and general agent tasks.

Benchmark results show that Qwen3.5‑397B‑A17B is competitive with leading proprietary models. It attains scores of 99.4 % on the HMMT February 2024 reasoning benchmark and reaches 100 % on the HMMT November 2023 test, while also delivering solid long‑context abilities (AA‑LCR 72.7 %). Its multilingual capability is highlighted by strong scores on C-Eval and other language‑agnostic evaluations, confirming that the model can handle diverse linguistic inputs without sacrificing visual understanding.

The repository provides Apache‑2.0 licensed weights, a ready‑to‑use Transformers checkpoint, and guidance for deployment on cloud services such as Alibaba Cloud. For users who need a hosted solution, the Qwen Plus API offers tool‑augmented conversational abilities, and a lightweight Qwen1.5‑14B‑Chat model is also available for cost‑effective experimentation. With its blend of massive scale, efficient MoE activation, and ultra‑long context support, Qwen3.5‑397B‑A17B opens new possibilities for building sophisticated vision‑language applications.