Fish Audio S2 Pro – Multilingual TTS with Fine‑Grained Inline Control

Fish Audio S2 Pro is a state‑of‑the‑art text‑to‑speech (TTS) model released by the Fish Audio research team. It supports more than 80 languages, including tier‑1 coverage for English, Chinese, and Japanese, and offers free‑form, inline prosody and emotion control via a simple ``[tag]`` syntax. The model is built on a decoder‑only transformer paired with a residual‑vector‑quantized (RVQ) audio codec and a dual‑autoregressive architecture: a 4 B‑parameter “slow” AR branch predicts the primary semantic codebook, while a 400 M‑parameter “fast” AR branch fills in nine residual codebooks to produce high‑fidelity audio. This asymmetric design enables low‑latency streaming—real‑time factor of 0.195 on a single NVIDIA H200 GPU—and efficient serving using SGLang’s continuous batching and KV‑cache optimizations.

The release includes model weights, fine‑tuning scripts, and an SGLang‑based streaming inference engine, making it ready for both research and production deployments. Users can embed expressive instructions directly in the input text, such as ``[whisper]``, ``[laughing]`` or custom phrases like ``[pitch up]``, allowing word‑level modulation of tone, volume, and emotion without predefined tag sets. The model is distributed under the Fish Audio Research License, permitting free research and non‑commercial use, while commercial usage requires a separate agreement. Its combination of multilingual coverage, granular control, and streaming performance has made it a trending model on Hugging Face.