Ant Group open-sources speech models small enough to run locally

Type a plain-text description of a voice, and a speech model that fits on a local machine can produce it. That is the bet behind Ming-3.0-tts, released by Ant Group's inclusionAI team: two open speech models, with code, weights and evaluation tables published on GitHub, Hugging Face and ModelScope under the MIT license.

The family has two members. Ming-3.0-tts-0.5b is a dense model of about half a billion parameters. Ming-3.0-tts-tiny is a mixture-of-experts model, an architecture where only part of the network works on each piece of text: it has 7.9 billion parameters in total, of which 1.3 billion are active per token. Both share one framework. Text and task-specific control tokens drive a language model, which steers a DiT (diffusion transformer) patch head and an audio tokenizer, the component that turns the generated sound into a compact sequence the model can work with.

The feature list is wide. A user can clone a reference voice, design a new one from a description, or pick from more than 100 built-in Chinese voices and five English ones. Delivery can be steered with natural-language instructions, and emotion, speed, pitch and volume can be set directly. Fourteen inline tags insert laughter, sighs and other vocal sounds in Chinese or English. Supported languages are Chinese, English, German, Spanish, French, Korean and Japanese, plus Sichuanese and Cantonese. The tiny model adds multi-speaker dialogue generated from a script and speaker references, aimed at long-form, podcast-style audio. The team stresses that these capabilities run as separate task modes, not as arbitrary combinations.

The headline results come from the project's own repository. Ming-3.0-tts ranked first among open-weights models on the Artificial Analysis Controlled Voice leaderboard, and first overall on Pronunciation Robustness, with 93.2% of highlighted spans judged correctly pronounced. On open benchmarks, the team reports state-of-the-art results among listed open-source models on all six NVBench single-tag paralinguistic metrics, the sounds beyond words such as laughter or sighs, along with two language-specific scores. The team itself warns that these rankings shift as new models are added.

The README is less flattering in places, and that matters for anyone who wants to use the models. Speaker similarity in the newly added languages mostly misses the top three. Pitch control is the least accurate attribute. The tiny model's Korean error rate trails the smaller model by a wide margin, and podcast audio quality still sits below an earlier Ming-omni-tts release. Emotion accuracy drops when the text carries no emotional cue of its own.

For developers and small teams, the change is that voice generation can run on hardware they control, under a permissive license, without a paid service in the loop. The roadmap lists vLLM-Omni accelerated inference and a technical report as next steps; neither is described as delivered yet. The project asks users not to clone real people's voices without consent, and to label synthetic speech where it could be mistaken for a real recording.