Tencent's open speech model AuK folds a whole toolchain into a single instruction
AuK does text-to-speech, denoising, music separation, emotion and timbre editing in one model, driven by a plain-language instruction. 1.5B parameters, MIT licence, weights published.
Tencent's Hunyuan team has released AuK, a speech model, as open source. The weights are on Hugging Face and ModelScope, the inference code is on GitHub, and the licence is MIT, so commercial use, modification and redistribution are all permitted.
The model is 1.5 billion parameters. By today's standards that is small, and part of the story sits right there: the list of things it does is out of proportion to its size.
One model, six separate jobs
Anyone working with audio has until now assembled a separate tool for each job: one for text-to-speech, another for noise removal, a third to pull vocals out of a track, a fourth for voice conversion. AuK puts all of them behind a single instruction interface. You tell the model in plain language what you want, optionally hand it a reference clip, and get edited or generated audio back.
The repository groups the tasks under five headings:
- Generation: zero-shot text-to-speech and instruction-guided speech
- Content editing: rewriting what is said, and lyric editing
- Acoustic editing: pitch, speed and volume
- Paralinguistic editing: emotion, timbre, de-accenting, nonverbal sounds, conversion to whisper
- Enhancement and separation: denoising, dereverberation, speaker and music separation
Architecturally, a multimodal language model reads the instruction together with any reference audio, a 50 Hz VAE supplies the acoustic latents, and a hybrid transformer fuses the two streams under a flow-matching objective.
A faster variant shipped alongside it
Next to AuK sits AuK-Flash, a distilled version that runs inference in four steps without classifier-free guidance. The team measures it at roughly 4.5 times the wall-clock speed of the full model under matched conditions. The distillation runs in two stages: a trajectory-level consistency initialisation produces a few-step student, which is then trained against a task-routing objective, with separation additionally supervised by clean-prediction regression.
Where to stop short
Neither the announcement nor the repository says which languages it supports. For a speech model that is not a footnote: a model that performs well in English and Chinese can fall apart in a language it was barely trained on. Anyone considering it should test in their own language before deciding.
GPU requirements are not stated either. One limit is spelled out, though, for the ComfyUI integration: source plus target must stay within 30 seconds, so longer recordings have to be processed in pieces.
What changes the picture nonetheless is the licence and the size. A 1.5-billion-parameter model under MIT, with weights in your hands, brings work that has mostly been sold by subscription into reach locally, without sending audio anywhere. The technical report is on arXiv.
