UNITE-AUDIO: Joint learning of continuous tokenization and
latent flow matching for text-to-audio generation

Runwu Shi1GS, Kai Li2GS, Yujin Wang3, Dong Yang4GS, Jiahui Li1, Jiang Wang1, Chang Zeng4, Benjamin Yen1, Ashizawa Takeshi1, Chunxiang Jin5, Kazuhiro Nakadai1GS

1Institute of Science Tokyo 2Tsinghua University 3Wuhan University 4The University of Tokyo 5Ant Group

arXiv GitHub Hugging Face

Method

UNITE-AUDIO jointly learns continuous tokenization and latent flow matching
Overview of UNITE-AUDIO: continuous tokenization and latent flow matching are jointly learned in a single stage through noisy partial-context prediction, balancing semantic fidelity and reconstruction quality.

Audio Samples