Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages
Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages
复制标题
DOI：
10.48550/arxiv.2303.01037
复制
发表时间：
2023-03
期刊：
影响因子：
--
通讯作者：
Yu Zhang;Wei Han;James Qin;Yongqiang Wang;Ankur Bapna;Zhehuai Chen;Nanxin Chen;Bo Li;Vera Axelrod;Gary Wang;Zhong Meng;Ke Hu;A. Rosenberg;Rohit Prabhavalkar;Daniel S. Park;Parisa Haghani;Jason Riesa;Ginger Perng;H. Soltau;Trevor Strohman;B. Ramabhadran;Tara N. Sainath;P. Moreno;Chung-Cheng Chiu;J. Schalkwyk;Franccoise Beaufays;Yonghui Wu
中科院分区：
文献类型：
--
作者：
Yu Zhang;Wei Han;James Qin;Yongqiang Wang;Ankur Bapna;Zhehuai Chen;Nanxin Chen;Bo Li;Vera Axelrod;Gary Wang;Zhong Meng;Ke Hu;A. Rosenberg;Rohit Prabhavalkar;Daniel S. Park;Parisa Haghani;Jason Riesa;Ginger Perng;H. Soltau;Trevor Strohman;B. Ramabhadran;Tara N. Sainath;P. Moreno;Chung-Cheng Chiu;J. Schalkwyk;Franccoise Beaufays;Yonghui Wu
我们介绍了通用语音模型（USM），这是一个单一的大型模型，可以跨100多种语言执行自动语音识别（ASR）。这是通过在跨越300多种语言的1200万(M)小时的大型未标记多语言数据集上预训练模型的编码器，并在较小的标记数据集上进行微调来实现的。我们使用随机投影量化和语音-文本模态匹配的多语言预训练来实现下游多语言ASR和语音到文本翻译任务的最先进性能。我们还证明，尽管使用的标记训练集的大小是Whisper模型的1/7，但我们的模型在许多语言的域内和域外语音识别任务上都表现出相当或更好的性能。
We introduce the Universal Speech Model (USM), a single large model that performs automatic speech recognition (ASR) across 100+ languages. This is achieved by pre-training the encoder of the model on a large unlabeled multilingual dataset of 12 million (M) hours spanning over 300 languages, and fine-tuning on a smaller labeled dataset. We use multilingual pre-training with random-projection quantization and speech-text modality matching to achieve state-of-the-art performance on downstream multilingual ASR and speech-to-text translation tasks. We also demonstrate that despite using a labeled training set 1/7-th the size of that used for the Whisper model, our model exhibits comparable or better performance on both in-domain and out-of-domain speech recognition tasks across many languages.