Multilingual Acoustic Word Embedding Models for Processing Zero-resource Languages
Multilingual Acoustic Word Embedding Models for Processing Zero-resource Languages
复制标题
DOI：
10.1109/icassp40776.2020.9054202
复制
发表时间：
2020-02
期刊：
影响因子：
--
通讯作者：
H. Kamper;Yevgen Matusevych;S. Goldwater
中科院分区：
文献类型：
--
作者：
H. Kamper;Yevgen Matusevych;S. Goldwater
声学词嵌入是可变长度语音段的固定维度表示。在未标记语音是唯一可用资源的设置中，这种嵌入可以用于“零资源”语音搜索、索引和发现系统。在这里，我们建议在来自多个资源丰富的语言的标记数据上训练一个监督嵌入模型，然后将其应用于看不见的零资源语言。对于这种迁移学习方法，我们考虑了两种多语言递归神经网络模型：一种是在所有训练语言的联合词汇上训练的判别分类器，另一种是训练用于重建单词对的对应自动编码器。我们使用六个目标零资源语言的单词歧视任务来测试这些。当在七种资源丰富的语言上训练时，两种模型的表现相似，并且优于在零资源语言上训练的无监督模型。在只有一种训练语言的情况下，第二种模型工作得更好，但性能更多地取决于特定的训练-测试语言对。
Acoustic word embeddings are fixed-dimensional representations of variable-length speech segments. In settings where unlabelled speech is the only available resource, such embeddings can be used in "zero-resource" speech search, indexing and discovery systems. Here we propose to train a single supervised embedding model on labelled data from multiple well-resourced languages and then apply it to unseen zero-resource languages. For this transfer learning approach, we consider two multilingual recurrent neural network models: a discriminative classifier trained on the joint vocabularies of all training languages, and a correspondence autoencoder trained to reconstruct word pairs. We test these using a word discrimination task on six target zero-resource languages. When trained on seven well-resourced languages, both models perform similarly and outperform unsupervised models trained on the zero-resource languages. With just a single training language, the second model works better, but performance depends more on the particular training–testing language pair.