Speaker-Independent Acoustic-to-Articulatory Speech Inversion
Speaker-Independent Acoustic-to-Articulatory Speech Inversion
复制标题
DOI：
10.1109/icassp49357.2023.10096796
复制
发表时间：
2023-02
期刊：
影响因子：
--
通讯作者：
Peter Wu;Li-Wei Chen;Cheol Jun Cho;Shinji Watanabe;L. Goldstein;A. Black;G. Anumanchipalli
中科院分区：
文献类型：
--
作者：
Peter Wu;Li-Wei Chen;Cheol Jun Cho;Shinji Watanabe;L. Goldstein;A. Black;G. Anumanchipalli
为了建立能够像人类一样自然地处理语音的语音处理方法，研究人员探索了多种方法来建立从语音到可解释空间的可逆映射。发音空间是一个很有前途的反转目标，因为这个空间捕获了语音产生的机制。为此，我们建立了一个声学-发音反转（AAI）模型，该模型利用自回归、对抗训练和自我监督来推广到看不见的说话者。我们的方法在电磁关节成像（EMA）数据集上获得了0.784的相关性，将最先进的技术提高了12.5%。此外，我们通过直接比较估计表征的行为与语音产生行为来展示这些表征的可解释性。最后，我们提出了一种不依赖发音标签的基于重新合成的AAI评估指标，用18个说话人的数据集证明了它的有效性。
To build speech processing methods that can handle speech as naturally as humans, researchers have explored multiple ways of building an invertible mapping from speech to an interpretable space. The articulatory space is a promising inversion target, since this space captures the mechanics of speech production. To this end, we build an acoustic-to-articulatory inversion (AAI) model that leverages autoregression, adversarial training, and self supervision to generalize to unseen speakers. Our approach obtains 0.784 correlation on an electromagnetic articulography (EMA) dataset, improving the state-of-the-art by 12.5%. Additionally, we show the interpretability of these representations through directly com-paring the behavior of estimated representations with speech production behavior. Finally, we propose a resynthesis-based AAI evaluation metric that does not rely on articulatory labels, demonstrating its efficacy with an 18-speaker dataset.