Learning Joint Articulatory-Acoustic Representations with Normalizing Flows
Learning Joint Articulatory-Acoustic Representations with Normalizing Flows
复制标题
通过标准化流学习联合发音声学表示
DOI：
10.21437/interspeech.2020-2004
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
S. Fels
中科院分区：
文献类型：
--
作者：
Pramit Saha;S. Fels
声道的发音几何配置和所得到的语音声音的声学特性被认为具有很强的因果关系。本文旨在通过可逆神经网络模型找到元音发音域和声域之间的联合潜在表示，同时保留各自的域特定特征。我们的模型利用卷积自动编码器架构和规范化的基于流的模型，以允许正向和反向映射在半监督的方式，中间矢状声道几何形状的两个自由度的发音合成器与1D声波模型和梅尔频谱图表示的合成语音之间。我们的方法在实现发音到声学以及声学到发音映射方面取得了令人满意的性能，从而证明了我们在实现两个域的联合编码方面的成功。
The articulatory geometric configurations of the vocal tract and the acoustic properties of the resultant speech sound are considered to have a strong causal relationship. This paper aims at finding a joint latent representation between the articulatory and acoustic domain for vowel sounds via invertible neural network models, while simultaneously preserving the respective domain-specific features. Our model utilizes a convolutional autoencoder architecture and normalizing flow-based models to allow both forward and inverse mappings in a semi-supervised manner, between the mid-sagittal vocal tract geometry of a two degrees-of-freedom articulatory synthesizer with 1D acoustic wave model and the Mel-spectrogram representation of the synthesized speech sounds. Our approach achieves satisfactory performance in achieving both articulatory-to-acoustic as well as acoustic-to-articulatory mapping, thereby demonstrating our success in achieving a joint encoding of both the domains.
影响因子：
2.4
作者：
Story, BH;Titze, IR;Hoffman, EA
通讯作者：
Hoffman, EA