CAREER: machine learning approches for articulatory inversion

发音反转是恢复声道形状序列的问题，产生一个给定的声学话语。发音表征对自动语音识别、语音产生研究、语言治疗和语言学习都很有用。发音反转是一个难题，因为不同的声道形状可以产生相同的声学效果，但发音轨迹必须服从人类声道的机械约束。序列上反转问题的其他例子，它们共享映射的多值性质和约束的存在，包括：与语音话语相关的面部手势的恢复；机器人臂的逆运动学；以及从视频中恢复3D运动。该项目基于PI引入的框架，从机器学习的角度研究发音反转。发音声学空间中的低维流形以概率方式由数据估计的密度模型表示（使用麦克风和电磁发音记录仪记录）。多值映射由该密度的条件分布模式显式表示，并使用连续性约束消除了铰接轨迹的歧义。该项目引入了降维、密度估计和正则化（如多值回归和噪声数据的图学习）方面的新问题，以及新的模型和算法。这项工作的预期结果是：进行机器学习的基础研究，并将映射反演问题引入研究和教育；改善发音反转（代码将免费提供）；并倡导在语音生产研究和教育中采用数据驱动的方法。

Articulatory inversion is the problem of recovering the sequence ofvocal tract shapes that produce a given acoustic utterance. Articulatory representations are useful for automatic speechrecognition, speech production research, language therapy, andlanguage learning. Articulatory inversion is a hard problem becausedifferent vocal tract shapes can produce the same acoustics, yet thearticulatory trajectory must obey the mechanical constraints of thehuman vocal tract. Other examples of inversion problems over asequence, which share the multivalued nature of the mappings and theexistence of constraints, are: the recovery of facial gesturesassociated with a speech utterance; the inverse kinematics of a robotarm; and the recovery of 3D motion from video.This project approaches articulatory inversion from a machine learningstandpoint, based on a framework introduced by the PI. Thelow-dimensional manifold in articulatory-acoustic space is representedin a probabilistic way by a density model estimated from data(recorded using a microphone and electromagnetic articulography). Multivalued mappings are explicitly represented by the modes ofconditional distributions of this density, and the articulatorytrajectory is disambiguated using a continuity constraint.The project introduces new problems in dimensionality reduction,density estimation and regularization (such as multivalued regressionand graph-learning from noisy data), and new models and algorithms. The expected results of this work are: performing basic research inmachine learning, and introducing mapping inversion problems toresearch and education; improving articulatory inversion (for whichcode will be made freely available); and advocating data-drivenapproaches in speech production research and education.