CAREER: machine learning approches for articulatory inversion

发音反转是指在一个给定的发音过程中，如何恢复声道形状的顺序。发音表征对于自动语音识别、语音产生研究、语言治疗和语言学习都是非常有用的。发音反转是一个困难的问题，因为不同的声道形状可以产生相同的声学效果，但发音轨迹必须遵守人类声道的机械约束。序列上的反演问题的其他例子，共享映射的多值性质和约束的存在，是：与语音发声相关联的面部手势的恢复;机器人的逆运动学;和从视频中恢复3D运动。该项目从机器学习的角度，基于PI引入的框架，接近发音反演。低维流形在关节声学空间表示的概率的方式由密度模型估计数据（记录使用麦克风和电磁articulography）。多值映射明确地由该密度的条件分布的模式表示，并且使用连续性约束来消除发音轨迹的歧义。该项目引入了降维、密度估计和正则化（例如多值回归和噪声数据的图学习）方面的新问题，以及新的模型和算法。这项工作的预期结果是：在机器学习中进行基础研究，并将映射反演问题引入研究和教育;改进发音反演（代码将免费提供）;并在语音生成研究和教育中倡导数据驱动的方法。

Articulatory inversion is the problem of recovering the sequence ofvocal tract shapes that produce a given acoustic utterance. Articulatory representations are useful for automatic speechrecognition, speech production research, language therapy, andlanguage learning. Articulatory inversion is a hard problem becausedifferent vocal tract shapes can produce the same acoustics, yet thearticulatory trajectory must obey the mechanical constraints of thehuman vocal tract. Other examples of inversion problems over asequence, which share the multivalued nature of the mappings and theexistence of constraints, are: the recovery of facial gesturesassociated with a speech utterance; the inverse kinematics of a robotarm; and the recovery of 3D motion from video.This project approaches articulatory inversion from a machine learningstandpoint, based on a framework introduced by the PI. Thelow-dimensional manifold in articulatory-acoustic space is representedin a probabilistic way by a density model estimated from data(recorded using a microphone and electromagnetic articulography). Multivalued mappings are explicitly represented by the modes ofconditional distributions of this density, and the articulatorytrajectory is disambiguated using a continuity constraint.The project introduces new problems in dimensionality reduction,density estimation and regularization (such as multivalued regressionand graph-learning from noisy data), and new models and algorithms. The expected results of this work are: performing basic research inmachine learning, and introducing mapping inversion problems toresearch and education; improving articulatory inversion (for whichcode will be made freely available); and advocating data-drivenapproaches in speech production research and education.