CAREER: machine learning approches for articulatory inversion

发音反转是恢复产生给定声音发声的声道形状序列的问题。发音表征在自动语音识别、语音生成研究、语言治疗和语言学习中非常有用。发音翻转是一个难题，因为不同的声道形状可以产生相同的声学效果，但发音轨迹必须服从人类声道的机械约束。序列上的反转问题的其他例子包括：与语音发声相关的面部手势的恢复；Robotarm的逆运动学；以及从视频中恢复3D运动。该项目从机器学习的角度，基于PI引入的框架，从机器学习的角度来处理发音反转。关节-声学空间中的低维流形由由数据(使用麦克风和电磁关节成像术记录)估计的密度模型以概率的方式表示。多值映射被显式地表示为该密度的条件分布模式，关节轨迹使用连续约束来消除歧义。该项目在降维、密度估计和正则化(如多值回归和噪声数据中的图学习)方面引入了新的问题，以及新的模型和算法。这项工作的预期结果是：开展机器学习的基础研究，并将映射倒置问题引入研究和教育；改进发音倒置(其代码将免费提供)；在语音产生研究和教育中倡导数据驱动的方法。

Articulatory inversion is the problem of recovering the sequence ofvocal tract shapes that produce a given acoustic utterance. Articulatory representations are useful for automatic speechrecognition, speech production research, language therapy, andlanguage learning. Articulatory inversion is a hard problem becausedifferent vocal tract shapes can produce the same acoustics, yet thearticulatory trajectory must obey the mechanical constraints of thehuman vocal tract. Other examples of inversion problems over asequence, which share the multivalued nature of the mappings and theexistence of constraints, are: the recovery of facial gesturesassociated with a speech utterance; the inverse kinematics of a robotarm; and the recovery of 3D motion from video.This project approaches articulatory inversion from a machine learningstandpoint, based on a framework introduced by the PI. Thelow-dimensional manifold in articulatory-acoustic space is representedin a probabilistic way by a density model estimated from data(recorded using a microphone and electromagnetic articulography). Multivalued mappings are explicitly represented by the modes ofconditional distributions of this density, and the articulatorytrajectory is disambiguated using a continuity constraint.The project introduces new problems in dimensionality reduction,density estimation and regularization (such as multivalued regressionand graph-learning from noisy data), and new models and algorithms. The expected results of this work are: performing basic research inmachine learning, and introducing mapping inversion problems toresearch and education; improving articulatory inversion (for whichcode will be made freely available); and advocating data-drivenapproaches in speech production research and education.