CAREER: machine learning approches for articulatory inversion

点击翻译按钮获取中文摘要

Articulatory inversion is the problem of recovering the sequence ofvocal tract shapes that produce a given acoustic utterance. Articulatory representations are useful for automatic speechrecognition, speech production research, language therapy, andlanguage learning. Articulatory inversion is a hard problem becausedifferent vocal tract shapes can produce the same acoustics, yet thearticulatory trajectory must obey the mechanical constraints of thehuman vocal tract. Other examples of inversion problems over asequence, which share the multivalued nature of the mappings and theexistence of constraints, are: the recovery of facial gesturesassociated with a speech utterance; the inverse kinematics of a robotarm; and the recovery of 3D motion from video.This project approaches articulatory inversion from a machine learningstandpoint, based on a framework introduced by the PI. Thelow-dimensional manifold in articulatory-acoustic space is representedin a probabilistic way by a density model estimated from data(recorded using a microphone and electromagnetic articulography). Multivalued mappings are explicitly represented by the modes ofconditional distributions of this density, and the articulatorytrajectory is disambiguated using a continuity constraint.The project introduces new problems in dimensionality reduction,density estimation and regularization (such as multivalued regressionand graph-learning from noisy data), and new models and algorithms. The expected results of this work are: performing basic research inmachine learning, and introducing mapping inversion problems toresearch and education; improving articulatory inversion (for whichcode will be made freely available); and advocating data-drivenapproaches in speech production research and education.