CAREER: From Imaging and Acoustic Data to Articulatory Synthesis

为了更好地理解我们的认知能力，为了开发高质量的语音合成器和自动语音识别系统，需要人类语音产生系统的定量模型。在本研究中，我们建立了摩擦辅音的生成模型。由于摩擦辅音的产生机制复杂，并且缺乏足够的发音和空气动力学数据，因此建模一直是一个具有挑战性的问题。在这项研究中，关节数据来自磁共振图像（MRI）和动态腭电图（EPG），空气动力学数据来自流量和压力测量。MRI显示声道的三维几何形状，而EPG对于研究发音动力学非常重要。气动数据对于研究摩擦生产过程中的湍流产生至关重要。建模方法基于估计理论、声学和信号处理技术，并使用从上述统一测量集获得的数据。虽然研究的重点是摩擦音，但提出的新方法可用于研究所有语音。此外，该研究还促进了电气工程、放射学和语言学的跨学科活动。通过建立参数化语音产生模型和从声学波形推断发音动态，可以实现更好的语音合成器和语音识别系统的性能。

Quantitative models of the human speech production system are needed for a better understanding of our cognitive abilities and for the development of high-quality speech synthesizers and automatic speech recognition systems. In the proposed research, production models for fricative consonants are developed. Modeling fricative consonants has always been a challenging problem because of the complex production mechanisms and the lack of sufficient articulatory and aerodynamic data for these sounds. In this study, articulatory data are obtained from Magnetic Resonance Images (MRI) and Dynamic Electropalatography (EPG), and aerodynamic data, from flow and pressure measurements. MRI reveals the 3D geometry of the vocal tract while EPG is important for studying articulatory dynamics. Aerodynamic data are crucial for studying turbulence generation during fricative production. The modeling approach is based on estimation theory, acoustics, and signal-processing techniques and uses the data obtained from the unified set of measurements described above. Although the study focuses on fricatives, the proposed novel approach can be used to study all speech sounds. In addition, the study fosters cross-disciplinary activities in Electrical Engineering, Radiology, and Linguistics. By developing parametric speech production models and by inferring articulatory dynamics from the acoustic waveform, better performance of speech synthesizers and speech recognition systems can be achieved.