3D Articulatory Speech Synthesis using Biomechanical Models of the Oral, Pharyngeal and Laryngeal Complex

多年来，传统的基于共振峰和声学的语音合成技术在很大程度上掩盖了发音合成的研究。虽然在某些领域取得了成功，但它们仍然无法从文本中产生自然的语音。相比之下，发音语音合成在工业界和学术界的兴趣中继续稳步发展，现在正准备提供必要的平台来克服语音处理中的基本问题，我相信，它代表了语音合成技术的下一个重大进展。作为我长期研究计划的一部分，我一直在建立口腔，咽和喉（OPAL）复合体的计算机建模和模拟，作为许多学科的关键和核心工具，包括工程，语言学，教育，娱乐，解剖学，生理学和医学。通过这些努力，现在可以获得在发音语音合成方面取得重大进展所需的基础设施。因此，我建议开发OPAL复杂的模型，包括航空声学模拟，以创建下一代的验证，发音语音合成。这项研究建立在详细的生物力学模型的人的上呼吸道，已被用来研究的力学语言行为，吞咽和咀嚼沿着与快速，准确耦合有限元和刚体模拟所需的工具。主要研究活动有：1.整合模型几何形状和生物力学，以创建能够进行发音语音合成的OPAL复合体的完整生物力学模型; 2.通过利用[Birkholz等人，2006]的工作，改进现有的1D Navier Stokes方法，以提供声道滤波器的喉部激励之间的改进的耦合; 3.发展了基于流体模拟和基于CFD近似的声道噪声源的三维气动声学模拟; 4.通过扩展[Stavness et al，2013]的工作，创建用于驱动语音清晰度的逆方法，以包括支持从语音和声学目标的图像数据或电磁关节成像（EMA）向后传播运动学目标，通过航空声学模型，基于逆动力学技术推导肌肉激活。语音清晰度约束将用于帮助更好地确定系统。5.创建先进的耦合声带模型，将在紧张和气流下振荡。我们准备在发音语音合成方面取得实质性进展。过去十年在创建医学成像技术、语音测量、OPAL复合体的复杂3D生物力学模型以及用于快速、准确耦合FEM/刚体模拟的工具方面的努力为该项目提供了必要的基础设施。加拿大站在支持下一波应用的语音合成技术的最前沿。将发音语音合成纳入我们的开源工具包将支持语音研究的国际努力，从对人类语音产生的更深入理解中贡献深刻的新知识。

For years, traditional formant-based and acoustic-based speech synthesis techniques have largely overshadowed articulatory synthesis research. While successful in some domains, they still cannot produce natural sounding speech from text. Articulatory speech synthesis, in contrast, continues to progress steadily at the fringes of both industrial and academic interest and is now poised to provide the necessary platform to overcome basic problems in speech processing and, I believe, represents the next major advance in speech synthesis technology. As part of my long-term research program, I have been establishing computer modeling and simulation of the oral, pharyngeal and laryngeal (OPAL) complex as a critical and central tool in many disciplines, including engineering, linguistics, education, entertainment, anatomy, physiology and medicine. From this effort, the infrastructure necessary to make large advances in articulatory speech synthesis is now available. Thus, I propose to develop the OPAL complex models to include aero-acoustic simulation to create the next generation of validated, articulatory speech synthesis. This research builds upon the detailed biomechanical models of the human of the upper airway that have been used to study the mechanics of speech behavior, swallowing and mastication along with the tools needed for fast, accurate coupled FEM and rigid body simulations. The main research activities are: 1. Integration of model geometries and biomechanics to create a complete biomechanical model of the OPAL complex capable of articulatory speech synthesis; 2. Improvement of the existing 1D Navier Stokes approach to provide improved coupling between laryngeal excitation of the vocal tract filter by leveraging the work of [Birkholz et al, 2006]; 3. Development of 3D aero-acoustic simulation of vocal tract noise sources based on fluid simulation as well as CFD based approximations; 4. Creation of inverse methods for driving speech articulation by extending the work of [Stavness et al, 2013] to include support for propagating both kinematic targets from image data or electromagnetic articulography (EMA) of speech and acoustic targets, backward through the aeroacoustic model, to derive muscle activations based on inverse dynamics techniques. Speech articulation constraints will be used to help make the system better determined. 5. Creation of advanced coupled vocal fold models that will oscillate under tension and airflow. We are poised to make a substantial advance in articulatory speech synthesis. The last decade of effort in creating the medical imaging techniques, speech measurements, complex 3D biomechanical models of the OPAL complex and tools for fast, accurate coupled FEM/rigid body simulation provides the necessary infrastructure for this project. Canada stands to benefit from being at the forefront of speech synthesis technology that support the next wave of applications. Including articulatory speech synthesis into our open source toolkit will support international efforts in speech research contributing profound new knowledge from a deeper understanding of human speech production.