3D Articulatory Speech Synthesis using Biomechanical Models of the Oral, Pharyngeal and Laryngeal Complex

多年来，传统的基于共振峰和基于声学的语音合成技术在很大程度上盖过了发音合成的研究。虽然它们在某些领域取得了成功，但仍然不能从文本中产生自然发音的语音。相比之下，发音语音合成继续在工业界和学术界的边缘稳步发展，现在准备提供必要的平台来克服语音处理中的基本问题，我相信这代表着语音合成技术的下一个重大进步。作为我长期研究计划的一部分，我一直在建立口腔、咽喉(Opal)复合体的计算机建模和模拟，将其作为许多学科的关键和核心工具，包括工程学、语言学、教育、娱乐、解剖学、生理学和医学。通过这一努力，在发音语音合成方面取得重大进展所需的基础设施现在已经可用。因此，我建议开发Opal Complex模型，以包括气动声学模拟，以创建下一代经过验证的发音语音合成。这项研究建立在人类上呼吸道的详细生物力学模型的基础上，这些模型已被用于研究言语行为、吞咽和咀嚼的力学，以及快速、准确地耦合有限元和刚体模拟所需的工具。主要的研究活动是：1.整合模型几何和生物力学以创建能够进行发音合成的完整的Opal复合体的生物力学模型；2.改进现有的一维Navier Stokes方法，通过利用Birkholz等人的工作来提供声道过滤器的喉部激励之间的改善耦合；3.基于流体模拟和基于CFD的近似，开发声道噪声源的3D气动声学模拟；4.通过扩展[Stavness et al，2013]的工作来创建用于驱动语音清晰度的逆方法，以包括支持从语音的图像数据或电磁关节图(EMA)向后传播语音和声学目标的运动目标，以基于逆动力学技术推导肌肉激活。语音清晰度约束将用于帮助更好地确定系统。5.创建先进的声带耦合模型，该模型将在张力和气流下振荡。我们准备在清晰度语音合成方面取得实质性进展。过去十年在创建医学成像技术、语音测量、Opal复合体的复杂3D生物力学模型以及快速、准确的有限元/刚体耦合模拟工具方面所做的努力为该项目提供了必要的基础设施。加拿大将受益于走在支持下一波应用程序的语音合成技术的前沿。将发音语音合成纳入我们的开源工具包将支持语音研究的国际努力，从对人类语音产生的更深层次理解中贡献深刻的新知识。

For years, traditional formant-based and acoustic-based speech synthesis techniques have largely overshadowed articulatory synthesis research. While successful in some domains, they still cannot produce natural sounding speech from text. Articulatory speech synthesis, in contrast, continues to progress steadily at the fringes of both industrial and academic interest and is now poised to provide the necessary platform to overcome basic problems in speech processing and, I believe, represents the next major advance in speech synthesis technology. As part of my long-term research program, I have been establishing computer modeling and simulation of the oral, pharyngeal and laryngeal (OPAL) complex as a critical and central tool in many disciplines, including engineering, linguistics, education, entertainment, anatomy, physiology and medicine. From this effort, the infrastructure necessary to make large advances in articulatory speech synthesis is now available. Thus, I propose to develop the OPAL complex models to include aero-acoustic simulation to create the next generation of validated, articulatory speech synthesis. This research builds upon the detailed biomechanical models of the human of the upper airway that have been used to study the mechanics of speech behavior, swallowing and mastication along with the tools needed for fast, accurate coupled FEM and rigid body simulations. The main research activities are: 1. Integration of model geometries and biomechanics to create a complete biomechanical model of the OPAL complex capable of articulatory speech synthesis; 2. Improvement of the existing 1D Navier Stokes approach to provide improved coupling between laryngeal excitation of the vocal tract filter by leveraging the work of [Birkholz et al, 2006]; 3. Development of 3D aero-acoustic simulation of vocal tract noise sources based on fluid simulation as well as CFD based approximations; 4. Creation of inverse methods for driving speech articulation by extending the work of [Stavness et al, 2013] to include support for propagating both kinematic targets from image data or electromagnetic articulography (EMA) of speech and acoustic targets, backward through the aeroacoustic model, to derive muscle activations based on inverse dynamics techniques. Speech articulation constraints will be used to help make the system better determined. 5. Creation of advanced coupled vocal fold models that will oscillate under tension and airflow. We are poised to make a substantial advance in articulatory speech synthesis. The last decade of effort in creating the medical imaging techniques, speech measurements, complex 3D biomechanical models of the OPAL complex and tools for fast, accurate coupled FEM/rigid body simulation provides the necessary infrastructure for this project. Canada stands to benefit from being at the forefront of speech synthesis technology that support the next wave of applications. Including articulatory speech synthesis into our open source toolkit will support international efforts in speech research contributing profound new knowledge from a deeper understanding of human speech production.