Control strategies for articulatory speech synthesis for natural user interfaces

我们建议通过建立和评估基于神经激活的模型来驱动一个参数化的、3D生物力学-发音语音合成器，从而推动发音合成研究的最新发展。我们的行业合作伙伴CTF(http://www.ctfmeg.com))提供脑磁图(MEG)和大脑和神经通路的多模式传感方面的专业知识，为开发新型低自由度语音控制策略做出了贡献。拟议的研究目标是ICT-人与数字信息的交互，因为我们专注于基于语音的自然用户界面的基本方面。我们计划从我们的生物力学驱动的语音合成器向后工作，以确定利用生物力学提供的耦合来减少产生自然发音所需的DOF的控制策略。创建低DOF控件表示是为自然用户界面所需的化身类型界面创建移动和听起来自然的说话头的重要步骤。**我们对这个项目有三个中心目标：1)改进我们现有的生物力学语音转换模型，2)开发神经控制拓扑结构，3)建立生物力学-神经系统耦合的控制模型。这些目标将会聚在一起，合成一个神经控制的说话头部，用于自然用户界面评估。作为一个最高的演示，我们将探索让一个人在脑磁图中使用神经控制的说话头部实时说话。**我们的成熟研究团队跨越了该项目的专业领域。Fels博士、Stavness博士和Gick博士共同发表了关于口腔、咽喉和喉部复合体的生物力学以及语音发音控制的论文。CTF工程师拥有十多年的经验，推动了脑磁图感知神经激活的技术水平。我们团队的世界级建模、仿真、人机交互、语音合成控制和脑磁图专业知识为HQP提供了学习基于3D物理的模拟、高级语音控制、机器学习和神经传感技术的前沿机会。

We propose to advance the state of the art of articulatory speech synthesis research by building and evaluating neural activation-based models to drive a parameterized, 3D biomechanical-articulatory speech synthesizer. Our industry partner, CTF (http://www.ctfmeg.com) provides expertise in magnetoencephalography (MEG) and multimodal sensing of the brain and neural pathways, contributing to developing novel low degree-of-freedom (DoF) speech control strategies. The proposed research targets ICT - Human Interaction with Digital Information, as we focus on fundamental aspects of speech-based natural user interfaces. We plan to work backwards from our biomechanically driven speech synthesizer to determine control strategies that take advantage of the coupling afforded by biomechanics to reduce the DoF needed for producing natural sounding speech. Creating low DoF control representations is an important step for creating talking heads that move and sound natural for avatar-type interfaces needed for natural user interfaces. ** We have three central goals for this project: 1) improving our existing biomechanics-to-speech model, 2) developing neural control topologies, and 3) building control models for the coupled biomechanical-neural system. These goals will converge to synthesize a neurally controlled talking head for use in natural user interface evaluations. As a capstone demonstration, we will explore having a person speak in real-time using the neurally controlled talking head while in the MEG.** Our team of established researchers span the fields of expertise for this project. Drs. Fels, Stavness and Gick have published together on the biomechanics of the oral, pharyngeal and laryngeal complex and articulatory control of speech. CTF engineers have over a decade of experience advancing the state of the art of MEG for sensing neural activation. Our team's world-class modeling, simulation, HCI, speech synthesis control and MEG expertise provides a leading-edge opportunity for HQP to learn techniques for 3D physics-based simulation, advanced speech control, machine learning and neural sensing.