Generating Personalized Synthetic Speech for Progressive Dysarthria Using Severity-Appropriate Adaptation Strategies for Neural Text-to-Speech and Voice Conversion

项目总结 200多万美国人患有复杂的沟通障碍，削弱了他们的说话能力。损失 言语障碍是肌萎缩侧索硬化症(ALS)等神经系统疾病最令人衰弱的影响之一， 95%的人会逐渐失去说话能力，陷入孤立状态。沟通 具有电子语音输出的设备允许患者将语言交流增强或取代为他们的语言 变坏了。这些设备上可用的文本(字母表、消息)可通过Function直接访问 身体部分(手指、头部、眼睛)，所选文本通过文语转换(TTS)转换为语音 技术当前设备上提供的电子TTS语音在年龄、性别和/或方面的选择有限 方言，它削弱了真实话语的体验，因为无论是用户还是他们的用户 通信伙伴可以与设备语音相关。声音是一个人身份不可分割的一部分，没有 一个捕捉到这种身份的声音，用户往往会退出互动，大大降低他们的生活质量， 并导致该技术的接受度较低。个性化TTS语音选项是ALS的关键需求 为了使他们能够在面临重大生活变化的情况下自由交流。 这项研究的长期目标是基于软件的高性能个性化语音合成，它可以 供有沟通障碍的人在移动平台和商业语音设备上使用。我们的 短期目标是研究利用最先进的端到端神经TTS的创新方法，以 为已经表现出语音丢失的人生成易于理解、自然和个性化的合成语音 肌萎缩侧索硬化。神经TTS的表现明显优于前几代TTS技术，并已降低 开发高质量TTS系统的障碍。虽然使用神经TTS显然是可取的，但需要大量的 大量的高质量语音数据禁止直接为ALS患者培训这样的系统。我们致力于 这个问题通过我们在这个探索性项目中的两个具体目标来解决：(I)通过使用语音来适应神经TTS输出 转换为针对肌萎缩侧索硬化症的个性化TTS语音选项，以及(Ii)适应神经TTS输入功能和网络 为ALS个性化TTS语音选项的参数。我们实现这两个目标的方法将保留TTS语音 可理解性和自然度，同时增强声音相似性，通过使用来自 肌萎缩侧索硬化症患者。 我们的改装神经TTS系统有望生成具有声音的个性化合成语音 肌萎缩侧索硬化症患者的个体特征以及可理解性和自然性，以促进交流和 听起来很舒服。该项目目标与NIH-NIDCD与“推进小说研究”相关的优先领域保持一致 增强和替代通信(AAC)方法“。预计项目成果将 为大量因各种原因(肌萎缩侧索硬化症、中风、 创伤)，具有个性化的声音表达和社会认同。

PROJECT SUMMARY More than 2 million Americans have a complex communication disorder that impairs their ability to talk. The loss of speech is among the most debilitating effects of neurological diseases like amyotrophic lateral sclerosis (ALS), where 95% will progressively lose their ability to speak and get trapped in a state of isolation. Communication devices with electronic voice output allow patients to augment or replace verbal communication as their speech deteriorates. The text (alphabet, messages) available on these devices is accessed directly using functioning body parts (fingers, head, eyes), and the selected text is converted to speech through text-to-speech (TTS) technology. Electronic TTS voices available on current devices have limited options in terms of age, sex, and/or dialect, which diminishes the experience of a genuine discourse because neither the user nor their communication partner can relate to the device voice. Voice is an integral part of a person’s identity and without a voice that captures this identity, users tend to withdraw from interactions, greatly reducing their quality of life, and leading to low acceptance of the technology. Personalized TTS voice options are a critical need for the ALS population in order for them to be able to communicate freely in the face of major life changes. The long-term goal of this research is software-based, high-performance personalized speech synthesis that can be used on mobile platforms and commercial speech devices by people with communication disorders. Our short-term goal is to investigate innovative methods that leverage state-of-the-art, end-to-end neural TTS, to generate intelligible, natural, and personalized synthetic speech for people who already exhibit speech loss from ALS. Neural TTS has significantly outperformed the previous generations of TTS technology, and has lowered the barrier to develop high-quality TTS systems. While it is clearly desirable to use neural TTS, the need for large quantities of high-quality speech data prohibits training such a system directly for those with ALS. We address this problem through our two specific aims in this exploratory project: (i) adapt neural TTS output by using voice conversion to personalize TTS voice options for ALS and (ii) adapt neural TTS input features and network parameters to personalize TTS voice options for ALS. Our methods for both aims will preserve TTS speech intelligibility and naturalness while enhancing voice similarity, by using modest amounts of speech data from persons with ALS. Our adapted neural TTS system is expected to generate personalized synthetic speech that has the voice characteristics of individual ALS users along with intelligibility and naturalness to promote communication and listening comfort. The project goals align with NIH-NIDCD’s priority area related to “Advancing Research in Novel Augmentative and Alternative Communication (AAC) Approaches”. The project outcomes are expected to provide a significant number of people who have communication disorders from varying etiologies (ALS, stroke, trauma) with personalized vocal expression and social identity.