RI: Small: Concatenative Resynthesis for Very High Quality Speech Enhancement

环境噪声是诸如助听器、移动的电话和自动语音识别等语音技术的用户面临的最大问题之一。目前的源分离和语音增强方法通常试图修改有噪信号，以使其更像原始信号，从而导致目标语音和残留噪声的失真。相比之下，该项目使用了一种创新的方法，即使用从嘈杂信号中提取的信息来驱动语音合成器，以创建一个全新的、高质量的、无噪音的原始句子版本。 这种方法在噪声抑制和语音质量方面的改进预计将对3600万听力受损的美国人和2亿使用智能手机的美国人产生重要的更广泛的影响。该项目也被纳入一所多元化城市大学的课程，并被纳入到附近高中的既定推广计划中，目的是鼓励代表性不足的群体的成员追求科学和工程事业。该项目旨在通过修改连接语音合成器，使用基于新型深度神经网络（DNN）的单元选择功能，来制作高质量的语音再合成系统架构初步结果表明，这种方法在小词汇量、依赖于说话者的任务中效果很好，目前的项目以三种方式将其扩展到大词汇量、依赖于说话者的环境中。首先，它试图通过利用感知动机的输入功能，更灵活的训练信号，和传统的语音增强来提高合成语音的可懂度。其次，它试图通过训练DNN来提高系统的可扩展性，以将噪声和干净的语音嵌入到可以有效计算相似性的联合低维空间中。第三，它试图通过结合基于声学、语音和语言兼容性的语音单元的顺序模型来提高合成语音的质量。在语音增强中使用语音合成模型是对传统方法的背离，并且有可能对增强语音的质量产生变革性的影响。

Environmental noise is one of the largest problem for users of voice technologies, such as hearing aids, mobile phones, and automatic speech recognition. Current approaches to source separation and speech enhancement typically attempt to modify the noisy signal in order to make it more like the original, leading to distortions in target speech and residual noise. In contrast, this project uses the innovative approach of driving a speech synthesizer using information extracted from the noisy signal to create a brand new, high quality, noise-free version of the original sentence. Improvements in noise suppression and speech quality from this approach are expected to have important broader impacts for both the 36 million Americans who are hearing impaired and the 200 million Americans who use smart phones. The project is also being incorporated into the curriculum in a diverse urban college and into established outreach programs to nearby high schools with the goal of encouraging members of under-represented groups to pursue careers in science and engineering.This project aims to produce a high quality speech resynthesis system by modifying a concatenative speech synthesizer to use a unit-selection function based on a novel deep neural network (DNN) architecture. Preliminary results have shown this approach to work well for a small-vocabulary, speaker-dependent task, and the current project expands this to the large-vocabulary, speaker-dependent setting in three ways. First, it seeks to improve the intelligibility of the synthesized speech by utilizing perceptually motivated input features, more flexible training signals, and traditional speech enhancement. Second, it seeks to improve the system's scalability by training DNNs to embed noisy and clean speech into a joint low-dimensional space in which similarity can be efficiently computed. And third, it seeks to improve the quality of the synthesized speech by incorporating sequential models of speech units based on acoustic, phonetic, and linguistic compatibility. The use of speech synthesis models in speech enhancement is a departure from traditional approaches and has the potential to make a transformative impact on the quality of enhanced speech.