I-Corps: Real-time visual feedback for speech learning

这个I-Corps项目更广泛的影响是开发一种语音学习技术，使用视觉-听觉生物反馈来帮助那些对改变自己的语音模式感兴趣的人。当一个人不能以正常的方式发出某些声音或不能成功地交流时，就会对参与社会和专业环境产生负面影响。治疗语言障碍的语言病理学家报告说，一些语音发音的循证治疗和资源很少。很少有经过经验检验的干预措施可以使语音信号可视化，这是一种以证据为基础的增强学习的方法。该软件还可以通过视频会议或远程实践远程实施，对服务提供商、临床医生和个人有潜在的重大影响。最终，提供更高效的技术可以改善许多人的沟通和生活质量。虽然一些语音在临床和语言学习环境中具有很高的重要性，但在未来的发展阶段，有很多声音会扩大这项技术的影响。另一个应用程序包括性别确认语音训练。这个I-Corps项目利用体验式学习和对行业生态系统的第一手调查来评估该技术的翻译潜力。该解决方案是基于一项技术的发展，该技术提供视觉-听觉生物反馈，以增强学习能力，并最终帮助个人改变他们的语言模式。该技术集成在软件中，允许学习者查看声道的共振频率，并将其与代表所需发音的视觉目标进行比较。该技术采用了对语音信号进行实时线性预测编码的形式。操纵语言的某些特征，如声道的共振频率，是一个复杂的概念。该软件可以在各种设备上使用，并通过视频会议使用。该技术包括增强个人语言学习过程的功能，包括刺激提示的自适应困难、进度跟踪和游戏化。最后，该语音可视化软件将与开发的机器学习分类器集成，以扩展该技术的实用性。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The broader impact of this I-Corps project is the development of a speech learning technology using visual-acoustic biofeedback for individuals interested in altering their speech patterns. When individuals are unable to pronounce certain sounds in a normal way or communicate successfully, there are negative impacts to participation in social and professional settings. Speech-language pathologists who treat speech disorders report that evidence-based treatment and resources for some sound pronunciations are scarce. Very few empirically-tested interventions are available that allow visualization of the acoustic speech signal, an evidence-based method to enhance learning. This software can also be implemented remotely via videoconferencing or telepractice, with the potential for significant impact on service providers, clinicians, and individuals. Ultimately, providing access to more efficient technology can improve communication and quality of life for many individuals. While some speech sounds have high salience in clinical and language learning contexts, there is a wide range of sounds that would broaden the impact of this technology in future stages of development. An additional application includes gender-affirming voice training.This I-Corps project utilizes experiential learning coupled with a first-hand investigation of the industry ecosystem to assess the translation potential of the technology. The solution is based on the development of a technology that provides visual-acoustic biofeedback to enhance learning and ultimately help individuals change their speech patterns. This technology is integrated in software that allows learners to view resonant frequencies of the vocal tract and compare them with a visual target representing the desired pronunciation. The technology uses the form of a real-time linear predictive coding spectrum of the speech signal. Manipulating certain characteristics of speech, such as resonant frequencies of the vocal tract, is a complex concept. This software can be used across a wide range of devices and via videoconferencing. The technology includes features to enhance an individual’s speech learning process, which includes adaptive difficulty in stimulus prompts, progress tracking, and gamification. Lastly, this speech visualization software will be integrated with a machine learning classifier developed to extend the utility of the technology.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.