I-Corps: Real-time visual feedback for speech learning

这个I-Corps项目的更广泛的影响是开发了一种语音学习技术，该技术使用视觉-听觉生物反馈，用于有兴趣改变其语音模式的个人。 当一个人无法正常地发出某些声音或无法成功地沟通时，对参与社会和专业环境会产生负面影响。治疗言语障碍的言语语言病理学家报告说，一些声音发音的循证治疗和资源是稀缺的。很少有经过实验测试的干预措施可以使声学语音信号可视化，这是一种以证据为基础的方法来增强学习。该软件还可以通过视频会议或远程实践远程实施，对服务提供商，临床医生和个人有可能产生重大影响。最终，提供更有效的技术可以改善许多人的沟通和生活质量。虽然一些语音在临床和语言学习环境中具有很高的显着性，但在未来的发展阶段，有各种各样的声音将扩大这项技术的影响。另一个应用包括性别肯定语音培训。这个I-Corps项目利用体验式学习加上对行业生态系统的第一手调查来评估该技术的翻译潜力。该解决方案基于一种技术的开发，该技术提供视觉-声学生物反馈，以增强学习并最终帮助个人改变他们的语音模式。该技术集成在软件中，允许学习者查看声道的谐振频率，并将其与代表所需发音的视觉目标进行比较。该技术使用语音信号的实时线性预测编码频谱的形式。操纵语音的某些特征，例如声道的共振频率，是一个复杂的概念。该软件可用于各种设备和视频会议。该技术包括增强个人语音学习过程的功能，包括刺激提示的自适应难度，进度跟踪和游戏化。最后，该语音可视化软件将与机器学习分类器集成，以扩展该技术的实用性。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The broader impact of this I-Corps project is the development of a speech learning technology using visual-acoustic biofeedback for individuals interested in altering their speech patterns. When individuals are unable to pronounce certain sounds in a normal way or communicate successfully, there are negative impacts to participation in social and professional settings. Speech-language pathologists who treat speech disorders report that evidence-based treatment and resources for some sound pronunciations are scarce. Very few empirically-tested interventions are available that allow visualization of the acoustic speech signal, an evidence-based method to enhance learning. This software can also be implemented remotely via videoconferencing or telepractice, with the potential for significant impact on service providers, clinicians, and individuals. Ultimately, providing access to more efficient technology can improve communication and quality of life for many individuals. While some speech sounds have high salience in clinical and language learning contexts, there is a wide range of sounds that would broaden the impact of this technology in future stages of development. An additional application includes gender-affirming voice training.This I-Corps project utilizes experiential learning coupled with a first-hand investigation of the industry ecosystem to assess the translation potential of the technology. The solution is based on the development of a technology that provides visual-acoustic biofeedback to enhance learning and ultimately help individuals change their speech patterns. This technology is integrated in software that allows learners to view resonant frequencies of the vocal tract and compare them with a visual target representing the desired pronunciation. The technology uses the form of a real-time linear predictive coding spectrum of the speech signal. Manipulating certain characteristics of speech, such as resonant frequencies of the vocal tract, is a complex concept. This software can be used across a wide range of devices and via videoconferencing. The technology includes features to enhance an individual’s speech learning process, which includes adaptive difficulty in stimulus prompts, progress tracking, and gamification. Lastly, this speech visualization software will be integrated with a machine learning classifier developed to extend the utility of the technology.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.