SBIR Phase I: A language learning app based on sound and mouth movements

SBIR 第一阶段：基于声音和嘴部动作的语言学习应用程序

• 批准号: 2323040
• 负责人: Margaret Smith
• 金额: $ 27.47万
• 依托单位: WORD OF MOUTH TECHNOLOGIES INC.
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2323040&HistoricalAwards=false
• 关键词: SBIR; Phase; language; learning; app

The broader/commercial impact of this Small Business Innovation Research (SBIR) Phase I project is advancing new language learning by incorporating facial and lip recognition along with sound analysis. This visual aspect of creating sounds is vital for mastering pronunciation, one of the significant hurdles of learning a foreign language and even improving a native language. Current language learning methods often fall short in helping learners achieve speaking proficiency and fail to provide real-life language usage experiences. This language learning platform aims to change this by addressing the growing need for multi-language proficiency in workplaces and academic settings, providing an effective and engaging language learning experience.Current language learning methods and apps often fail to develop speaking and writing proficiency, focusing instead on memorization and standardized tests. This language trainer addresses this gap by offering insights into the science of speech production. By combining visual cues of oral shapes with auditory input, learners can master pronunciation, a significant challenge in language acquisition. This research will include obtaining near-perfect voice files for machine learning model training, signal processing of the voice and video files, development and comparison of machine learning models, data visualization development, incorporation into the mobile test suite, and preliminary testing. The machine learning algorithms will use the insights extracted from students' voice data to provide learners with highly targeted, fine-tuned activities.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

这个小企业创新研究（SBIR）第一阶段项目的更广泛/商业影响是通过结合面部和嘴唇识别以及声音分析来推进新的语言学习。创造声音的视觉方面对于掌握发音至关重要，发音是学习外语甚至提高母语的重要障碍之一。目前的语言学习方法往往不能帮助学习者达到口语的熟练程度，也不能提供真实的语言使用体验。该语言学习平台旨在通过解决工作场所和学术环境中对多语言能力日益增长的需求来改变这一现状，提供有效且引人入胜的语言学习体验。目前的语言学习方法和应用程序往往不能培养口语和写作能力，而是侧重于记忆和标准化测试。这个语言培训师通过提供对语言产生科学的见解来解决这个差距。通过将口头形状的视觉线索与听觉输入相结合，学习者可以掌握发音，这是语言习得中的一个重大挑战。这项研究将包括为机器学习模型训练获取近乎完美的语音文件、语音和视频文件的信号处理、机器学习模型的开发和比较、数据可视化开发、纳入移动测试套件以及初步测试。机器学习算法将使用从学生语音数据中提取的见解，为学习者提供高度针对性的微调活动。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。