EAGER: Collaborative Research: Second Language Speech Production: Formulation of Objective Speech Intelligibility Measures and Learner-Specific Feedback

EAGER：协作研究：第二语言语音生成：客观语音清晰度测量和学习者特定反馈的制定

• 批准号: 2140414
• 负责人: Stephen Looney
• 金额: $ 4.2万
• 依托单位: Pennsylvania State Univ University Park
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-10-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2140414&HistoricalAwards=false
• 关键词: EAGER; Collaborative; Research; Second; Language

This Early-concept Grants for Exploratory Research (EAGER) project focuses on exploring and developing a novel operational collection of speech, language and perception-based measures to objectively assess speech intelligibility for second language (L2) speech production, as well as providing effective learner-specific feedback. With the rise of English as an international language, intelligibility-based successful communication has been emphasized over native-like accents. However, L2 teachers often raise concerns about learners’ slow or stagnant pronunciation progress. Several primary reasons for this problem may include difficulties in perceptually discerning changes in learners’ speech and interpreting learners’ speech patterns without any learner-specific intelligibility assessment profile. Today, teachers have no systematic way to assess each student’s speech changes, nor can students monitor and track feedback related to their pronunciation learning progression. Therefore, an exploratory and transformative method is introduced for measuring speech intelligibility that provides both teachers and learners with objective and individualized feedback. This exploratory project is proposed for EAGER funding in order to establish a baseline working framework for operational objective measure creation, and proof-of-concept assessment feedback for teachers and learners. This approach will help teachers gauge learners’ intelligibility levels and allow learners to self-regulate their learning progress incrementally over time. The long-term innovation is expected to benefit skilled US professionals from non-English speaking countries, who work in various STEM (science, technology, engineering, and mathematics) fields. Additionally, this interdisciplinary project provides various opportunities for hands-on training and experience for both graduate and undergraduate students in the fields of language education, applied linguistics, computer engineering, and speech technology.This project explores an idea to assess intelligibility in speech communications based on multiple individual speech measures for non-native speakers. The ideas are currently in their very early stages of development, and a large portion of the research ideas are untested. In order to establish the ground truth of potential individual speech production intelligibility measures, the implementation and feasibility of this intelligibility feedback approach must be validated with evidence. By employing advanced Automatic Speech Recognition-based accent classification technology based on machine learning, the team of researchers plan to provide learners with measured speech property information through operational and a discriminating set of objective speech intelligibility measures. The current innovation builds on language skill acquisition theory with a functional analytic-linguistic approach, arguing that explicit and metalinguistic feedback plays a pivotal role in moving learners forward in their L2 development. The vision is enabled by on-going research on auditory-based neurogram and spectrogram orthogonal polynomial measures that predict speech intelligibility, employing the learners’ unconstrained speech utterances. The project will contribute to the scientific knowledge of what constitutes L2 intelligible speech, understanding how individualized objective speech intelligibility feedback affects L2 speech development, and creating a foundational collection of speech/auditory/signal processing measures as well as ASR/DNN driven measures that assess a speaker’s intelligibility and identify efficient ways of implementing this technology in L2 learning contexts.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

这个早期概念探索性研究资助 (EAGER) 项目的重点是探索和开发一套新颖的、基于语音、语言和感知的测量方法的操作集合，以客观地评估第二语言 (L2) 语音生成的语音清晰度，并提供针对学习者的有效反馈。随着英语作为一种国际语言的崛起，基于可理解性的成功沟通已经超过了母语口音。然而，第二语言教师经常对学习者的发音进展缓慢或停滞表示担忧。造成此问题的几个主要原因可能包括在没有任何特定于学习者的可懂度评估配置文件的情况下，难以感知地辨别学习者语音的变化以及解释学习者的语音模式。如今，教师没有系统的方法来评估每个学生的言语变化，学生也无法监控和跟踪与其发音学习进度相关的反馈。因此，引入了一种探索性和变革性的方法来测量语音清晰度，为教师和学习者提供客观和个性化的反馈。该探索性项目提议获得 EAGER 资助，以便为创建操作目标措施以及为教师和学习者提供概念验证评估反馈建立基线工作框架。这种方法将帮助教师衡量学习者的可懂度水平，并允许学习者随着时间的推移逐步自我调节他们的学习进度。这项长期创新预计将使来自非英语国家、在各个 STEM（科学、技术、工程和数学）领域工作的美国熟练专业人士受益。此外，这个跨学科项目为语言教育、应用语言学、计算机工程和语音技术领域的研究生和本科生提供了各种实践培训和经验的机会。该项目探索了一种基于非母语人士的多种个人语音测量来评估语音通信清晰度的想法。这些想法目前还处于发展的早期阶段，很大一部分研究想法未经测试。为了确定潜在的个人语音产生清晰度测量的基本事实，必须用证据验证这种清晰度反馈方法的实施和可行性。通过采用基于机器学习的先进自动语音识别口音分类技术，研究团队计划通过一套可操作的、有辨别力的客观语音清晰度测量方法，为学习者提供测量的语音属性信息。当前的创新建立在语言技能习得理论的基础上，采用功能分析语言学方法，认为显性和元语言反馈在推动学习者的第二语言发展方面发挥着关键作用。这一愿景是通过对基于听觉的神经图和声谱图正交多项式测量的持续研究来实现的，这些测量可以利用学习者不受约束的语音表达来预测语音清晰度。该项目将有助于加深对 L2 可理解性语音构成的科学认识，了解个性化客观语音清晰度反馈如何影响 L2 语音发展，并创建语音/听觉/信号处理措施以及 ASR/DNN 驱动措施的基础集合，以评估说话者的可懂度并确定在 L2 学习环境中实施该技术的有效方法。该奖项反映了 NSF 的法定使命，并被认为值得通过评估获得支持 利用基金会的智力优势和更广泛的影响审查标准。

项目成果

FLUENCY BENCHMARKS AND IMPACTS OF PRACTICE WITH INSTANTANEOUS ASSESSMENT ON INTERNATIONAL TEACHING ASSISTANTS’ SPEECH RATE AND PAUSE UNITS

国际助教的流利度基准和即时评估实践的影响——语速和停顿单位

• DOI: 10.31274/psllt.15711
• 发表时间: 2023
• 期刊: Iowa State University Digital Press
• 作者: Hirschi, Kevin;Kang, Okim;Hansen, John;Looney, Stephen D
• 通讯作者: Looney, Stephen D

Characterization and normalization of second language speech intelligibility through lexical stress, speech rate, rhythm, and pauses

通过词汇重音、语速、节奏和停顿来表征和规范第二语言语音清晰度

• DOI: 10.1121/10.0016224
• 发表时间: 2022
• 期刊: The Journal of the Acoustical Society of America
• 作者: Kang, Okim;Hirschi, Kevin;Hansen, John H.;Looney, Stephen
• 通讯作者: Looney, Stephen