RI: Small: Collaborative Research: Developing Golden Speakers for Second-Language Pronunciation Training

RI：小型：合作研究：开发第二语言发音训练的黄金音箱

• 批准号: 1619212
• 负责人: Ricardo Gutierrez-Osuna
• 金额: $ 18万
• 依托单位: Texas A&M Engineering Experiment Station
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-09-01 至 2022-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1619212&HistoricalAwards=false
• 关键词: RI; Small; Collaborative; Research; Developing

People who learn a second language (L2) as adults often speak with a persistent foreign accent. This can make them less intelligible, more subject to discrimination, and less confident when interacting with others. Surprisingly, though, L2 learners rarely receive formal training in pronunciation, in part because effective training must be customized to meet each learner's individual needs. To address this gap, the investigators propose to develop algorithms to synthesize a personalized "golden speaker" for each learner: his or her own voice but with a native accent. The rationale is that, by listening to their own golden speaker, learners can more easily perceive differences between their actual and ideal pronunciations. This work focuses on developing the technology for golden speakers, which the investigators plan to evaluate in the future as a new tool for pronunciation learning systems. As such, this research can benefit a large number of workers in the US who are non-native speakers of English, particularly in higher education, health care and the technology sector. The project also provides opportunities for graduate and undergraduate students to conduct research in a multi-disciplinary team with expertise in signal processing, machine learning, and language acquisition. Two types of golden-speaker model are proposed. The first type is based on a reformulation of parametric statistical models for voice conversion, where instead of force-aligning source (native) and target (non-native) frames, they are matched based on their phonetic similarity. Several similarity metrics are proposed, from vocal-tract-length normalization to deep auto-encoders. The second type is based on a sparse representation of speech, which models individual frames as linear combinations of phonetic anchors. This requires new techniques to transform the constellation of anchors in the L2 speech to match the structure of native anchors (e.g., pairwise distances). Two types of evaluation are proposed for the golden-speaker models: their ability to interpolate phones not included in the learner's inventory, and the accent, intelligibility and comprehensibility of the resulting speech, as rated by native English listeners. For this purpose, the investigators propose to collect a large speech corpus from multiple Spanish and Korean learners of English and Indian speakers of English, each at different levels of English proficiency.

成年后学习第二语言（L2）的人经常带着持续的外国口音说话。 这可能会使他们更不容易理解，更容易受到歧视，在与他人互动时更不自信。然而，令人惊讶的是，L2学习者很少接受正式的发音培训，部分原因是有效的培训必须根据每个学习者的个人需求进行定制。 为了解决这一差距，研究人员建议开发算法，为每个学习者合成一个个性化的“黄金扬声器”：他或她自己的声音，但带有本地口音。其基本原理是，通过倾听自己的黄金演讲者，学习者可以更容易地感知他们的实际发音和理想发音之间的差异。这项工作的重点是开发黄金扬声器的技术，研究人员计划在未来将其作为发音学习系统的新工具进行评估。因此，这项研究可以使大量非英语母语的美国工人受益，特别是在高等教育，医疗保健和技术部门。该项目还为研究生和本科生提供了在具有信号处理，机器学习和语言习得专业知识的多学科团队中进行研究的机会。提出了两种类型的黄金说话人模型。第一种类型是基于语音转换的参数统计模型的重新制定，其中不是强制对齐源（本地）和目标（非本地）帧，而是基于它们的语音相似性进行匹配。 提出了几种相似性度量，从声道长度归一化到深度自动编码器。第二种类型是基于语音的稀疏表示，其将各个帧建模为语音锚点的线性组合。 这需要新的技术来变换L2语音中的锚点的星座以匹配本地锚点的结构（例如，pairwise distances）。两种类型的评价提出了黄金扬声器模型：他们的能力，插入电话不包括在学习者的库存，和口音，清晰度和可理解性的语音，由母语为英语的听众。 为此，研究人员建议收集一个大型的语音语料库，从多个西班牙语和韩国学习者的英语和印度语的英语，每个在不同的英语水平。