RI: Small: Collaborative Research: Developing Golden Speakers for Second-Language Pronunciation Training

RI：小型：合作研究：开发第二语言发音训练的黄金音箱

• 批准号: 1619212
• 负责人: Ricardo Gutierrez-Osuna
• 金额: $ 18万
• 依托单位: Texas A&M Engineering Experiment Station
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-09-01 至 2022-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1619212&HistoricalAwards=false
• 关键词: RI; Small; Collaborative; Research; Developing

People who learn a second language (L2) as adults often speak with a persistent foreign accent. This can make them less intelligible, more subject to discrimination, and less confident when interacting with others. Surprisingly, though, L2 learners rarely receive formal training in pronunciation, in part because effective training must be customized to meet each learner's individual needs. To address this gap, the investigators propose to develop algorithms to synthesize a personalized "golden speaker" for each learner: his or her own voice but with a native accent. The rationale is that, by listening to their own golden speaker, learners can more easily perceive differences between their actual and ideal pronunciations. This work focuses on developing the technology for golden speakers, which the investigators plan to evaluate in the future as a new tool for pronunciation learning systems. As such, this research can benefit a large number of workers in the US who are non-native speakers of English, particularly in higher education, health care and the technology sector. The project also provides opportunities for graduate and undergraduate students to conduct research in a multi-disciplinary team with expertise in signal processing, machine learning, and language acquisition. Two types of golden-speaker model are proposed. The first type is based on a reformulation of parametric statistical models for voice conversion, where instead of force-aligning source (native) and target (non-native) frames, they are matched based on their phonetic similarity. Several similarity metrics are proposed, from vocal-tract-length normalization to deep auto-encoders. The second type is based on a sparse representation of speech, which models individual frames as linear combinations of phonetic anchors. This requires new techniques to transform the constellation of anchors in the L2 speech to match the structure of native anchors (e.g., pairwise distances). Two types of evaluation are proposed for the golden-speaker models: their ability to interpolate phones not included in the learner's inventory, and the accent, intelligibility and comprehensibility of the resulting speech, as rated by native English listeners. For this purpose, the investigators propose to collect a large speech corpus from multiple Spanish and Korean learners of English and Indian speakers of English, each at different levels of English proficiency.

成年人学习第二语言（L2）的人通常说话带有持久的外国口音。这可能会使他们难以理解，更容易受到歧视，在与他人互动时缺乏自信。然而，令人惊讶的是，二语学习者很少接受正式的发音培训，部分原因是有效的培训必须根据每个学习者的个人需求进行定制。为了解决这一差距，研究人员建议开发算法，为每个学习者合成个性化的“黄金扬声器”：他或她自己的声音，但带有本地口音。这样做的理由是，通过听自己的黄金演讲者，学习者可以更容易地感知到他们实际发音和理想发音之间的差异。这项工作的重点是开发黄金扬声器技术，研究人员计划在未来将其作为语音学习系统的新工具进行评估。因此，这项研究可以使大量非英语为母语的美国工人受益，特别是在高等教育、医疗保健和科技领域。该项目还为研究生和本科生提供了在信号处理、机器学习和语言习得方面具有专业知识的多学科团队中进行研究的机会。提出了两种金扬声器模型。第一种类型是基于语音转换参数统计模型的重新表述，其中不是强制对齐源（本机）和目标（非本机）帧，而是基于它们的语音相似性进行匹配。提出了几个相似度量，从声道长度归一化到深度自编码器。第二种类型是基于语音的稀疏表示，它将单个框架建模为语音锚点的线性组合。这需要新的技术来转换二语话语中的锚点群，以匹配本地锚点的结构（例如，成对距离）。对黄金说话者模型提出了两种类型的评估：他们插入不包括在学习者清单中的电话的能力，以及由此产生的语音的口音，可理解性和可理解性，由母语为英语的听众评分。为此，研究人员建议收集多个西班牙语和韩语英语学习者以及印度英语学习者的大量语料库，每个人的英语熟练程度不同。