RI: Small: Collaborative Research: Developing Golden Speakers for Second-Language Pronunciation Training.

成年后学习第二语言(L2)的人说话时往往带有顽固的外国口音。这可能会使他们更难理解，更容易受到歧视，在与他人互动时也会变得更不自信。然而，令人惊讶的是，二语学习者很少接受正式的发音培训，部分原因是有效的培训必须定制以满足每个学习者的个人需求。为了弥补这一差距，研究人员提议开发算法，为每个学习者合成一个个性化的“黄金演讲者”：他或她自己的声音，但带有母语口音。理由是，通过听他们自己的黄金演讲者，学习者可以更容易地感知到他们的实际发音和理想发音之间的差异。这项工作的重点是开发黄金扬声器技术，研究人员计划在未来对其进行评估，将其作为发音学习系统的新工具。因此，这项研究可以使大量非英语母语的美国工人受益，特别是在高等教育、医疗保健和科技行业。该项目还为研究生和本科生提供了在具有信号处理、机器学习和语言习得专业知识的多学科团队中进行研究的机会。提出了两种黄金说话人模型。第一种类型基于用于语音转换的参数统计模型的重新表述，其中，不是强制对准源(本地)和目标(非本地)帧，而是基于它们的语音相似性进行匹配。提出了几种相似性度量，从声道长度归一化到深度自动编码器。第二种类型基于语音的稀疏表示，它将单个帧建模为语音锚的线性组合。这需要新的技术来变换L2语音中的锚的星座以匹配本地锚的结构(例如，成对距离)。对于黄金说话人模型，我们提出了两种评估方法：一种是插入学习者清单中未包含的音素的能力，另一种是由英语母语者评定的最终语音的口音、可理解性和可理解性。为此，研究人员建议从多名西班牙语和韩语英语学习者和说英语的印度人那里收集大量语音语料库，每个人的英语熟练程度都不同。

People who learn a second language (L2) as adults often speak with a persistent foreign accent. This can make them less intelligible, more subject to discrimination, and less confident when interacting with others. Surprisingly, though, L2 learners rarely receive formal training in pronunciation, in part because effective training must be customized to meet each learner's individual needs. To address this gap, the investigators propose to develop algorithms to synthesize a personalized "golden speaker" for each learner: his or her own voice but with a native accent. The rationale is that, by listening to their own golden speaker, learners can more easily perceive differences between their actual and ideal pronunciations. This work focuses on developing the technology for golden speakers, which the investigators plan to evaluate in the future as a new tool for pronunciation learning systems. As such, this research can benefit a large number of workers in the US who are non-native speakers of English, particularly in higher education, health care and the technology sector. The project also provides opportunities for graduate and undergraduate students to conduct research in a multi-disciplinary team with expertise in signal processing, machine learning, and language acquisition. Two types of golden-speaker model are proposed. The first type is based on a reformulation of parametric statistical models for voice conversion, where instead of force-aligning source (native) and target (non-native) frames, they are matched based on their phonetic similarity. Several similarity metrics are proposed, from vocal-tract-length normalization to deep auto-encoders. The second type is based on a sparse representation of speech, which models individual frames as linear combinations of phonetic anchors. This requires new techniques to transform the constellation of anchors in the L2 speech to match the structure of native anchors (e.g., pairwise distances). Two types of evaluation are proposed for the golden-speaker models: their ability to interpolate phones not included in the learner's inventory, and the accent, intelligibility and comprehensibility of the resulting speech, as rated by native English listeners. For this purpose, the investigators propose to collect a large speech corpus from multiple Spanish and Korean learners of English and Indian speakers of English, each at different levels of English proficiency.