Establishing a ground truth for focus placement in naturally-occurring speech

为自然发生的语音中的焦点放置建立基本事实

• 批准号: 1737846
• 负责人: Jonathan Howell
• 金额: $ 10.59万
• 依托单位: Montclair State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-07-01 至 2022-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1737846&HistoricalAwards=false
• 关键词: Establishing; ground; truth; focus; placement

By emphasizing words acoustically, people can convey the information about which concepts they wish to contrast. This feature of speech, known as focus, is pervasive in English, yet is inadequately modeled in state-of-the-art speech technologies. The challenge, which this Early Grant for Exploratory Research addresses, is that it is often difficult to identify phonetic emphasis independently of semantic contrast: words whose meanings are focused are usually realized with increased acoustic prominence, but not all cases of increased acoustic prominence are due to focus. The project is innovative in its use both of speech that has been recorded in a laboratory under controlled conditions, and also of speech that occurs naturally, such as in podcasts and videos. Judgments of focus location in laboratory speech and in naturally-occurring speech are collected from ordinary, non-expert listeners using online crowd-sourcing. Using the comparative construction (for example, "He liked it better than I did" or "I like it better now than I did") in which focus can be independently verified, computational procedures are developed to mimic the judgment of subjects who read but do not listen to the utterance being investigated. The findings will inform research in speech synthesis and in automatic speech recognition. Commercial applications may include aids for the deaf and hearing impaired, robot assistants for the elderly, language instruction and speech therapy.In a previous proof-of-concept study, the researcher collected utterances of "than I did" in laboratory experiments and from transcribed podcasts available on the web. Machine learning classifiers (using linear discriminant analysis and support vector machines) were trained to detect focus from acoustic features alone, including measures of fundamental frequency, duration and intensity. Location of focus can be determined independently from prosody in the comparative construction by observing the presence or absence of co-reference between subjects in the main and comparative clauses. This research generalizes that study to variations of the comparative with different pronouns and auxiliaries and also introduces updated methods of acoustic extraction and classification. Then, a verification dataset is created in order to reject annotations from participants who annotate non-focal prominence or who mark focus location incorrectly. Finally, classifiers are trained to detect focus on pronouns and auxiliaries in contexts other than the comparative, using the crowd-sourced annotation data to infer correct location of focus independently from prosody.

通过在听觉上强调单词，人们可以传达他们希望对比的概念的信息。 这种语音特征，被称为焦点，在英语中很普遍，但在最先进的语音技术中没有充分建模。 这项探索性研究早期资助所面临的挑战是，通常很难独立于语义对比来识别语音强调：其含义被聚焦的单词通常会随着声学突出度的增加而实现，但并非所有声学突出度增加的情况都是由于聚焦。 该项目创新地使用了在受控条件下在实验室中记录的语音，以及自然发生的语音，如播客和视频。 实验室语音和自然发生的语音中的焦点位置的判断收集从普通的，非专家的听众使用在线众包。 使用比较结构（例如，“他比我更喜欢它”或“我现在比我更喜欢它”），其中焦点可以独立验证，计算程序被开发来模仿阅读但不听被调查话语的受试者的判断。 这些发现将为语音合成和自动语音识别的研究提供信息。 商业应用可能包括聋人和听力受损者的辅助设备，老年人的机器人助手，语言教学和言语治疗。在之前的一项概念验证研究中，研究人员在实验室实验中收集了“比我做的”的话语，并从网上转录的播客中获得。 机器学习分类器（使用线性判别分析和支持向量机）被训练成仅从声学特征（包括基频、持续时间和强度的测量）检测焦点。 在比较结构中，焦点的位置可以通过观察主句和比较分句中主语之间是否存在共指来独立于韵律来确定。 本研究将比较级的研究归纳到不同代词和助词的比较级变体中，并介绍了声学提取和分类的最新方法。 然后，创建验证数据集，以便拒绝来自注释非焦点突出或不正确地标记焦点位置的参与者的注释。 最后，训练分类器以检测在比较级之外的上下文中对代词和助动词的关注，使用众包注释数据来独立于韵律推断正确的焦点位置。