Establishing a ground truth for focus placement in naturally-occurring speech

通过从听觉上强调词语，人们可以传达出他们希望对比哪些概念的信息。这种被称为焦点的语音功能在英语中很普遍，但在最先进的语音技术中却没有得到充分的模拟。探索性研究早期拨款所解决的挑战是，通常很难独立于语义对比来识别语音重点：其含义被聚焦的单词通常是通过增强声学突显来实现的，但并不是所有声学突显增强的情况都是由于聚焦。该项目在使用实验室在受控条件下录制的语音以及自然出现的语音(如播客和视频)方面具有创新性。对实验室语言和自然语言中焦点位置的判断是使用在线众包从普通的、非专家的听众那里收集的。使用比较句(例如，“他比我更喜欢它”或“我现在比我更喜欢它”)，焦点可以被独立验证，计算程序被开发来模仿阅读但不听正在调查的话语的受试者的判断。这些发现将为语音合成和自动语音识别方面的研究提供参考。商业应用可能包括为聋人和听力受损的人提供辅助设备，为老年人提供机器人助手，语言教学和语言治疗。在之前的一项概念验证研究中，研究人员收集了实验室实验中和网络上转录的播客中“Than I do”的话语。机器学习分类器(使用线性判别分析和支持向量机)被训练成仅从声学特征中检测焦点，包括基频、持续时间和强度的测量。通过观察主句和比较句中主语之间有无共指，可以独立于比较句中的韵律来确定焦点的位置。本研究将这一研究推广到不同代词和助词比较级的变化，并介绍了声学提取和分类的最新方法。然后，创建验证数据集，以便拒绝来自注释非震源突起或错误标记焦点位置的参与者的注释。最后，分类器被训练以检测除比较级之外的上下文中的代词和助词上的焦点，使用众包注释数据独立于韵律推断焦点的正确位置。

By emphasizing words acoustically, people can convey the information about which concepts they wish to contrast. This feature of speech, known as focus, is pervasive in English, yet is inadequately modeled in state-of-the-art speech technologies. The challenge, which this Early Grant for Exploratory Research addresses, is that it is often difficult to identify phonetic emphasis independently of semantic contrast: words whose meanings are focused are usually realized with increased acoustic prominence, but not all cases of increased acoustic prominence are due to focus. The project is innovative in its use both of speech that has been recorded in a laboratory under controlled conditions, and also of speech that occurs naturally, such as in podcasts and videos. Judgments of focus location in laboratory speech and in naturally-occurring speech are collected from ordinary, non-expert listeners using online crowd-sourcing. Using the comparative construction (for example, "He liked it better than I did" or "I like it better now than I did") in which focus can be independently verified, computational procedures are developed to mimic the judgment of subjects who read but do not listen to the utterance being investigated. The findings will inform research in speech synthesis and in automatic speech recognition. Commercial applications may include aids for the deaf and hearing impaired, robot assistants for the elderly, language instruction and speech therapy.In a previous proof-of-concept study, the researcher collected utterances of "than I did" in laboratory experiments and from transcribed podcasts available on the web. Machine learning classifiers (using linear discriminant analysis and support vector machines) were trained to detect focus from acoustic features alone, including measures of fundamental frequency, duration and intensity. Location of focus can be determined independently from prosody in the comparative construction by observing the presence or absence of co-reference between subjects in the main and comparative clauses. This research generalizes that study to variations of the comparative with different pronouns and auxiliaries and also introduces updated methods of acoustic extraction and classification. Then, a verification dataset is created in order to reject annotations from participants who annotate non-focal prominence or who mark focus location incorrectly. Finally, classifiers are trained to detect focus on pronouns and auxiliaries in contexts other than the comparative, using the crowd-sourced annotation data to infer correct location of focus independently from prosody.