ITR: Recognizing and Understanding Emotion in Speech

口语不仅仅是简单的可听文本。我们创造单词和短语的方式告诉我们的听众许多关于我们的精神状态和我们言语和行为背后的意图。虽然机器在识别当前口语对话系统中的句子方面已经变得越来越熟练，但它们在检测说话人是沮丧还是自信，或者他们是否试图欺骗或向听者传达有用信息等方面仍然非常糟糕。尽管在行为、实验室言语中识别情感和意图的语言和非语言线索以及在更自然的环境中识别有限类型的情感的早期结果是有希望的，但我们对这些现象的可靠语言线索(声学、韵律、词汇和句法)的理解有限，因此不知道如何自动识别这些现象。PI将首先进行一系列实验室实验，以识别诱导(非行为)言语中情感和意图的声学、韵律、词汇和句法线索。同时，他们将发现新的特征，这些特征可能有助于使用具有扩展标记的可用语料库来自动识别情感和意图，例如欺骗性、信心和沮丧/愤怒，并为这些说话者状态/意图标记新的语料库。他们随后将在实验室录音中测试这些特征，并确定通过分析这些录音可能提出的新特征。从实用的角度来看，自动识别说话人状态/意图对于诸如呼叫中心响应系统、电话银行或旅行预订系统或教学系统之类的交互式语音响应系统应该有相当大的好处。说话人状态/意图的自动识别还应为当前依赖于人类对说话人状态/意图的评估的各种应用中的说话人筛选提供有用的信息。

Spoken language is much more than simply audible text. The way we produce words and phrases tells our hearers much about our mental state and the intentions that underlie our words and actions. While machines have become increasingly proficient at recognizing sentences in current spoken dialog systems, they are still very poor at detecting, inter alia, whether speakers are frustrated or confident, or whether they are trying to deceive or to convey helpful information to their hearers. Despite promising work in identifying verbal and non-verbal cues to emotion and intention in acted, laboratory speech, and early results identifying limited types of emotion in more natural settings, we have only a limited understanding of reliable verbal cues (acoustic, prosodic, lexical and syntactic) to these phenomena and consequently do not know how to automatically recognize the phenomena automatically.The PIs will first conduct a series of laboratory experiments to identify acoustic, prosodic, lexical and syntactic cues to emotion and intention in elicited (non-acted) speech. They will, in parallel, discover new features that may be useful in automatically identifying emotions and intentions such as deceptiveness, confidence, and frustration/anger using available corpora with augmented labeling and labeling new corpora for these speaker states/intentions. They will subsequently test these features on the laboratory recordings and identify new features that may be suggested by analysis of these recordings. The result should provide a better understanding of what auditory cues characterize certain speaker states/intentions and which of these provide reliable features for their automatic identification.From a practical point of view, identifying speaker state/intentions automatically should be of considerable benefit for interactive voice response systems such as call center response systems, over the phone banking or travel reservation systems, or tutorial systems. Automatic identification of speaker state/intentions should also provide useful information for speaker screening in a variety of applications that currently depend upon human assessment of speaker state/intention.