Collaborative Research: Monitoring Student State in Tutorial Spoken Dialogue

本研究探讨在口语对话教学系统中监控学生情绪的可行性与效用。 虽然人类导师对学生话语的内容和潜在的感知情感做出响应，但大多数教程对话系统无法检测学生的情感，而且是基于文本的，这可能会限制他们在情感预测方面的成功。 虽然已经有越来越多的兴趣，在确定有问题的情绪（如挫折，愤怒）在口语对话应用程序，如呼叫中心，很少的工作已经解决了tutorial domain.The PI正在调查使用词汇，句法，对话，韵律和声学线索，使计算机导师自动预测和响应学生的情绪。 该研究是在ITSPOKE的背景下进行的，ITSPOKE是一个基于语音的概念物理教学对话系统。 PI正在记录学生与ITSPOKE的互动，手动注释这些以及人与人对话中的学生情绪，识别注释的语言和非语言线索，并使用机器学习来预测潜在线索中的情绪。 PI然后得出的策略，以适应系统的辅导的基础上的情感identifiation.The主要的科学贡献将是一个了解是否线索可用的口语对话系统可以用来预测情绪，并最终提高辅导性能。 这些结果对于其他可以从监控情绪语音中受益的应用程序具有价值。 缩小人类导师和当前机器导师之间的性能差距的进展也将扩大当前计算机导师的有用性。

This research investigates the feasibility and utility of monitoring student emotions in spoken dialogue tutorial systems. While human tutors respond to both the content of student utterances and underlying perceived emotions, most tutorial dialogue systems cannot detect student emotions, and furthermore are text-based, which may limit their success at emotion prediction. While there has been increasing interest in identifying problematic emotions (e.g. frustration, anger) in spoken dialogue applications such as call centers, little work has addressed the tutorial domain.The PIs are investigating the use of lexical, syntactic, dialogue, prosodic and acoustic cues to enable a computer tutor to automatically predict and respond to student emotions. The research is being performed in the context of ITSPOKE, a speech-based tutoring dialogue system for conceptual physics. The PIs are recording students interacting with ITSPOKE, manually annotating student emotions in these as well as in human-human dialogues, identifying linguistic and paralinguistic cues to the annotations, and using machine learning to predict emotions from potential cues. The PIs are then deriving strategies for adapting the system's tutoring based upon emotion identification.The major scientific contribution will be an understanding of whether cues available to spoken dialogue systems can be used to predict emotion, and ultimately to improve tutoring performance. The results will be of value to other applications that can benefit from monitoring emotional speech. Progress towards closing the performance gap between human tutors and current machine tutors will also expand the usefulness of current computer tutors.