RI: Small: An Affect-Adaptive Spoken Dialogue System that Responds Based on User Model and Multiple Affective States

RI：Small：基于用户模型和多种情感状态进行响应的情感自适应口语对话系统

• 批准号: 0914615
• 负责人: Diane Litman
• 金额: $ 45.27万
• 依托单位: University of Pittsburgh
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2009
• 资助国家: 美国
• 起止时间: 2009-09-01 至 2013-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0914615&HistoricalAwards=false
• 关键词: RI; Small; Affect; Adaptive; Spoken

There has been increasing interest in affective dialogue systems, motivated by the belief that in human-human dialogues, participants seem to be (at least to some degree) detecting and responding to the emotions, attitudes and metacognitive states of other participants. The goal of the proposed research is to improve the state of the art in affective spoken dialogue systems along three dimensions, by drawing on the results of prior research in the wider spoken dialogue and affective system communities. First, prior research hasshown that not all users interact with a system in the same way; the proposed research hypothesizes that employing different affect adaptations for users with different domain aptitude levels will yield further performance improvement in affective spoken dialogue systems. Second, prior research has shown that users display a range of affective states and attitudes while interacting with a system; the proposed research hypothesizes that adapting to multiple user states will yield further performance improvement in affective spoken dialogue systems. Third, while prior research has shown preliminary performance gains for affect adaptation in semi-automated dialogue systems, similar gains have not yet been realized in fully automated systems. The proposed research will use state of the art empirical methods to build fully automated affect detectors. It is hypothesized that both fully and semi-automated versions of a dialogue systemthat either adapts to affect differently depending on user class, or that adapts to multiple user affective states, can improve performance compared to non-adaptive counterparts, with semi-automation generating the most improvement. The three hypotheses will be investigated in the context of an existing spoken dialogue tutoring system that adapts to the user state of uncertainty. The task domain is conceptual physics typically covered in a first-year physics course (e.g., Newtons Laws, gravity, etc.). To investigate the first hypothesis, a first enhanced system version will be developed; it will use the existing uncertainty adaptation for lower aptitude users with respect to domain knowledge, and a new uncertainty adaptation will be developed and implemented to be employed for higher aptitude users. To investigate the second hypothesis, a second enhanced systemversion will be developed; it will use the existing uncertainty adaptation for all turns displaying uncertainty, and a new disengagement adaptation will be developed and implemented to be employed for all student turns displaying a second state of disengagement. A controlled experiment with the two enhanced systems will then be conducted in a Wizard-of-Oz (WOZ) setup, with a human Wizard detecting affect and performing speech recognition and language understanding. To investigate the third hypothesis, a second controlled experiment will be conducted, which replaces the WOZ system versions with fully-automated systems.The major intellectual contribution of this research will be to demonstrate whether significant performance gains can be achieved in both partially and fully-automated affective spoken dialogue tutoring systems 1) by adapting to user uncertainty based on user aptitude levels, and 2) by adapting to multiple user states hypothesized to be of primary importance within the tutoring domain, namely uncertainty and disengagement. The research project will thus advance the state of the art in both spoken dialogue and computer tutoring technologies, while at the same time demonstrating any differing effects of affect-adaptive systems under ideal versus realistic conditions. More broadly, the research and resulting technology will lead to more natural and effective spoken dialogue-based systems, both for tutoring as well as for more traditional information-seeking domains. In addition, improving the performance of computer tutors will expand their usefulness and thus have substantial benefits for education and society.

人们对情感对话系统的兴趣越来越大，其动机是相信在人与人之间的对话中，参与者似乎（至少在某种程度上）发现并回应其他参与者的情绪、态度和元认知状态。本研究的目标是通过借鉴更广泛的口语对话和情感系统社区的先前研究结果，在三个维度上提高情感口语对话系统的技术水平。首先，先前的研究表明，并非所有用户都以相同的方式与系统交互；本研究假设对不同领域能力水平的用户采用不同的情感适应将进一步提高情感口语对话系统的性能。其次，先前的研究表明，用户在与系统交互时表现出一系列的情感状态和态度；提出的研究假设，适应多种用户状态将进一步提高情感口语对话系统的性能。第三，虽然先前的研究已经显示了半自动化对话系统中情感适应的初步性能提升，但在完全自动化系统中尚未实现类似的提升。拟议的研究将使用最先进的经验方法来建立全自动的情感探测器。假设对话系统的完全和半自动化版本，要么根据用户类别适应不同的影响，要么适应多种用户情感状态，与非自适应对应物相比，都可以提高性能，其中半自动产生的改进最大。这三个假设将在现有的口语对话辅导系统的背景下进行研究，该系统适应用户的不确定状态。任务领域是概念物理通常涵盖在第一年的物理课程（例如，牛顿定律，重力等）。为了调查第一个假设，将开发第一个增强系统版本；它将在领域知识方面使用现有的针对低资质用户的不确定性适应，并将开发和实施新的针对高资质用户的不确定性适应。为了研究第二个假设，将开发第二个增强的系统版本；它将对所有显示不确定性的回合使用现有的不确定性适应，并将开发和实施一种新的脱离接触适应，用于所有显示第二种脱离状态的学生回合。随后，将在《绿野仙踪》（Wizard-of- oz）设置中对这两种增强系统进行对照实验，由一名人类向导检测情绪，并执行语音识别和语言理解。为了调查第三个假设，将进行第二个对照实验，用全自动系统取代WOZ系统版本。本研究的主要智力贡献将是证明是否可以在部分和完全自动化的情感口语对话辅导系统中实现显著的性能提升：1)通过适应基于用户能力水平的用户不确定性，以及2)通过适应在辅导领域中假设最重要的多种用户状态，即不确定性和脱离。因此，该研究项目将推动口语对话和计算机辅导技术的发展，同时展示情感适应系统在理想和现实条件下的不同效果。更广泛地说，这项研究和由此产生的技术将导致更自然、更有效的基于口语对话的系统，既适用于辅导，也适用于更传统的信息搜索领域。此外，提高计算机教师的表现将扩大他们的用途，从而对教育和社会有实质性的好处。