ITR: Multimodal Human Computer Interaction: Toward a Proactive Computer

ITR：多模式人机交互：迈向主动式计算机

• 批准号: 0085980
• 负责人: Thomas Huang
• 金额: $ 315.21万
• 依托单位: University of Illinois at Urbana-Champaign
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2000
• 资助国家: 美国
• 起止时间: 2000-09-01 至 2006-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0085980&HistoricalAwards=false
• 关键词: ITR; Multimodal; Human; Computer; Interaction

This is the first year funding of a five-year continuing award. This project is based on the belief that, to be more accessible to the general population, computers must be more proactive in their interactions with people. In human interaction, someone who waits for each command before making any communication attempt would be regarded as uncooperative and unhelpful. In order for a computer to be more proactive and, thus, to bear its part of the burden of initiation in interactions, it must have (1) much more real-time information about its user, and (2) algorithms that select actions based on this information rather than simply on user commands. The computer needs information about the user's current and past emotional, motivational and cognitive state as well as the state of the task at hand. A theory, is needed to guide the development of algorithms that select appropriate actions based on user and task state. This research constitutes the next steps in an attempt by the PI's multidisciplinary team to develop this capability.Proposed research includes: (1) further development of methods to sense user postures, movements, expressions and speech; (2) analysis and fusion of this information to identify and track user states; (3) task state tracking; (4) creating a corpus of emotion- and action-labeled videotapes for use with computer learning; (5) further development of affective communication; (6) development of the basis for human-centered state-based action decisions; and (7) evaluation of computer proaction on human behavior and response. The testbed is an environment for hands-on education in science and engineering, using the Lego Mindstorms construction and robotics environment, with children of middle school age. An emphasis will be on developing proactive computing methods for encouraging interest and conceptual development of minority children and females, who often show lower achievement in science. Although the work will be conducted within an educational environment, the methods developed and studied will be broadly applicable, and this project should serve as an exemplar of the type of work that is needed in other computer-aided situations.

这是五年持续奖的第一年资助。这个项目是基于这样一种信念，即为了让普通民众更容易接触到计算机，计算机必须在与人的交互中更加主动。在人际交往中，在进行任何沟通尝试之前等待每个命令的人将被视为不合作和没有帮助。为了使计算机更主动，从而承担其在交互中启动的部分负担，它必须具有(1)关于其用户的更多实时信息，以及(2)基于该信息而不是简单地根据用户命令来选择操作的算法。计算机需要关于用户当前和过去的情绪、动机和认知状态以及手头任务状态的信息。需要一种理论来指导基于用户和任务状态选择适当操作的算法的开发。这项研究构成了PI多学科团队发展这一能力的下一步。建议的研究包括：(1)进一步开发感知用户姿势、动作、表情和言语的方法；(2)分析和融合这些信息以识别和跟踪用户状态；(3)任务状态跟踪；(4)创建用于计算机学习的情感和动作标记录像带语料库；(5)进一步发展情感交流；(6)开发以人为中心的基于状态的行动决策的基础；以及(7)评价计算机ProAction对人类行为和反应的影响。试验台是一个进行科学和工程实践教育的环境，使用乐高MindStorms建筑和机器人环境，面向中学年龄的孩子。重点将是开发积极主动的计算方法，以鼓励少数族裔儿童和妇女的兴趣和概念发展，因为他们在科学上的成就往往较低。虽然这项工作将在教育环境中进行，但制定和研究的方法将广泛适用，该项目应成为其他计算机辅助情况下所需工作类型的典范。