HRI: Perceptually Situated Human-Robot Dialog Models

HRI：感知情境人机对话模型

• 批准号: 0819984
• 负责人: Trevor Darrell
• 金额: $ 81.5万
• 依托单位: International Computer Science Institute
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2008
• 资助国家: 美国
• 起止时间: 2008-01-01 至 2012-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0819984&HistoricalAwards=false
• 关键词: HRI; Perceptually; Situated; Human; Robot

Humans naturally use dialog and gestures to discuss complex phenomena and plans, especially when they refer to physical aspects of the environment while they communicate with each other. Existing robot vision systems can sense people and the environment, but are limited in their ability to detect the detailed conversational cues people often rely upon (such as head pose, eye gaze, and body gestures), and to exploit those cues in multimodal conversational dialog. Recent advances in computer vision have made it possible to track such detailed cues. Robots can use passive measures to sense the presence of people, estimate their focus of attention and body pose, and to recognize human gestures and identify physical references. But they have had limited means of integrating such information into models of natural language; heretofore, they have used dialog models for specific domains and/or were limited to one-on-one interaction. Separately, recent advances in natural language processing have led to dialog models that can track relatively free-form conversation among multiple participants, and extract meaningful semantics about people's intentions and actions. These multi-party dialog models have been used in meeting environments and other domains. In this project, the PI and his team will fuse these two lines of research to achieve a perceptually situated, natural conversation model that robots can use to interact multimodally with people. They will develop a reasonably generic dialog model that allows a situated agent to track the dialog around it, know when it is being addressed, and take direction from a human operator regarding where it should find or place various objects, what it should look for in the environment, and which individuals it should attend to, follow, or obey. Project outcomes will extend existing dialog management techniques to a more general theory of interaction management, and will also extend current state-of-the-art vision research to be able to recognize the subtleties of nonverbal conversational cues, as well as methods for integrating those cues with ongoing dialog interpretation and interaction with the world.Broader Impacts: There are clearly many positive societal impacts that will derive from this research. Ultimately, development of effective human-robot interfaces will allow greater deployment of robots to perform dangerous tasks that humans would otherwise have to perform, and will also enable greater use of robots for service tasks in domestic environments. As part of the project, the PI will conduct outreach efforts to engage secondary-school students in the hope that exposure to HRI research may increase their interest in science and engineering studies.

人类自然地使用对话和手势来讨论复杂的现象和计划，特别是当他们在相互交流时提到环境的物理方面时。 现有的机器人视觉系统可以感知人和环境，但在检测人们经常依赖的详细会话线索（如头部姿势，眼睛凝视和身体姿势）以及在多模态会话对话中利用这些线索的能力方面受到限制。 计算机视觉的最新进展使得跟踪这些细节线索成为可能。 机器人可以使用被动措施来感知人的存在，估计他们的注意力和身体姿势，并识别人类手势和识别物理参考。 但是，他们有有限的手段将这些信息集成到自然语言的模型;迄今为止，他们已经使用特定领域的对话模型和/或仅限于一对一的互动。 另外，自然语言处理的最新进展已经产生了对话模型，可以跟踪多个参与者之间相对自由形式的对话，并提取有关人们意图和行为的有意义的语义。 这些多方对话模型已被用于会议环境和其他领域。 在这个项目中，PI和他的团队将融合这两条研究路线，以实现一个感知定位的自然对话模型，机器人可以使用它与人进行多模式交互。 他们将开发一个合理的通用对话模型，允许一个定位的代理跟踪它周围的对话，知道它何时被寻址，并从人类操作员那里得到关于它应该在哪里找到或放置各种对象的指示，它应该在环境中寻找什么，以及它应该关注，跟随或服从哪些人。 项目成果将把现有的对话管理技术扩展到更一般的互动管理理论，也将扩展当前最先进的视觉研究，使其能够识别非语言对话线索的微妙之处，以及将这些线索与正在进行的对话解释和与世界的互动相结合的方法。显然，这项研究将产生许多积极的社会影响。 最终，有效的人机界面的开发将允许更多地部署机器人来执行人类必须执行的危险任务，并且还将使机器人能够在家庭环境中更多地用于服务任务。 作为该项目的一部分，PI将开展外展工作，让中学生参与，希望接触HRI研究可以增加他们对科学和工程研究的兴趣。