HCC: Large: Collaborative Research: Human-Robot Dialog for Collaborative Navigation Tasks

这项研究涉及三个机构的研究人员之间的合作。pi预测未来人类和智能机器人将在共享任务上合作。为了实现这一愿景，机器人必须具有足够丰富的任务领域知识，并且这些知识必须能够用于支持人与机器人之间有效通信的方式。导航空间是为数不多的几个任务领域之一，在这些领域中，知识的结构已被充分理解，从而使物理体现的机器人代理成为一个有用的合作者，满足真正的人类需求。在这个项目中，pi将开发和评估一种智能机器人，这种机器人能够真正对人类有用，并且能够与人类就他们共同的任务进行自然对话。混合空间语义层次（HSSH）是一种受人启发的对导航空间知识的多本体表示。HSSH中的空间表示提供了有效的增量学习，在资源限制下的优雅退化，以及不同类型的人机交互的自然接口。语音是使用自然语言与机器人交流的一种自然方式，虽然要求很高。为了保持实时性能，必须组织自然语言理解，以最大限度地减少根据后期信息从早期结论回溯的数量。这个项目将回答三个科学问题。(1)基于实时计算机视觉的hsh框架能否表达与导航任务相关的自然人类环境的各种知识？(2) HSSH表示是否支持空间导航领域有效的自然语言交流？3)我们能否开发有效的人机交互，满足人的需求并提高系统的性能？为此，pi将用两种不同类型的导航机器人进行这项研究，每种机器人都从其旅行经验中学习，并构建一个日益复杂的认知地图：一种智能机器人轮椅，它可以将人类驾驶员带到期望的目的地，另一种远程呈现机器人，当它在环境中导航时，它可以将其感知传递给远程人类驾驶员，这样驾驶员就可以实现虚拟存在，并与他人远程通信。在设计过程中，项目顾问会与潜在用户进行焦点小组讨论。他们还将在整个过程中评估他们实现的系统，创建一个迭代的设计测试周期。更广泛的影响：要想成功，智能机器人不仅要能够感知世界，代表它所学到的东西，做出有用的推断和计划，并有效地行动。它还必须能够与其他代理进行有效的沟通，特别是与人沟通。这种基础知识表示、情境自然语言理解和人机交互之间的融合是智力上的基础，也是本研究的重点。由于空间知识领域是人类知识几乎所有方面的基础，因此项目成果将具有广泛的适用性。这项工作将为在感知（失明或弱视）、认知（发育迟缓或痴呆）或一般虚弱（老年）方面有残疾的人创造行动辅助技术。它还将支持远程办公、远程医疗和搜索救援等远程呈现应用。该项目包括在一些场所向K-12和社区大学学生、K-12教师和公众推广。

This research involves collaboration among investigators at three institutions. The PIs anticipate a future in which humans and intelligent robots will collaborate on shared tasks. To achieve this vision, a robot must have sufficiently rich knowledge of the task domain and that knowledge must be usable in ways that support effective communication between a human and the robot. Navigational space is one of the few task domains where the structure of the knowledge is sufficiently well understood for a physically-embodied robot agent to be a useful collaborator, meeting genuine human needs. In this project, the PIs will develop and evaluate an intelligent robot capable of being genuinely useful to a human, and capable of natural dialog with a human about their shared task.The Hybrid Spatial Semantic Hierarchy (HSSH) is a human-inspired multi-ontology representation for knowledge of navigational space. The spatial representations in the HSSH provide for efficient incremental learning, graceful degradation under resource limitations, and natural interfaces for different kinds of human-robot interactions. Speech is a natural though demanding way to use natural language to communicate with a robot. To maintain real-time performance, natural language understanding must be organized to minimize the amount of backtracking from early conclusions in light of later information. This project will answer three scientific questions.(1) Can the HSSH framework, extended with real-time computer vision, express the kinds of knowledge of natural human environments that are relevant to navigation tasks? (2) Can the HSSH representation support effective natural language communication in the spatial navigation domain? 3) Can we develop effective human-robot interaction that meets the needs of a person and improves the performance of the system?To these ends, the PIs will perform this research with two different kinds of navigational robots, each learning from its travel experiences and building an increasingly sophisticated cognitive map: an intelligent robotic wheelchair which carries its human driver to desired destinations, and a telepresence robot that transmits its perceptions to a remote human driver as it navigates within an environment so the driver can achieve virtual presence and communicate with others remotely. To inform the design process, the PIs will conduct focus groups with potential users. They will also evaluate their implemented systems throughout the process, creating an iterative design-test cycle.Broader Impacts: To be successful, an intelligent robot must not only be able to perceive the world, represent what it learns, make useful inferences and plans, and act effectively. It must also be able to communicate effectively with other agents, and particularly with people. This confluence among grounded knowledge representation, situated natural language understanding, and human-robot interaction is intellectually fundamental, and is the focus of this research. Since the domain of spatial knowledge is foundational for virtually all aspects of human knowledge, project outcomes will have broad applicability. This work will create technologies for mobility assistance for people with disabilities in perception (blindness or low vision), cognition (developmental delay or dementia), or general frailty (old age). It will also support telepresence applications such as telecommuting, telemedicine and search and rescue. The project includes outreach to K-12 and community college students, K-12 teachers, and the public in a number of venues.