RI: Medium: An Interaction Manager for Language and Force Exhanges in Human-Robot Physical Collaboration

该项目将为机器人助手开发一个计算和数据驱动的框架，使其在涉及物理交互的日常任务中与人类合作，例如一起移交或移动物体。使用从观察人类执行这些任务中学习到的模型，机器人将进行来回交流，其中轮流可以是口头表达和力量交换。可以协作完成体力任务的机器人可以在各种环境中提供帮助，例如做家务，支持老年人保持独立，以及在工厂车间协助人类工人。该提案的变革思想是将对话处理的方法推广到包括物理交互。最根本的挑战是如何弥合语言的符号处理和力量交换的低级控制之间的差距。引入交互原语（ip）的概念对物理交互进行建模。此外，还提出了交互管理器形式的计划和执行框架。所建议的交互管理器扩展了传统的对话建模范例，以便信息可以从更抽象的层次流向更低的层次，反之亦然：语言影响物理交互，物理交互影响所说的内容。为了构建交互管理器，将执行一个目标数据集合，其中将执行人工执行感兴趣的任务。由于物理交互数据总是稀疏的，对数据的统计学习将由基于模型的泛化来补充，从而允许机器人在高度可变和非结构化的环境中与人类合作。这项研究将为新课程的开发提供信息，并涉及几名本科生和研究生。

This project will develop a computational and data-driven framework for a robot assistant to collaborate with humans in everyday tasks that involve physical interaction, such as handing over or moving an object together. Using models learned from observing humans perform such tasks, the robot will engage in back-and-forth communication, where turns can be both spoken utterances and force exchanges. Robots that can collaborate to perform physical tasks could provide assistance in a variety of settings, such as performing household chores, supporting the elderly to remain independent, and assisting human workers on the factory floor.The transformative idea of the proposal is to generalize the methodology of dialog processing to include physical interaction. The fundamental challenge is how to bridge the gap between the symbolic processing of language and the low-level control of force exchanges. The concept of interaction primitives (IPs) is introduced to model physical interactions. Further, a planning and execution framework in the form of an interaction manager is proposed. The proposed interaction manager broadens the traditional dialogue modeling paradigm so that information can flow from more abstract to lower levels, and vice versa: language affects physical interaction, and physical interaction affects what is said. To build the interaction manager, a targeted data collection where humans perform tasks of interest will be performed. Since physical interaction data is invariably sparse, statistical learning on the data will be complemented by model-based generalizations, allowing robots to collaborate with humans in highly variable and unstructured environments. The research will inform new course development, and involve several undergraduate and graduate students.