Anticipative Human-Robot Collaboration (P8)

预期人机协作 (P8)

• 批准号: 332518894
• 负责人: Professor Dr. Sven Behnke
• 金额: --
• 依托单位: Informatik 6: Intelligente Systeme und Robotik
• 依托单位国家: 德国
• 项目类别: Research Units
• 财政年份: 2017
• 资助国家: 德国
• 起止时间: 2016-12-31 至 2020-12-31
• 项目状态: 已结题
• 来源: https://gepris.dfg.de/gepris/projekt/332518894?language=en
• 关键词: Anticipative; Human; Robot; Collaboration; P8

Effective human-robot collaboration requires the robot system to observe human actions and to make predictions on the future state of the collaborative work space, in order to generate anticipatory robot behavior. The objective of this project is to model a shared human-robot workspace, predict semantically meaningful future states on multiple spatio-temporal scales, and plan anticipatory robot actions for realizing human-robot collaboration. We will extend the small work space for collaborative manipulation developed in the first-phase project "Learning Hierarchical Representations for Anticipative Human-Robot Collaboration" to a larger collaborative mobile manipulation scenario. As the work space will be observed from multiple viewpoints, the semantic perception and prediction of individual views must be merged to a 3D allocentric semantic model of the collaborative scene and its changes. We will explicitly model the intrinsic sensor calibrations and the 3D transformations between the sensor coordinate frames and the allocentric scene frame. The scene will be modeled by simple localized 3D representations, like occupancy grids, surfels or distance fields, their semantic attributes like class probabilities, and their changes. The entire model will be a differentiable function graph. The parameters of these transformations and representations will be learned by instantaneously predicting individual views, given the other views that are aggregated in the 3D representation. This will yield e.g. camera pose estimates. The predictive aspects of the representations will be learned by predicting future views from aggregated past views. As the intentions and actions of the agents in the scene, i.e. humans and robots, have important predictive information, predictions of future scene states will be conditioned not only on the current scene state, but also on human or robot intents and actions. This yields a predicted semantic state of the joint human-robot work space on multiple levels of spatio-temporal abstraction, which will be used to plan anticipative robot behavior in a coarse-to-fine abstract-to-concrete manner. We will demonstrate the utility of our approach in collaborative mobile manipulation tasks, where the robot supports the human by providing the needed objects in the right order at the right moment.

有效的人机协作需要机器人系统观察人类的行为，并对协作工作空间的未来状态进行预测，以产生预期的机器人行为。该项目的目标是建立一个共享的人-机器人工作空间模型，在多个时空尺度上预测语义上有意义的未来状态，并规划预期的机器人动作，以实现人-机器人协作。我们将在第一阶段的项目“学习层次表示的预期人机协作”开发的协作操作的小工作空间扩展到一个更大的协作移动的操作场景。由于工作空间将从多个视点观察，因此必须将各个视图的语义感知和预测合并到协作场景及其变化的3D allocentric语义模型中。我们将明确建模的内在传感器校准和传感器坐标系和allocentric场景帧之间的3D变换。场景将通过简单的本地化3D表示进行建模，如占用网格，曲面或距离场，其语义属性如类概率及其变化。整个模型将是一个可微函数图。这些变换和表示的参数将通过即时预测各个视图来学习，给定在3D表示中聚合的其他视图。这将产生例如相机姿态估计。将通过从聚合的过去视图预测未来视图来学习表示的预测方面。由于场景中的代理（即人类和机器人）的意图和动作具有重要的预测信息，因此对未来场景状态的预测将不仅取决于当前场景状态，还取决于人类或机器人的意图和动作。这产生了一个预测的语义状态的联合人类-机器人工作空间的多个层次的时空抽象，这将被用来规划预期的机器人行为，在一个粗略到精细的抽象到具体的方式。我们将展示我们的方法在协作移动的操作任务中的实用性，其中机器人通过在正确的时刻以正确的顺序提供所需的对象来支持人类。