Implicit mobile human-robot communication for spatial action coordination with context-specific semantic environment modeling

机器人在工业以及工作和日常生活中的使用变得越来越灵活。当前的机器学习和自适应运动规划方法正在导致机器人的更鲁棒的行为和更高的自主性。然而，在人类无法理解机器人的运动行为的情况下，协作的人机交互仍然会发生中断和故障。一个常见的原因是人类对机器人当前感知的内容及其内部状态的了解不正确或有限。如果机器人能够理解并将交互伙伴的心理状态和视角纳入其自身的动作生成中，以主动生成对交互的共同理解，则可以避免这种情况。人类和机器人之间这种协作的关键能力是通过身体语言和运动的隐含信号进行沟通和相互协调的能力。本计画以一个货架的联合建造为例，探讨人与机器人在协同动作中的隐式沟通。在实验研究中，将创建并记录人类和机器人之间的交互和感知受到干扰的情况。一方面，探索新的感知方法，鲁棒地检测基于头部和身体姿势和面部表情对遮挡的交互相关特征。这些在动作和环境的上下文中被解释，使得隐式通信信号（例如，转向、转向、顺从、暗示等）和内部状态（例如，赞成、反对、互动意愿等）可以推断。另一方面，正在探索新的方法，使机器人在自己的行动规划中推断人类对话者的视角和状态，并积极请求用户的反应。这导致了在货架建造过程中，考虑到相互感知和行动目标的合作伙伴的空间协调。通过主动使用机器人的身体姿势、相对取向和运动，可以提前解决冲突情况，而不需要对机器人的明确指令。

The use of robots in the industry as well as in the work and everyday life is becoming more and more flexible. Current methods for machine learning and adaptive motion planning are leading to a more robust behavior and a higher autonomy of the robots. Nevertheless, collaborative human-robot interactions still happen to have interruptions and breakdowns in cases where the human is not able to comprehend the robot's movement behavior. A common cause is that the human has an incorrect or limited picture of what the robot is currently perceiving and what its internal state is. This could be avoided if the robot could understand and incorporate the mental states and the perspective of the interaction partner in its own action generation in order to actively generate a common understanding of the interaction.A key competence for such a collaboration between humans and robots is the ability of communication and mutual coordination via implicit signals of body language and movement. The project investigates the implicit human-robot communication in collaborative actions by using the example of the joint construction of a shelf. In experimental studies, situations will be created and recorded in which the interaction and perception between the human and the robot is disturbed. On the one hand, new perception methods are explored, that robustly detect interaction-relevant features based on head and body poses and facial expressions against occlusions. These are interpreted in the context of the action and the environment, so that implicit communication signals (e.g., turning toward, turning away, compliance, hinting, etc.) and internal states (e.g., approval, disapproval, willingness to interact, etc.) can be inferred. On the other hand, new methods are being explored to make the robot infer the perspective and the state of the human interlocutor in its own action planning and actively requests user reactions.This leads to a spatial coordination of the partners during the construction of the shelf by taking into consideration the mutual perception and the goal of the action. Via an active use of body pose, relative orientation and movement of the robot, conflict situations can be solved in advance without the need for explicit instructions to the robot.