LISI - Learning to Imitate Nonverbal Communication Dynamics for Human-Robot Social Interaction

我们正在接近一个未来，机器人将逐渐在我们日常生活的许多方面普及，包括教育，医疗保健，工作和个人使用。所有这些实际应用都要求人类和机器人在人类环境中一起工作，在人类环境中，社会互动是不可避免的。沿着语言交流，成功的社会互动与非语言感知和行动机制之间的相互作用密切相关，例如观察一个人的凝视行为并跟随他们的注意力，协调手臂手势的形式和功能。人类以一种本能和适应性的方式进行社会互动，不需要付出任何努力。因此，机器人要想在我们的社交领域取得成功，就应该以类似人类的方式参与社交互动，并提高自主性水平。尽管人机交互和社交机器人领域呈指数级增长，但当前社交机器人的能力仍然有限。首先，大多数的交互上下文已经通过远程操作，即人类操作员远程控制机器人处理。然而，随着机器人在我们的社会中变得越来越普遍，这种方法将是劳动密集型和不切实际的。其次，考虑到问题的复杂性，通过手动编程每个行为来设计交互逻辑是非常困难的。一旦固定，它将是有限的，不可转移到看不见的交互上下文，并且对来自机器人环境的不可预测输入（例如，传感器噪声）。数据驱动的方法是解决这些缺点的一条很有前途的道路，因为建模人机交互是设计每个人都可以使用和理解的人机交互界面的最自然的指南。本项目的目的是（1）开发新的方法，用于从数据中自主学习人机交互的原理，并使用计算机视觉和机器学习技术通过机器人学习模仿这些原理，以及（2）将这些方法协同集成到真实的人形机器人的感知和控制中。该项目将为下一代机器人奠定基础，这些机器人将能够通过观看人机交互视频来学习如何与人类合作。

We are approaching a future where robots will progressively become widespread in many aspects of our daily lives, including education, healthcare, work and personal use. All of these practical applications require that humans and robots work together in human environments, where social interaction is unavoidable. Along with verbal communication, successful social interaction is closely coupled with the interplay between nonverbal perception and action mechanisms, such as observation of one's gaze behaviour and following their attention, coordinating the form and function of hand-arm gestures. Humans perform social interaction in an instinctive and adaptive manner, with no effort. For robots to be successful in our social landscape, they should therefore engage in social interactions in a human-like manner, with increasing levels of autonomy. Despite the exponential growth in the fields of human-robot interaction and social robotics, the capabilities of current social robots are still limited. First, most of the interaction contexts has been handled through tele-operation, whereby a human operator controls the robot remotely. However, this approach will be labour-intensive and impractical as the robots become more commonplace in our society. Second, designing interaction logic by manually programming each behaviour is exceptionally difficult, taking into account the complexity of the problem. Once fixed, it will be limited, not transferrable to unseen interaction contexts, and not robust to unpredicted inputs from the robot's environment (e.g., sensor noise). Data-driven approaches are a promising path for addressing these shortcomings as modelling human-human interaction is the most natural guide to designing human-robot interaction interfaces that can be usable and understandable by everyone. This project aims (1) to develop novel methods for learning the principles of human-human interaction autonomously from data and learning to imitate these principles via robots using the techniques of computer vision and machine learning, and (2) to synergistically integrate these methods into the perception and control of real humanoid robots. This project will set the basis for the next generation of robots that will be able to learn how to act in concert with humans by watching human-human interaction videos.