LISI - Learning to Imitate Nonverbal Communication Dynamics for Human-Robot Social Interaction

我们正在接近这样一个未来，机器人将逐渐在我们日常生活的许多方面得到广泛应用，包括教育、医疗保健、工作和个人使用。所有这些实际应用都需要人类和机器人在人类环境中共同工作，在人类环境中，社交互动是不可避免的。除了语言交流，成功的社会互动还与非语言感知和行动机制之间的相互作用密切相关，如观察一个人的凝视行为并跟踪他们的注意力，协调手势的形式和功能。人类以一种本能和适应性的方式进行社交，不费力气。因此，为了让机器人在我们的社会环境中取得成功，它们应该以类似人类的方式参与社会互动，并拥有越来越多的自主性。尽管在人-机器人交互和社交机器人领域取得了指数级的增长，但目前社交机器人的能力仍然有限。首先，大多数交互情景都是通过远程操作来处理的，由此操作员可以远程控制机器人。然而，随着机器人在我们的社会中变得越来越普遍，这种方法将是劳动密集型和不切实际的。其次，考虑到问题的复杂性，通过手动编程每个行为来设计交互逻辑是异常困难的。一旦修复，它将是有限的，不能转移到不可见的交互环境中，并且对来自机器人环境的不可预测的输入(例如，传感器噪声)不是很健壮。数据驱动的方法是解决这些缺点的一条有希望的途径，因为建立人与人之间的交互模型是设计每个人都可以使用和理解的人-机器人交互界面的最自然的指南。该项目的目标是(1)开发新的方法，用于从数据中自主地学习人与人交互的原理，并利用计算机视觉和机器学习技术通过机器人学习模仿这些原理，以及(2)将这些方法协同地集成到真实的类人机器人的感知和控制中。该项目将为下一代机器人奠定基础，这些机器人将能够通过观看人与人的互动视频来学习如何与人类协调行动。

We are approaching a future where robots will progressively become widespread in many aspects of our daily lives, including education, healthcare, work and personal use. All of these practical applications require that humans and robots work together in human environments, where social interaction is unavoidable. Along with verbal communication, successful social interaction is closely coupled with the interplay between nonverbal perception and action mechanisms, such as observation of one's gaze behaviour and following their attention, coordinating the form and function of hand-arm gestures. Humans perform social interaction in an instinctive and adaptive manner, with no effort. For robots to be successful in our social landscape, they should therefore engage in social interactions in a human-like manner, with increasing levels of autonomy. Despite the exponential growth in the fields of human-robot interaction and social robotics, the capabilities of current social robots are still limited. First, most of the interaction contexts has been handled through tele-operation, whereby a human operator controls the robot remotely. However, this approach will be labour-intensive and impractical as the robots become more commonplace in our society. Second, designing interaction logic by manually programming each behaviour is exceptionally difficult, taking into account the complexity of the problem. Once fixed, it will be limited, not transferrable to unseen interaction contexts, and not robust to unpredicted inputs from the robot's environment (e.g., sensor noise). Data-driven approaches are a promising path for addressing these shortcomings as modelling human-human interaction is the most natural guide to designing human-robot interaction interfaces that can be usable and understandable by everyone. This project aims (1) to develop novel methods for learning the principles of human-human interaction autonomously from data and learning to imitate these principles via robots using the techniques of computer vision and machine learning, and (2) to synergistically integrate these methods into the perception and control of real humanoid robots. This project will set the basis for the next generation of robots that will be able to learn how to act in concert with humans by watching human-human interaction videos.