Enabling rich, open-ended human-robot interaction through robust, advanced multimodal perceptual capabilities for high-level reasoning

人-机器人交互的最终目标是拥有自主的机器人，能够自然地与人互动，在家里或工作中帮助他们的日常生活，并自动适应新的情况。建造这样的机器人带来了大量的科学挑战：安全、自然和有效的交互意味着先进的感知能力，为健全和健壮的推理提供信息。随着人类在动态环境中运作，机器人将需要对我们生活的世界有广泛的理解，并需要推理能力，而不是通常僵化的、预定义的场景，它们目前的编程是针对这些场景的。 我的研究项目旨在实现人类和自主移动机器人之间丰富的、开放的互动。要做到这一点，需要高水平的推理，我认为只有在机器人具有大规模和多模式感知能力的情况下，这才是可能的。为了提供这种能力，正如神经科学中关于选择性注意(SA)的研究所证实的那样，越来越多的证据表明，大脑中发生了自上而下的调制，通过根据动态环境中不断变化的任务需求和期望对已经编码的项目进行优先排序来维持工作记忆的性能。一个旨在进行丰富交互的机器人将不得不适应这样的环境，并对需要在自己的工作内存中编码的内容进行优先排序。此外，它还必须从对过去经验的长期记忆中预测感知需求，并为未来预期的刺激管理其计算资源。 为了研究这种机制，我将在现有开源技术的基础上，建立一个大规模分布式感知处理的框架，将多个固定和移动机器人嵌入传感器的刺激结合起来，并平衡多个计算系统的资源。这样做是为了研究机器人如何建立对周围环境的多模式理解。然后，受SA启发的机制还将过滤在工作和长期记忆中编码的感知，以避免计算资源超载。最后，为了超越反应性资源管理并为预期的未来事件做准备，将设计一种预见性监督机制，以推断从过去的经验中预期哪些刺激。 这项研究计划的验证将使用轮式类人机器人进行，这些机器人在现实环境中具有先进的操作和传感能力。计划参加诸如RoboCup@Home这样的国际比赛，以作为评估与该领域最先进技术有关的机制附加值的共同基础。 这一研究计划代表着一个独特的机会，可以研究具体化人工智能(AI)，以响应和预测真实的与人类互动环境中的事件。该项目将包括面向硕士和博士生的多个项目，以将他们培训为应用人工智能和机器人方面的专家。

The ultimate goal in Human-Robot Interaction is to have autonomous robots that can naturally interact with people, assist them in their daily lives at home or at work, and automatically adapt to new situations. Building such robots brings a great number of scientific challenges: safe, natural and effective interaction implies advanced perceptual capabilities to supply information for sound and robust reasoning. As humans operate in dynamic environments, robots will need a broad understanding of the world we live in and reasoning capabilities outside of the usually rigid, pre-defined scenarios they are currently programmed for. My research program is oriented toward enabling rich, open-ended interactions between humans and autonomous mobile robots. Doing so require high-level reasoning, which I believe is only possible with large-scale and multimodal perceptual capabilities on robots. To provide such capabilities, and as identified by studies on selective attention (SA) in neurosciences, there is growing evidence that top-down modulation occurs in the brain to maintain performance of the working memory by prioritizing already encoded items depending on the changing tasks demands and expectations of dynamic environments. A robot meant for rich interaction will have to adapt itself to such environments and prioritize what needs to be encoded in its own working memory. Furthermore, it will have to anticipate perceptual requirements from long-term memories of past experiences and manage its computing resources for future expected stimuli. To study such mechanisms, I will build a framework for distributed perceptual processing at a large scale based on existing open-source technologies to combine the stimuli of multiple stationary and mobile robot-embedded sensors and balancing the resources of multiple computing systems. This will be done to investigate how a robot can build multimodal understandings of their surroundings. Then, an SA-inspired mechanism will also filter the percepts encoded in both working and long-term memories to avoid overloading computing resources. Finally, to go beyond reactive resource management and prepare for expected future events, an anticipatory supervision mechanism will be designed to infer which stimuli to expect from past experiences. Validation of this research program will be done with wheeled humanoid robots with advanced manipulation and sensing capabilities in realistic settings. Participating to an international competition such as RoboCup@Home is planned to serve as common ground for evaluation of the added value of the mechanisms developed in relation to state-of-the-art in the field. This research program represents a unique opportunity to study embodied artificial intelligence (AI) to respond and anticipate to events in real interactive environments with humans. The program will include multiple projects for MSc and PhD students to train them as experts in applied AI and robotics.