EAPSI: Multimodal Interaction Algorithm for Human-Robot Interaction with Biologically-Inspired Robots

为了让机器人与人类进行物理交互，它们的行为需要本质安全。目前最先进的算法通过使用单模态数据模型（例如，从相机导出的肢体位置）来保持对人类的感知来实现。然而，仅使用单一输入模态的方法通常会忽略其他潜在的关键信息源。此外，由于没有冗余的数据源，它们容易出现噪声、故障或视线问题。该项目将通过开发一种机器学习算法来解决这个问题，该算法可以执行人类意图推断，并使用多模态数据源（如摄像头，加速度计，基于鞋的压力传感器和肌电图数据）确定适当的机器人响应以进行人机交互。这项研究将与大坂大学的Shuhei Ikemoto博士和Koh Hosoda博士合作进行，他们专门研究生物启发的机器人技术和人机交互。该项目得益于池本博士和细田博士的宝贵专业知识，以及大坂大学的一个人类启发的、自动驱动的机器人，该机器人将实现独特的人机交互场景，这在研究机构目前是不可能的。Interaction Primitives是一个最先进的框架，用于建模机器人和人类之间在协作过程中发生的交互，身体活动。然而，这个现有的框架被设计成在低维空间中工作，每个代理具有单个数据模态。当在这个项目中引入额外的模式时，输入数据的维度将迅速增加。与此同时，训练样本的数量将保持不变，因为这在物理上受到可以进行的演示数量的限制。为了解决这一问题，本项目将开发一个扩展的Interaction Primitives，能够对高维、低样本数据集进行密度估计，目标是产生更安全、更准确的交互。该奖项是由美国国家科学基金会（NSF）和日本科学促进会（Japan Society for the Promotion of Science）共同资助的东亚和太平洋夏季研究所（East Asia and Pacific Summer Institutes）项目下的一个奖项，旨在支持美国研究生的夏季研究。

In order for robots to engage in physical interactions with humans, their behavior needs to be intrinsically safe. Current state-of-the-art algorithms do so by maintaining an awareness of humans with single-modality data models, for example limb positions derived from a camera. However, approaches using only a single input modality typically neglect other potentially critical sources of information. In addition, they are prone to noise, failures, or line-sight problems, since no redundant data sources are available. This project will address this issue with the development of a machine learning algorithm which performs human intention inference and determines an appropriate robot response for human-robot interaction using multi-modal data sources, such as camera, accelerometer, shoe-based pressure sensor, and electromyography data. This research will be conducted in collaboration with Dr. Shuhei Ikemoto and Dr. Koh Hosoda at Osaka University, who specialize in biologically-inspired robotics and human-robot interaction. This project benefits from Dr. Ikemoto's and Dr. Hosoda's invaluable expertise, as well as access to a human-inspired, pneumatically-actuated robot located at Osaka University which will enable unique human-robot interaction scenarios that are not currently possible at the researcher's institution.Interaction Primitives are a state-of-the-art framework for modeling the interaction that takes place between a robot and a human during collaborative, physical activities. However, this existing framework is designed to work in low dimensional space with a single data modality for each agent. When introducing additional modalities in this project, the dimensionality of the input data will increase rapidly. At the same time, the number of training samples will remain constant, since that is physically constrained by the number of demonstrations that can be performed. To resolve this issue, this project will develop an extension of Interaction Primitives capable of performing density estimation with high dimension, low sample size data sets with the goal of producing safer, and more accurate interactions.This award under the East Asia and Pacific Summer Institutes program supports summer research by a U.S. graduate student and is jointly funded by NSF and the Japan Society for the Promotion of Science.