EAPSI: Multimodal Interaction Algorithm for Human-Robot Interaction with Biologically-Inspired Robots

为了让机器人与人类进行身体互动，它们的行为需要本质上是安全的。目前最先进的算法是通过单模态数据模型来保持对人类的感知，例如从相机获取的肢体位置。然而，仅使用单一输入模式的方法通常会忽略其他潜在的关键信息源。此外，由于没有可用的冗余数据源，它们容易出现噪声、故障或视线问题。该项目将通过开发一种机器学习算法来解决这个问题，该算法可以执行人类意图推理，并使用多模态数据源（如摄像头、加速度计、基于鞋子的压力传感器和肌电图数据）确定人机交互的适当机器人响应。这项研究将与大阪大学的Shuhei Ikemoto博士和Koh Hosoda博士合作进行，他们专门研究生物启发机器人和人机交互。该项目得益于Ikemoto博士和细田博士的宝贵专业知识，以及位于大阪大学的以人为灵感的气动驱动机器人，该机器人将实现目前在研究机构不可能实现的独特人机交互场景。交互原语是一个最先进的框架，用于在协作的物理活动中对机器人和人之间发生的交互进行建模。然而，这个现有框架被设计为在低维空间中工作，每个代理使用单一的数据模式。当在本项目中引入额外的模态时，输入数据的维数将迅速增加。同时，训练样本的数量将保持不变，因为它受到可以执行的演示数量的物理限制。为了解决这个问题，这个项目将开发一个交互原语的扩展，能够对高维、低样本大小的数据集执行密度估计，目标是产生更安全、更准确的交互。该奖项隶属于东亚和太平洋暑期研究所项目，由美国国家科学基金会和日本科学促进会共同资助，支持一名美国研究生进行暑期研究。

In order for robots to engage in physical interactions with humans, their behavior needs to be intrinsically safe. Current state-of-the-art algorithms do so by maintaining an awareness of humans with single-modality data models, for example limb positions derived from a camera. However, approaches using only a single input modality typically neglect other potentially critical sources of information. In addition, they are prone to noise, failures, or line-sight problems, since no redundant data sources are available. This project will address this issue with the development of a machine learning algorithm which performs human intention inference and determines an appropriate robot response for human-robot interaction using multi-modal data sources, such as camera, accelerometer, shoe-based pressure sensor, and electromyography data. This research will be conducted in collaboration with Dr. Shuhei Ikemoto and Dr. Koh Hosoda at Osaka University, who specialize in biologically-inspired robotics and human-robot interaction. This project benefits from Dr. Ikemoto's and Dr. Hosoda's invaluable expertise, as well as access to a human-inspired, pneumatically-actuated robot located at Osaka University which will enable unique human-robot interaction scenarios that are not currently possible at the researcher's institution.Interaction Primitives are a state-of-the-art framework for modeling the interaction that takes place between a robot and a human during collaborative, physical activities. However, this existing framework is designed to work in low dimensional space with a single data modality for each agent. When introducing additional modalities in this project, the dimensionality of the input data will increase rapidly. At the same time, the number of training samples will remain constant, since that is physically constrained by the number of demonstrations that can be performed. To resolve this issue, this project will develop an extension of Interaction Primitives capable of performing density estimation with high dimension, low sample size data sets with the goal of producing safer, and more accurate interactions.This award under the East Asia and Pacific Summer Institutes program supports summer research by a U.S. graduate student and is jointly funded by NSF and the Japan Society for the Promotion of Science.