Towards Multi-User Activity Recognition through Facilitated Training Data and Deep Learning for Human-Robot Collaboration Applications
Towards Multi-User Activity Recognition through Facilitated Training Data and Deep Learning for Human-Robot Collaboration Applications
复制标题
DOI：
10.1109/ijcnn54540.2023.10191782
复制
发表时间：
2023-02
期刊：
影响因子：
--
通讯作者：
F. Semeraro;Jonathan Carberry;A. Cangelosi
中科院分区：
文献类型：
--
作者：
F. Semeraro;Jonathan Carberry;A. Cangelosi
人-机器人交互(HRI)研究正在逐步解决多方场景，即机器人同时与多个人类用户交互。相反，人类-机器人协作的研究仍处于早期阶段使用机器学习技术来处理这种类型的协作需要的数据比典型的HRC设置中的数据更不可行。这项工作概述了用于非二元HRC应用的并发任务的场景。基于这些概念，本研究还提出了一种收集关于多用户活动的数据的替代方法，通过收集与单个用户相关的数据并在后处理中将其合并，以减少产生配对设置的记录所涉及的工作量。为了验证这一说法，收集了单个用户活动的3D骨架姿势，并成对合并。之后，使用这些数据点分别训练一个长短期记忆(LSTM)网络和一个由时空图卷积网络(STGCN)组成的变分自动编码器(VAE)，以识别这两个人对的联合活动。结果表明，可以将以这种方式收集的数据用于配对HRC设置，并且与使用关于在相同设置下记录的用户组的训练数据相比，可以获得与使用关于在相同设置下记录的用户组的训练数据类似的性能，从而消除了产生这些数据涉及的技术困难。相关代码和收集的数据可公开获得11代码存储库位于：https://github.com/francescosemeraro/Multi_User_through_Single_User.git.数据存储库可从以下网址获得：https://figshare.com/s/64bd023e968e6eb096f3..
Human-robot interaction (HRI) research is progressively addressing multi-party scenarios, where a robot interacts with more than one human user at the same time. Conversely, research is still at an early stage for human-robot collaboration The use of machine learning techniques to handle such type of collaboration requires data that are less feasible to produce than in a typical HRC setup. This work outlines scenarios of concurrent tasks for non-dyadic HRC applications. Based upon these concepts, this study also proposes an alternative way of gathering data regarding multi-user activity, by collecting data related to single users and merging them in post-processing, to reduce the effort involved in producing recordings of pair settings. To validate this statement, 3D skeleton poses of activity of single users were collected and merged in pairs. After this, such datapoints were used to separately train a long short-term memory (LSTM) network and a variational autoencoder (VAE) composed of spatio-temporal graph convolutional networks (STGCN) to recognise the joint activities of the pairs of people. The results showed that it is possible to make use of data collected in this way for pair HRC settings and get similar performances compared to using training data regarding groups of users recorded under the same settings, relieving from the technical difficulties involved in producing these data. The related code and collected data are publicly available11The code repository is available at: https://github.com/francescosemeraro/Multi_User_through_Single_User.git. The data repository is available at: https://figshare.com/s/64bd023e968e6eb096f3..