Robust Activity Recognition for Adaptive Worker-Robot Interaction using Transfer Learning
Robust Activity Recognition for Adaptive Worker-Robot Interaction using Transfer Learning
复制标题
DOI：
10.48550/arxiv.2308.14843
复制
发表时间：
2023-08
期刊：
影响因子：
--
通讯作者：
Farid Shahnavaz;Riley Tavassoli;Reza Akhavian
中科院分区：
文献类型：
--
作者：
Farid Shahnavaz;Riley Tavassoli;Reza Akhavian
使用机器学习的人类活动识别（HAR）在检测建筑工人的活动方面显示出巨大的前景。HAR在人机交互研究中有许多应用，使机器人能够理解人类同行的活动。然而，许多现有的HAR方法缺乏鲁棒性，通用性和适应性。本文提出了一种用于建筑工人活动识别的迁移学习方法，该方法需要数量级更少的数据和计算时间，以获得可比或更好的分类精度。开发的算法从原始作者预先训练的模型中转移特征，并对它们进行微调，以用于施工活动识别的下游任务。该模型在Kinetics-400上进行了预训练，Kinetics-400是一个大规模的基于视频的人类活动识别数据集，具有400个不同的类别。该模型进行了微调，并使用从YouTube上找到的手动材料处理（MMH）活动中捕获的视频进行测试。结果表明，微调模型可以识别不同的MMH任务，在一个强大的和自适应的方式，这是至关重要的协作机器人在建筑中的广泛部署。
Human activity recognition (HAR) using machine learning has shown tremendous promise in detecting construction workers' activities. HAR has many applications in human-robot interaction research to enable robots' understanding of human counterparts' activities. However, many existing HAR approaches lack robustness, generalizability, and adaptability. This paper proposes a transfer learning methodology for activity recognition of construction workers that requires orders of magnitude less data and compute time for comparable or better classification accuracy. The developed algorithm transfers features from a model pre-trained by the original authors and fine-tunes them for the downstream task of activity recognition in construction. The model was pre-trained on Kinetics-400, a large-scale video-based human activity recognition dataset with 400 distinct classes. The model was fine-tuned and tested using videos captured from manual material handling (MMH) activities found on YouTube. Results indicate that the fine-tuned model can recognize distinct MMH tasks in a robust and adaptive manner which is crucial for the widespread deployment of collaborative robots in construction.