Worker Activity Recognition in Smart Manufacturing Using IMU and sEMG Signals with Convolutional Neural Networks
Worker Activity Recognition in Smart Manufacturing Using IMU and sEMG Signals with Convolutional Neural Networks
复制标题
DOI：
10.29007/bld3
复制
发表时间：
2018-08
期刊：
影响因子：
--
通讯作者：
Wenjin Tao;Ze-Hao Lai;M. Leu;Zhaozheng Yin
中科院分区：
文献类型：
--
作者：
Wenjin Tao;Ze-Hao Lai;M. Leu;Zhaozheng Yin
在涉及工人的智能制造系统中，对工人活动的识别可以用于量化和评估工人的表现，并通过增强现实提供现场指示。在本文中，我们提出了一种使用惯性测量单元（IMU）和从 Myo 臂带获得的表面肌电图（sEMG）信号进行活动识别的方法。原始 10 通道 IMU 信号堆叠起来形成信号图像。通过应用离散傅里叶变换（DFT）将该图像转换为活动图像，然后输入卷积神经网络（CNN）进行特征提取，从而产生高级特征向量。使用原始 8 通道 sEMG 信号评估代表肌肉激活水平的另一个特征向量。然后将这两个向量连接起来并用于工作活动分类。建立了工人活动数据集，目前包含装配任务中的 6 个常见活动，即抓取工具/零件、钉子、使用电动螺丝刀、休息臂、旋转螺丝刀和使用扳手。开发的 CNN 模型在此数据集上进行评估，在半半实验和留一实验中分别实现了 98% 和 87% 的识别准确率。
In a smart manufacturing system involving workers, recognition of the worker's activity can be used for quantification and evaluation of the worker's performance, as well as to provide onsite instructions with augmented reality. In this paper, we propose a method for activity recognition using Inertial Measurement Unit (IMU) and surface electromyography (sEMG) signals obtained from a Myo armband. The raw 10-channel IMU signals are stacked to form a signal image. This image is transformed into an activity image by applying Discrete Fourier Transformation (DFT) and then fed into a Convolutional Neural Network (CNN) for feature extraction, resulting in a high-level feature vector. Another feature vector representing the level of muscle activation is evaluated with the raw 8-channel sEMG signals. Then these two vectors are concatenated and used for work activity classification. A worker activity dataset is established, which at present contains 6 common activities in assembly tasks, i.e., grab tool/part, hammer nail, use power-screwdriver, rest arm, turn screwdriver, and use wrench. The developed CNN model is evaluated on this dataset and achieves 98% and 87% recognition accuracy in the half-half and leave-one-out experiments, respectively.