NeuroPose: 3D Hand Pose Tracking using EMG Wearables
NeuroPose: 3D Hand Pose Tracking using EMG Wearables
复制标题
DOI：
10.1145/3442381.3449890
复制
发表时间：
2021-04
期刊：
影响因子：
--
通讯作者：
Yilin Liu;Shijia Zhang;Mahanth K. Gowda
中科院分区：
文献类型：
--
作者：
Yilin Liu;Shijia Zhang;Mahanth K. Gowda
无处不在的手指运动跟踪在增强现实、运动分析、康复-医疗保健、触觉等领域具有许多令人兴奋的应用。本文介绍了NeuroPose系统，该系统展示了使用可穿戴式肌电(EMG)传感器平台进行3D手指运动跟踪的可行性。肌电传感器可以感知由于手指激活而产生的肌肉电位，从而为细粒度的手指运动传感提供丰富的信息。然而，由于来自多个手指的信号以复杂的图案叠加在传感器处，因此将传感器信息转换为3D手指姿势并不容易。为了解决这一问题，NeuroPose融合了手指运动的解剖学约束信息与基于递归神经网络(RNN)、编解码器网络和ResNet的机器学习结构，以从噪声EMG数据中提取3D手指运动。生成的运动模式在时间上是平滑的，并且在解剖学上是一致的。此外，利用转移学习算法以最小的训练开销使一个用户的预训练模型适应新用户。一项有12名用户参与的系统研究显示，在跟踪3D手指关节角度时，中位误差为6.24°，90%误差为18.33°。该精度对传感器安装位置的自然变化以及用户手腕位置的变化具有很强的鲁棒性。NeuroPose在智能手机上实现，处理延迟为0.101s，能量开销较低。
Ubiquitous finger motion tracking enables a number of exciting applications in augmented reality, sports analytics, rehabilitation-healthcare, haptics etc. This paper presents NeuroPose, a system that shows the feasibility of 3D finger motion tracking using a platform of wearable ElectroMyoGraphy (EMG) sensors. EMG sensors can sense electrical potential from muscles due to finger activation, thus offering rich information for fine-grained finger motion sensing. However converting the sensor information to 3D finger poses is non trivial since signals from multiple fingers superimpose at the sensor in complex patterns. Towards solving this problem, NeuroPose fuses information from anatomical constraints of finger motion with machine learning architectures on Recurrent Neural Networks (RNN), Encoder-Decoder Networks, and ResNets to extract 3D finger motion from noisy EMG data. The generated motion pattern is temporally smooth as well as anatomically consistent. Furthermore, a transfer learning algorithm is leveraged to adapt a pretrained model on one user to a new user with minimal training overhead. A systematic study with 12 users demonstrates a median error of 6.24° and a 90%-ile error of 18.33° in tracking 3D finger joint angles. The accuracy is robust to natural variation in sensor mounting positions as well as changes in wrist positions of the user. NeuroPose is implemented on a smartphone with a processing latency of 0.101s, and a low energy overhead.