Markerless Motion Tracking With Noisy Video and IMU Data
Markerless Motion Tracking With Noisy Video and IMU Data
复制标题
DOI：
10.1109/tbme.2023.3275775
复制
发表时间：
2023-05
影响因子：
4.6
通讯作者：
Soyong Shin;Zhixiong Li;Eni Halilaj
中科院分区：
文献类型：
--
作者：
Soyong Shin;Zhixiong Li;Eni Halilaj
Objective: Marker-based motion capture, considered the gold standard in human motion analysis, is expensive and requires trained personnel. Advances in inertial sensing and computer vision offer new opportunities to obtain research-grade assessments in clinics and natural environments. A challenge that discourages clinical adoption, however, is the need for careful sensor-to-body alignment, which slows the data collection process in clinics and is prone to errors when patients take the sensors home. Methods: We propose deep learning models to estimate human movement with noisy data from videos (VideoNet), inertial sensors (IMUNet), and a combination of the two (FusionNet), obviating the need for careful calibration. The video and inertial sensing data used to train the models were generated synthetically from a marker-based motion capture dataset of a broad range of activities and augmented to account for sensor-misplacement and camera-occlusion errors. The models were tested using real data that included walking, jogging, squatting, sit-to-stand, and other activities. Results: On calibrated data, IMUNet was as accurate as state-of-the-art models, while VideoNet and FusionNet reduced mean $\pm$ std root-mean-squared errors by 7.6 $\pm$ 5.4$^{\circ }$ and 5.9 $\pm$ 3.3$^{\circ }$, respectively. Importantly, all the newly proposed models were less sensitive to noise than existing approaches, reducing errors by up to 14.0 $\pm$ 5.3$^{\circ }$ for sensor-misplacement errors of up to 30.0 $\pm$ 13.7$^{\circ }$ and by up to 7.4 $\pm$ 5.5$^{\circ }$ for joint-center-estimation errors of up to 101.1 $\pm$ 11.2 mm, across joints. Conclusion: These tools offer clinicians and patients the opportunity to estimate movement with research-grade accuracy, without the need for time-consuming calibration steps or the high costs associated with commercial products such as Theia3D or Xsens, helping democratize the diagnosis, prognosis, and treatment of neuromusculoskeletal conditions.
