Development of Smartphone Application for Markerless Three-Dimensional Motion Capture Based on Deep Learning Model.
Development of Smartphone Application for Markerless Three-Dimensional Motion Capture Based on Deep Learning Model.
复制标题
DOI：
10.3390/s22145282
复制
发表时间：
2022-07-14
期刊：
影响因子：
--
通讯作者：
中科院分区：
文献类型：
--
作者：
为了定量评估病理步态，我们开发了一种新的智能手机应用程序，用于使用智能手机单目摄像头和深度学习从基于无标记视频的图像中真实的实时全身人体运动跟踪。作为深度学习的训练数据，准备了原始三维（3D）数据集，包括来自90个人形角色的3D运动的100多万张捕获图像和COCO 2017的二维数据集。3D热图偏移数据由整个身体运动的24个关键点处的28 × 28 × 28个块组成，具有红、绿、蓝三种颜色，是使用改进的ResNet 34卷积神经网络学习的。在每个关键点处，使用tanh函数学习偏离单元中心的最热点。我们的新iOS应用程序可以在真实的时间内检测以肚脐为中心的24个全身关键点的相对三轴坐标，而无需任何标记来进行动作捕捉。通过使用相对坐标，颈部，腰椎，双侧髋关节，膝关节和踝关节的三维角度进行了估计。任何人体运动都可以使用一种名为三维姿势跟踪器步态测试（TDPT-GT）的新智能手机应用程序进行定量和轻松评估，而无需任何身体标记或多点摄像头。
To quantitatively assess pathological gait, we developed a novel smartphone application for full-body human motion tracking in real time from markerless video-based images using a smartphone monocular camera and deep learning. As training data for deep learning, the original three-dimensional (3D) dataset comprising more than 1 million captured images from the 3D motion of 90 humanoid characters and the two-dimensional dataset of COCO 2017 were prepared. The 3D heatmap offset data consisting of 28 × 28 × 28 blocks with three red–green–blue colors at the 24 key points of the entire body motion were learned using the convolutional neural network, modified ResNet34. At each key point, the hottest spot deviating from the center of the cell was learned using the tanh function. Our new iOS application could detect the relative tri-axial coordinates of the 24 whole-body key points centered on the navel in real time without any markers for motion capture. By using the relative coordinates, the 3D angles of the neck, lumbar, bilateral hip, knee, and ankle joints were estimated. Any human motion could be quantitatively and easily assessed using a new smartphone application named Three-Dimensional Pose Tracker for Gait Test (TDPT-GT) without any body markers or multipoint cameras.
登录
查看更多内容
DOI：
10.3390/s21041356
发表时间：
2021-02-14
期刊：
Sensors (Basel, Switzerland)
影响因子：
--
作者：
Büker LC;Zuber F;Hein A;Fudickar S
通讯作者：
Fudickar S
影响因子：
2.4
作者：
Kitade, Ippei;Kitai, Ryuhei;Matsumine, Akihiko
通讯作者：
Matsumine, Akihiko
影响因子：
7.4
作者：
Yamada, Shigeki;Aoyagi, Yukihiko;Ishikawa, Masatsune
通讯作者：
Ishikawa, Masatsune
DOI：
10.3390/s16020194
发表时间：
2016-02-04
期刊：
Sensors (Basel, Switzerland)
影响因子：
--
作者：
Springer S;Yogev Seligmann G
通讯作者：
Yogev Seligmann G
DOI：
10.3390/s21103496
发表时间：
2021-05-17
期刊：
Sensors (Basel, Switzerland)
影响因子：
--
作者：
Wang L;Li Y;Xiong F;Zhang W
通讯作者：
Zhang W