In-Bed Pose Estimation: Deep Learning With Shallow Dataset
In-Bed Pose Estimation: Deep Learning With Shallow Dataset
复制标题
DOI：
10.1109/jtehm.2019.2892970
复制
发表时间：
2019-01-01
影响因子：
3.4
通讯作者：
Ostadabbas, Sarah
中科院分区：
文献类型：
--
作者：
Liu, Shuangjun;Yin, Yu;Ostadabbas, Sarah
本文提出了一种鲁棒的人体姿态和身体部位检测方法下的一个特定的应用场景称为在床上姿势估计。虽然在过去的几十年中，各种计算机视觉（CV）应用中的人体姿态估计已经得到了广泛的研究，但使用基于相机的视觉方法的床内姿态估计已经被CV社区所忽略，因为它被假设为与通用姿态估计问题相同。然而，床内姿态估计具有其自身的专门方面并且伴随有特定挑战，包括全天照明条件的显著差异以及具有不同于常见人类监视视角的姿态分布。在本文中，我们证明了这些挑战显着降低了现有的通用姿态估计模型的有效性。为了应对光照变化的挑战，提出了红外选择性（IRS）图像采集技术，以在各种光照条件下提供均匀质量的数据。此外，针对非常规的姿态透视，提出了一种双端方向梯度直方图（HOG）校正方法。深度学习框架被证明是人体姿势估计中最有效的模型;然而，缺乏床上姿势的大型公共数据集使我们无法从头开始使用大型网络。在本文中，我们探索了采用在一般人类姿势的大型公共数据集上训练的预训练卷积神经网络（CNN）模型，并使用我们自己的浅（大小有限，视角和颜色不同）床上IRS数据集微调模型的想法。我们开发了一个IRS成像系统，并收集IRS图像数据，从几个现实的真人大小的人体模型在模拟医院的房间环境。预先训练的CNN称为卷积姿态机（CPM），通过重新训练其特定的中间层来微调床内姿态估计。使用HOG校正方法，CPM的姿态估计性能显着提高了26.4%，在正确的关键点（PCK）标准的概率在PCK 0.1相比，没有这样的校正模型。即使只使用对齐良好的床内姿势图像进行测试，我们的微调模型仍然超过了传统调整的CNN，姿势估计准确度提高了16.6%。
This paper presents a robust human posture and body parts detection method under a specific application scenario known as in-bed pose estimation. Although the human pose estimation for various computer vision (CV) applications has been studied extensively in the last few decades, the in-bed pose estimation using camera-based vision methods has been ignored by the CV community because it is assumed to be identical to the general purpose pose estimation problems. However, the in-bed pose estimation has its own specialized aspects and comes with specific challenges, including the notable differences in lighting conditions throughout the day and having pose distribution different from the common human surveillance viewpoint. In this paper, we demonstrate that these challenges significantly reduce the effectiveness of the existing general purpose pose estimation models. In order to address the lighting variation challenge, the infrared selective (IRS) image acquisition technique is proposed to provide uniform quality data under various lighting conditions. In addition, to deal with the unconventional pose perspective, a 2-end histogram of oriented gradient (HOG) rectification method is presented. The deep learning framework proves to be the most effective model in human pose estimation; however, the lack of large public dataset for in-bed poses prevents us from using a large network from scratch. In this paper, we explored the idea of employing a pre-trained convolutional neural network (CNN) model trained on large public datasets of general human poses and fine-tuning the model using our own shallow (limited in size and different in perspective and color) in-bed IRS dataset. We developed an IRS imaging system and collected IRS image data from several realistic life-size mannequins in a simulated hospital room environment. A pre-trained CNN called convolutional pose machine (CPM) was fine-tuned for in-bed pose estimation by re-training its specific intermediate layers. Using the HOG rectification method, the pose estimation performance of CPM improved significantly by 26.4% in the probability of correct key-point (PCK) criteria at PCK0.1 compared to the model without such rectification. Even testing with only well aligned in-bed pose images, our fine-tuned model still surpassed the traditionally tuned CNN by another 16.6% increase in pose estimation accuracy.