Advanced methods for depth-based human pose estimation and motion analysis: application to vital signs monitoring in the intensive care unit

尽管今天在商品化的距离传感器中提供了成熟和负担得起的解决方案，如微软的Kinect及其骨骼跟踪算法，但在基于深度的人体姿势和运动分析方面仍然存在一些挑战。现有的算法大多依赖于在近乎理想的条件下获得训练数据的监督学习。在培训中，几乎没有考虑到距离传感技术、受试者的衣服和视觉遮挡。另一个问题是当受试者坐着甚至躺着时，这些方法的表现如何。此外，从历史上看，人体运动分析的主要焦点一直是随着时间的推移估计和/或跟踪人体关节。尽管基于关节的表示很受欢迎，但它不能提供足够的分辨率来分析细微的和局部的运动，例如人体震动、呼吸运动和点头。有必要超越对人体的直线条图形的看法，恢复对形状和运动的更丰富的描述。在我以前工作的基础上，这个研究计划的长期目标是从深度图像中提出新的密集的人体姿势和运动表示，从而能够在实时和无约束的环境中对精细局部和充分的全局运动进行稳健的分析。为了实现这一目标，我们将开发和验证新的计算工具和方法，以应对与距离传感器可变性、姿势可变性、精细运动描述和严重遮挡相关的挑战。这些工具随后将应用于儿科重症监护病房(PICU)的患者监测，以实时检测生命危险的迹象。该程序由四个具体目标组成：(SO1)增强基于深度的人体姿势估计对传感器特定伪影和卧床姿势的稳健性，(SO2)根据深度图像序列提出新的语义密集运动描述符，同时考虑到SO1中开发的策略，(SO3)提出在存在严重遮挡的情况下人体姿势估计和运动跟踪的新策略，(SO4)通过应用SO2和SO3中开发的工具来检测PICU患者的异常头部和肢体运动。这些目标将培养2名博士、3名硕士和5名本科生。通过解决重要的技术挑战，拟议的计划将导致人体姿势估计和运动分析的重大进步。这些计算工具将使机器人、监控、游戏和先进制造等广泛的应用受益。尽管床上运动分析为商品化的儿童监控系统市场带来了创新价值，但到目前为止，它几乎没有引起人们的兴趣。最后，这项研究计划将通过防止管理延误，提高医务人员的效率，从而提高患者的预后，对PICU的护理质量产生直接影响。

Despite the mature and affordable solutions offered today in commoditized range sensors such as Microsoft's Kinect with its skeletal tracking algorithm, several challenges remain regarding depth-based human pose and motion analysis. Existing algorithms rely mostly on supervised learning with training data obtained in almost ideal conditions. Considerations with regards to range sensing technology, to subject's clothing and to visual occlusions are barely taken into account in the training. Another question remains regarding the performance of these methods when the subject is sitting or even lying down. Moreover, historically, the primary focus in human motion analysis has been on estimating and/or tracking human body joints over time. Despite its popularity, joint-based representation does not offer enough resolution to analyze fine and local motion such as human tremors, respiratory motion and nodding for example. There is a need in moving beyond the stick-figure view of the human body toward recovering richer descriptions of shape and motion. Building on my previous work, the long term objective of this research program is to propose novel dense representations of human pose and motion from depth images that allow a robust analysis of both fine-local and ample-global motion in real-time and in unconstrained environment. To achieve this, we will develop and validate novel computational tools and methods that address the challenges related to range sensor variability, to posture variability, to fine motion description and to severe occlusions. These tools will then be applied in the context of patients monitoring in the pediatric intensive care unit (PICU) for real-time detection of signs of vital distress. The program is composed of four specific objectives: (SO1) to enhance the robustness of depth-based human pose estimation to sensors specific artifacts and to bed-ridden postures, (SO2) to propose a new semantic dense motion descriptor from a sequence of depth images while taking into account the strategies developed in SO1, (SO3) to propose new strategies for human pose estimation and motion tracking in the presence of sever occlusions, (SO4) to detect abnormal head and limb movements in PICU patients by applying the tools developed in SO2 and SO3. These objectives will train 2 PhD, 3 master and 5 undergraduate students. By tackling important technical challenges, the proposed program will lead to major advances in human pose estimation and motion analysis. The computational tools will benefit a wide variety of applications such as robotics, surveillance, gaming and advanced manufacturing. In-bed motion analysis has received little interest so far although it brings innovative value into the market of commoditized child monitoring systems. Finally, this research program will have a direct impact on the quality of care in the PICU by preventing management delays, improving medical staff's efficiency and thus patients' outcome.