Advanced methods for depth-based human pose estimation and motion analysis: application to vital signs monitoring in the intensive care unit

基于深度的人体姿势估计和运动分析的先进方法：应用于重症监护病房的生命体征监测

• 批准号: RGPIN-2020-06695
• 负责人: Seoud, Lama
• 金额: $ 1.75万
• 依托单位: École Polytechnique de Montréal
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=754214
• 关键词: Advanced; methods; depth; based; human

Despite the mature and affordable solutions offered today in commoditized range sensors such as Microsoft's Kinect with its skeletal tracking algorithm, several challenges remain regarding depth-based human pose and motion analysis. Existing algorithms rely mostly on supervised learning with training data obtained in almost ideal conditions. Considerations with regards to range sensing technology, to subject's clothing and to visual occlusions are barely taken into account in the training. Another question remains regarding the performance of these methods when the subject is sitting or even lying down. Moreover, historically, the primary focus in human motion analysis has been on estimating and/or tracking human body joints over time. Despite its popularity, joint-based representation does not offer enough resolution to analyze fine and local motion such as human tremors, respiratory motion and nodding for example. There is a need in moving beyond the stick-figure view of the human body toward recovering richer descriptions of shape and motion. Building on my previous work, the long term objective of this research program is to propose novel dense representations of human pose and motion from depth images that allow a robust analysis of both fine-local and ample-global motion in real-time and in unconstrained environment. To achieve this, we will develop and validate novel computational tools and methods that address the challenges related to range sensor variability, to posture variability, to fine motion description and to severe occlusions. These tools will then be applied in the context of patients monitoring in the pediatric intensive care unit (PICU) for real-time detection of signs of vital distress. The program is composed of four specific objectives: (SO1) to enhance the robustness of depth-based human pose estimation to sensors specific artifacts and to bed-ridden postures, (SO2) to propose a new semantic dense motion descriptor from a sequence of depth images while taking into account the strategies developed in SO1, (SO3) to propose new strategies for human pose estimation and motion tracking in the presence of sever occlusions, (SO4) to detect abnormal head and limb movements in PICU patients by applying the tools developed in SO2 and SO3. These objectives will train 2 PhD, 3 master and 5 undergraduate students. By tackling important technical challenges, the proposed program will lead to major advances in human pose estimation and motion analysis. The computational tools will benefit a wide variety of applications such as robotics, surveillance, gaming and advanced manufacturing. In-bed motion analysis has received little interest so far although it brings innovative value into the market of commoditized child monitoring systems. Finally, this research program will have a direct impact on the quality of care in the PICU by preventing management delays, improving medical staff's efficiency and thus patients' outcome.

尽管今天在商品化的距离传感器中提供了成熟且负担得起的解决方案，例如微软的Kinect及其骨骼跟踪算法，但在基于深度的人体姿势和运动分析方面仍然存在一些挑战。现有的算法主要依赖于监督学习，训练数据几乎是在理想的条件下获得的。在训练中几乎没有考虑到距离传感技术、受试者的服装和视觉遮挡。另一个问题是，当受试者坐着甚至躺下时，这些方法的性能如何。此外，历史上，人体运动分析的主要焦点一直是随着时间的推移估计和/或跟踪人体关节。尽管其受欢迎程度，基于关节的表示不提供足够的分辨率来分析精细和局部运动，例如人体震颤，呼吸运动和点头。有必要超越对人体的简笔画观点，恢复对形状和运动的更丰富的描述。 在我以前的工作的基础上，这个研究项目的长期目标是从深度图像中提出人类姿势和运动的新的密集表示，允许在实时和无约束的环境中对精细局部和广泛全局运动进行鲁棒分析。为了实现这一目标，我们将开发和验证新的计算工具和方法，解决与距离传感器的变化，姿态变化，精细运动描述和严重闭塞的挑战。然后，这些工具将应用于儿科重症监护室（PICU）中的患者监测，以实时检测生命体征。 该方案包括四个具体目标：（SO 1）增强基于深度的人类姿势估计对传感器特定伪影和卧床姿势的鲁棒性，（SO2）根据深度图像序列提出新的语义密集运动描述符，同时考虑在SO 1中开发的策略，（SO 3）在存在严重遮挡的情况下提出用于人体姿势估计和运动跟踪的新策略，（SO 4）通过应用在SO2和SO 3中开发的工具来检测PICU患者中的异常头部和肢体运动。这些目标将培养2名博士，3名硕士和5名本科生。 通过解决重要的技术挑战，拟议的计划将导致人类姿势估计和运动分析的重大进展。计算工具将有利于各种各样的应用，如机器人，监控，游戏和先进制造。尽管床内运动分析为商品化的儿童监控系统市场带来了创新价值，但迄今为止，它几乎没有引起人们的兴趣。最后，这项研究计划将有一个直接的影响，在PICU的护理质量，防止管理延误，提高医务人员的效率，从而提高病人的结果。