Advanced methods for depth-based human pose estimation and motion analysis: application to vital signs monitoring in the intensive care unit

基于深度的人体姿势估计和运动分析的先进方法：应用于重症监护病房的生命体征监测

• 批准号: RGPIN-2020-06695
• 负责人: Seoud, Lama
• 金额: $ 1.75万
• 依托单位: École Polytechnique de Montréal
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2021
• 资助国家: 加拿大
• 起止时间: 2021-01-01 至 2022-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=741265
• 关键词: Advanced; methods; depth; based; human

Despite the mature and affordable solutions offered today in commoditized range sensors such as Microsoft's Kinect with its skeletal tracking algorithm, several challenges remain regarding depth-based human pose and motion analysis. Existing algorithms rely mostly on supervised learning with training data obtained in almost ideal conditions. Considerations with regards to range sensing technology, to subject's clothing and to visual occlusions are barely taken into account in the training. Another question remains regarding the performance of these methods when the subject is sitting or even lying down. Moreover, historically, the primary focus in human motion analysis has been on estimating and/or tracking human body joints over time. Despite its popularity, joint-based representation does not offer enough resolution to analyze fine and local motion such as human tremors, respiratory motion and nodding for example. There is a need in moving beyond the stick-figure view of the human body toward recovering richer descriptions of shape and motion. Building on my previous work, the long term objective of this research program is to propose novel dense representations of human pose and motion from depth images that allow a robust analysis of both fine-local and ample-global motion in real-time and in unconstrained environment. To achieve this, we will develop and validate novel computational tools and methods that address the challenges related to range sensor variability, to posture variability, to fine motion description and to severe occlusions. These tools will then be applied in the context of patients monitoring in the pediatric intensive care unit (PICU) for real-time detection of signs of vital distress. The program is composed of four specific objectives: (SO1) to enhance the robustness of depth-based human pose estimation to sensors specific artifacts and to bed-ridden postures, (SO2) to propose a new semantic dense motion descriptor from a sequence of depth images while taking into account the strategies developed in SO1, (SO3) to propose new strategies for human pose estimation and motion tracking in the presence of sever occlusions, (SO4) to detect abnormal head and limb movements in PICU patients by applying the tools developed in SO2 and SO3. These objectives will train 2 PhD, 3 master and 5 undergraduate students. By tackling important technical challenges, the proposed program will lead to major advances in human pose estimation and motion analysis. The computational tools will benefit a wide variety of applications such as robotics, surveillance, gaming and advanced manufacturing. In-bed motion analysis has received little interest so far although it brings innovative value into the market of commoditized child monitoring systems. Finally, this research program will have a direct impact on the quality of care in the PICU by preventing management delays, improving medical staff's efficiency and thus patients' outcome.

尽管如今商品化的距离传感器（如带有骨骼跟踪算法的微软Kinect）提供了成熟且经济的解决方案，但在基于深度的人体姿势和运动分析方面仍存在一些挑战。现有的算法主要依赖于监督学习，训练数据在几乎理想的条件下获得。训练中几乎没有考虑到距离传感技术、受试者的服装和视觉遮挡。另一个问题是，当受试者坐着甚至躺着时，这些方法的效果如何。此外，从历史上看，人体运动分析的主要焦点一直是随着时间的推移估计和/或跟踪人体关节。尽管它很受欢迎，但基于关节的表示并不能提供足够的分辨率来分析细微的局部运动，例如人体震颤、呼吸运动和点头。有必要超越对人体的简笔画观点，恢复对形状和运动的更丰富的描述。在我之前工作的基础上，这个研究项目的长期目标是从深度图像中提出新颖的人体姿势和运动的密集表示，允许在实时和不受约束的环境中对精细局部和样本全局运动进行鲁棒分析。为了实现这一目标，我们将开发和验证新的计算工具和方法，以解决与距离传感器可变性、姿态可变性、精细运动描述和严重遮挡相关的挑战。然后，这些工具将应用于儿科重症监护病房（PICU）患者监测的背景下，以实时检测生命窘迫的迹象。该计划由四个具体目标组成：（SO1）增强基于深度的人体姿态估计对传感器特定伪像和卧床姿势的鲁棒性；（SO2）在考虑SO1中开发的策略的同时，从深度图像序列中提出新的语义密集运动描述符；（SO3）提出严重闭塞情况下人体姿态估计和运动跟踪的新策略；（SO4）应用SO2和SO3中开发的工具检测PICU患者的头部和肢体异常运动。这些目标将培养2名博士，3名硕士和5名本科生。通过解决重要的技术挑战，该计划将在人体姿态估计和运动分析方面取得重大进展。计算工具将有利于各种各样的应用，如机器人、监视、游戏和先进制造。床上运动分析虽然给商品化的儿童监测系统市场带来了创新价值，但迄今为止还没有引起多大的兴趣。最后，本研究项目将对PICU的护理质量产生直接影响，防止管理延误，提高医护人员的工作效率，从而提高患者的预后。