CRII: RI: Understanding Activities of Daily Living in Indoor Scenarios

据预测，到2050年，世界上每六个人中就有一个将超过65岁。这比2019年的十一分之一有所上升。因此，老年人是社会中一个不断增长的人口群体。由于老龄化与医疗保健利用率的增加有关，流行病也是如此，人口的增加意味着需要增加医疗保健领域的劳动力。通过部署活动监测系统，可以应对不断增长的医疗保健需求，这可以帮助监测老年患者的健康状况，并支持早期发现健康问题。因此，构建这样的监测系统需要自动理解人类进行的日常生活活动（ADL）。由于计算机视觉的进步，大多数对人类活动建模的研究都是针对一般的互联网视频。现有的模型是为网络视频中的识别而制造的，而描述ADL特征的视点变化和细微运动通常不能处理不确定性，并且在现实世界的场景中往往表现不佳。此外，他们很难区分相似的活动。因此，本项目的主要目标是建立一个框架，以识别可部署在监测系统中的ADL。该项目还将开展补充教育和外展活动，吸引STEM学生。该项目将开发一个多模态框架，主要基于RGB颜色模型和姿势模态，因为它们在室内场景中易于访问。该框架旨在解决两个重要的挑战-有限的可用性标记的ADL视频和如何联合收割机不同的异构模式（RGB和姿势）分类活动。因此，本项目将探索两个相互关联的研究方向的整合：（1）从有限的训练分布中学习的研究;（2）结合RGB和深度等模式的研究。 在第一项研究中，该项目将探索减轻ADL领域可用数据规模限制的可能性，以有效训练用于视频理解的神经网络。在第二项研究中，该团队的目标是通过利用场景中的人类局部区域来提高基于RGB的人类活动识别的有效性。最后，我们将开发一个多模态的神经网络模型ADL整合人类本地化的RGB和3D构成的一个人演员。这项研究将揭示理解ADL的几个新维度，这也将使计算机视觉社区受益。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

It is projected that, by 2050, one-in-six people in the world will be over the age of 65. This is up from one-in-11 in 2019. Thus, older adults are a growing demographic group in society. Because aging is related to increased healthcare utilization, as are pandemics, this increasing demographic translates into the need to increase the workforce in healthcare. The rising demand for healthcare can be combated by deploying activity monitoring systems, which could help monitor the health status of older patients and support the early detection of health issues. Thus, building such monitoring systems requires an automated understanding of Activities of Daily Living (ADL) performed by humans. Most of the investigations towards modeling human activities owing to the advances in computer vision are targeted for generic internet videos. Existing models are fabricated for recognition in web videos whereas viewpoint variation and subtle motion that characterize ADLs generally cannot handle uncertainty and tend to underperform in real-world scenarios. Moreover, they have difficulties distinguishing similarly looking activities. Thus, the key objective of this project is to build a framework for recognizing ADLs that can be deployed in monitoring systems. The project will also perform complementary educational and outreach activities that engage STEM students.This project will develop a multi-modal framework predominantly based on the RGB color model and pose modalities due to their easy accessibility in indoor scenarios. This framework aims at addressing two important challenges - the limited availability of labeled ADL videos and how to combine different heterogeneous modalities (RGB and Poses) for classifying activities. Thus, this project will explore the integration of two interrelated research directions: (1) a study on learning from limited training distribution; and, (2) a study on combining modalities like RGB and depth. In the first study, the project will explore the possibilities of mitigating the limitation of the scale of available data in the ADL domain for effective training of neural networks for video understanding. In the second study, the team will aim at improving the effectiveness of RGB-based human activity recognition by leveraging the human localized regions within a scene. Finally, we will develop a multi-modal neural network model for ADL by integrating human localized RGB and 3D poses of a human actor. This research study will reveal several new dimensions towards understanding ADLs that will also benefit the computer vision community.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.