RI: Small: A Unified Compositional Model for Explainable Video-based Human Activity Parsing

RI：小型：用于可解释的基于视频的人类活动解析的统一组合模型

• 批准号: 1815561
• 负责人: Ying Wu
• 金额: $ 44.9万
• 依托单位: Northwestern University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-09-01 至 2023-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1815561&HistoricalAwards=false
• 关键词: RI; Small; Unified; Compositional; Model

An ultimate goal of computer vision is understanding scene and activities from images and video. This task involves many perceptual and cognitive processes at various semantic levels. A next step beyond visual classification is visual interpretation, that is, to explain the relations among visual entities through visual inference and reasoning. Due to the enormous variability across instances of this problem, semantic parsing for explaining a visual scene and activities is highly challenging. This project studies how the structural composition of visual entities can be used to overcome the diversity in the visual scene and activities. It advances and enrich the basic research of computer vision, and brings significant impact on many merging applications, including autonomous or assisted driving, intelligent robots, and intelligent video surveillance. This research also contributes to education through curriculum development, student training, and knowledge dissemination. It includes interactions with K-12 students for participation and research opportunities. This research is to develop a unified visual compositional model that can effectively learn complex semantic concepts in a scalable end-to-end fashion, while achieving good generalizability and providing explainable parsing of the visual data. The project is focused on: (1) a principled model and its theoretical foundation, by designing a stochastic grammar based on the probabilistic And/Or-Graph to model the structural composition; (2) an effective computational approach for learning and parsing, by exploiting data-driven pattern mining to discover structural components and by exploring how the patterns may be self-formed; (3) a solid case study on video human activity parsing and interpretation, by inferring the complex compositions of human actions, body movements, and interaction with the environment; and (4) tools and prototype systems for human articulated body pose estimation, contextual object discovery, and video-based human activity analysis and interpretation.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

计算机视觉的最终目标是从图像和视频中理解场景和活动。这一任务涉及到许多不同语义层次的知觉和认知过程。视觉分类的下一步是视觉解释，即通过视觉推理和推理来解释视觉实体之间的关系。由于这个问题的实例之间存在巨大的差异，用于解释视觉场景和活动的语义解析是非常具有挑战性的。本项目研究如何利用视觉实体的结构组合来克服视觉场景和活动的多样性。它推动和丰富了计算机视觉的基础研究，并对许多融合应用产生了重大影响，包括自动驾驶或辅助驾驶，智能机器人和智能视频监控。这项研究也有助于通过课程开发，学生培训和知识传播的教育。它包括与K-12学生的参与和研究机会的互动。 这项研究是开发一个统一的视觉组合模型，可以有效地学习复杂的语义概念，在一个可扩展的端到端的方式，同时实现良好的泛化能力，并提供可解释的分析的视觉数据。该项目的重点是：（1）通过设计一种基于概率与或图的随机文法来模拟结构组成，建立了一个原则模型及其理论基础：（2）通过利用数据驱动的模式挖掘来发现结构成分，并探索模式是如何自形成的，建立了一种有效的学习和分析计算方法;（3）通过推断人类动作、身体运动以及与环境的交互的复杂组成，对视频人类活动解析和解释进行了坚实的案例研究;以及（4）用于人类关节式身体姿态估计，上下文对象发现，视频-该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值进行评估来支持和更广泛的影响审查标准。

项目成果

Order-Preserving Wasserstein Discriminant Analysis

• DOI: 10.1109/iccv.2019.00998
• 发表时间: 2019-10
• 期刊: 2019 IEEE/CVF International Conference on Computer Vision (ICCV)
• 作者: Bing Su;Jiahuan Zhou;Ying Wu

Unsupervised Depth Completion and Denoising for RGB-D Sensors

RGB-D 传感器的无监督深度补全和去噪

• DOI: 10.1109/icra46639.2022.9812392
• 发表时间: 2022
• 期刊: 2022 International Conference on Robotics and Automation (ICRA
• 作者: Fan, Lei;Li, Yunxuan;Jiang, Chen;Wu, Ying
• 通讯作者: Wu, Ying

Balancing Between Forgetting and Acquisition in Incremental Subpopulation Learning

• DOI: 10.1007/978-3-031-19809-0_21
• 发表时间: 2022
• 作者: Mingfu Liang;Jiahuan Zhou;Wei Wei-Wei;Yingying Wu

Online Joint Multi-Metric Adaptation From Frequent Sharing-Subset Mining for Person Re-Identification

• DOI: 10.1109/cvpr42600.2020.00298
• 发表时间: 2020-06
• 期刊: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
• 作者: Jiahuan Zhou;Bing Su;Ying Wu

Learning Meta-Distance for Sequences by Learning a Ground Metric via Virtual Sequence Regression

通过虚拟序列回归学习基本度量来学习序列的元距离

• DOI: 10.1109/tpami.2020.3010568
• 发表时间: 2022
• 期刊: IEEE Transactions on Pattern Analysis and Machine Intelligence
• 影响因子: 23.6
• 作者: Su, Bing;Wu, Ying
• 通讯作者: Wu, Ying