权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

EAGER: Vision-Based Activity Forecasting by Mining Temporal Causalities

EAGER：通过挖掘时间因果关系进行基于视觉的活动预测

基本信息

批准号：
1651902
负责人：
Yun Fu
金额：
$ 18万
依托单位：
Northeastern University
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2016
资助国家：
美国
起止时间：
2016-09-01 至 2019-08-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1651902&HistoricalAwards=false
关键词：
EAGER Vision Based Activity Forecasting

项目摘要

This project explores methodologies for forecasting long-term human group activity from videos. Forecasting future activities in real-world videos is an emerging computer vision problem with important applications in visual surveillance for security. This project systematically and rigorously formulates long-term group activity forecasting problem as causalities of visual entities, and designs visual intelligence systems using machine learning and data mining methods. The research considers multiple visual identities of different types, models their interactions, and mines the sequential causalities between these visual entities in videos. These causalities are used for forecasting future group and individual activities. The project creates new mathematical models for describing and simplifying understanding statistical properties of human activity videos. The project leads to important and timely technology that can help to design future video analysis systems with optimal performance in understanding and searching activities from videos. The project tightly integrates research and education activities for the purpose of providing young researchers with project-based learning opportunities in an interdisciplinary environment that offers exceptional professional and personal growth opportunities. This research discovers complex causality patterns between visual entities from noisy visual data, in order to gain rich and useful knowledge for the forecasting of future visual activities. This essentially bridges the gap between human understandable visual semantics and high-dimensional noisy visual data. The research enables to efficiently capture interactions between multiple visual entities and their temporal causalities, and provides rich knowledge for guiding long-term group activity forecasting. The project also explores several innovative ways to leverage rich sequential context and builds progress level-invariant features. This naturally enriches feature representations from temporally partially observed data, and allows building more time efficient activity prediction machines. Moreover, the project develops an effective forecasting model that can elegantly utilize causalities mined from visual data for long-term forecasting. The developed technologies can lead to new intelligent systems.

这个项目探索了从视频中预测长期人类群体活动的方法。预测真实世界视频中的未来活动是一个新兴的计算机视觉问题，在安全视觉监控中有着重要的应用。本项目系统而严谨地将长期群体活动预测问题描述为视觉实体的因果关系，并利用机器学习和数据挖掘方法设计视觉智能系统。该研究考虑了不同类型的多个视觉身份，对它们之间的交互进行建模，并挖掘视频中这些视觉实体之间的顺序因果关系。这些因果关系被用来预测未来的群体和个人活动。该项目创建了新的数学模型，用于描述和简化对人类活动视频的统计属性的理解。该项目带来了重要和及时的技术，可以帮助设计未来的视频分析系统，在理解和搜索视频活动方面具有最佳性能。该项目将研究和教育活动紧密结合在一起，旨在为年轻的研究人员提供基于项目的学习机会，在一个提供非凡的职业和个人成长机会的跨学科环境中。本研究从噪声视觉数据中发现视觉实体之间复杂的因果关系模式，为未来视觉活动的预测提供丰富而有用的知识。这在本质上弥合了人类可理解的视觉语义和高维噪声视觉数据之间的差距。该研究能够有效地捕捉多个视觉实体之间的相互作用及其时间因果关系，并为指导长期的群体活动预测提供丰富的知识。该项目还探索了几种创新的方法来利用丰富的顺序上下文并构建进度级别不变的功能。这自然地丰富了来自时间部分观察数据的特征表示，并允许构建更具时间效率的活动预测机器。此外，该项目还开发了一种有效的预测模型，可以优雅地利用从视觉数据中挖掘的因果关系进行长期预测。所开发的技术可以带来新的智能系统。

项目成果

期刊论文数量（74）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

Early Recognition of 3D Human Actions

DOI：
10.1145/3131344
发表时间：
2018-03
期刊：
ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM)
影响因子：
0
作者：
Sheng Li;Kang Li;Y. Fu
通讯作者：
Sheng Li;Kang Li;Y. Fu

Learning Consensus Representation for Weak Style Classification