EAGER: Vision-Based Activity Forecasting by Mining Temporal Causalities

这个项目探索了从视频中预测长期人类群体活动的方法。预测现实世界视频中的未来活动是一个新兴的计算机视觉问题，在安全视觉监控中有着重要的应用。本项目系统严谨地将长期群体活动预测问题表述为视觉实体的因果关系，并利用机器学习和数据挖掘方法设计视觉智能系统。该研究考虑了不同类型的多个视觉身份，建立了它们之间相互作用的模型，并挖掘了视频中这些视觉实体之间的顺序因果关系。这些因果关系用于预测未来的群体和个人活动。该项目创建了新的数学模型，用于描述和简化对人类活动视频统计特性的理解。该项目带来了重要而及时的技术，可以帮助设计未来的视频分析系统，在理解和搜索视频活动方面具有最佳性能。该项目将研究和教育活动紧密结合，旨在为年轻研究人员提供跨学科环境中基于项目的学习机会，提供卓越的专业和个人成长机会。本研究从嘈杂的视觉数据中发现视觉实体之间复杂的因果关系模式，为预测未来的视觉活动获得丰富而有用的知识。这在本质上弥补了人类可理解的视觉语义和高维噪声视觉数据之间的差距。该研究能够有效地捕捉多个视觉实体之间的相互作用及其时间因果关系，为指导长期群体活动预测提供丰富的知识。该项目还探索了几种创新的方法来利用丰富的顺序上下文并构建进度级别不变的特性。这自然丰富了暂时部分观察到的数据的特征表示，并允许构建更省时的活动预测机器。此外，该项目开发了一个有效的预测模型，可以优雅地利用从视觉数据中挖掘的因果关系进行长期预测。发达的技术可以导致新的智能系统。

This project explores methodologies for forecasting long-term human group activity from videos. Forecasting future activities in real-world videos is an emerging computer vision problem with important applications in visual surveillance for security. This project systematically and rigorously formulates long-term group activity forecasting problem as causalities of visual entities, and designs visual intelligence systems using machine learning and data mining methods. The research considers multiple visual identities of different types, models their interactions, and mines the sequential causalities between these visual entities in videos. These causalities are used for forecasting future group and individual activities. The project creates new mathematical models for describing and simplifying understanding statistical properties of human activity videos. The project leads to important and timely technology that can help to design future video analysis systems with optimal performance in understanding and searching activities from videos. The project tightly integrates research and education activities for the purpose of providing young researchers with project-based learning opportunities in an interdisciplinary environment that offers exceptional professional and personal growth opportunities. This research discovers complex causality patterns between visual entities from noisy visual data, in order to gain rich and useful knowledge for the forecasting of future visual activities. This essentially bridges the gap between human understandable visual semantics and high-dimensional noisy visual data. The research enables to efficiently capture interactions between multiple visual entities and their temporal causalities, and provides rich knowledge for guiding long-term group activity forecasting. The project also explores several innovative ways to leverage rich sequential context and builds progress level-invariant features. This naturally enriches feature representations from temporally partially observed data, and allows building more time efficient activity prediction machines. Moreover, the project develops an effective forecasting model that can elegantly utilize causalities mined from visual data for long-term forecasting. The developed technologies can lead to new intelligent systems.