RI: Small: Using Prediction to Build a Compact Visual Memex Memory for Rapid Analysis and Understanding of Egocentric Video Data

该项目为以自我为中心的(第一人称)视频流分析开发了新的数据驱动技术，该技术利用在几天、几个月甚至几年捕获的流中的结构和冗余，在不丢失最有用的视觉信息的情况下显著减少这些数据集的大小。与此同时，研究团队正在开发并行编程框架，以简化这些视频分析算法的大规模表达和加速。虽然这项研究的重点是核心算法和系统的设计，但成功意味着能够开发新的应用程序类别(在导航、个人辅助、健康/行为监测等领域)，这些应用程序使用摄像机的广泛视觉历史来智能地解释连续的可视数据源，并立即响应观察到的输入。这项研究的进一步成果是收集和组织了一个来自个人生活的以自我为中心的大型视频数据库。本研究的核心思想是识别和利用日常生活中的冗余。虽然保持摄像机所见过的所有视频的易于分析的表示形式并不容易，但很可能识别并为未来的应用程序提供对最重要的可视信息的快速访问。挑战在于确定哪些视觉数据是最重要的。这项工作探索了视频流可预测性作为重要性概念的使用。具体地说，相机的庞大视觉历史(例如，由头戴式相机捕捉的生活经历)被用来预测相机接下来将看到什么，这些预测的准确性决定了保留什么数据。(高度可预测的事件被认为在数据库中保留的价值较低。)此外，这项研究正在表征始终在线的以自我为中心的视频流的结构(一个人一天的“工作集”是什么？每天收集多少新信息？)，利用这种结构来为视频语料库分析(数据压缩、加速检索)的新算法的设计提供信息，并探索专门的编程抽象的设计，以便大规模地创作可视数据理解应用程序。URL：http://graphics.cs.cmu.edu/projects/egocentricPrediction

This project develops new data-driven techniques for egocentric (first-person) video stream analysis that exploit the structure and redundancy in streams captured over days, months, and even years, to significantly reduce the size of these datasets without losing the most useful visual information. Simultaneously, the research team is developing parallel programming frameworks that simplify expression and acceleration of these video analysis algorithms at scale. While the focus of this research is the design of core algorithms and systems, success stands to enable the development of new classes of applications (in domains such as navigation, personal assistance, health/behavior monitoring) that use the extensive visual history of a camera to intelligently interpret continuous visual data sources and immediately respond to the observed input. A further output of this research is the collection and organization of a large egocentric video database from the life of a single individual.The core idea of this research is to identify and exploit redundancy in everyday life. While it is not tractable to maintain an easily analyzable representation of all video ever seen by a camera, it is likely possible to identify and provide future applications fast access to the most important visual information. The challenge is to determine what visual data is the most important. This work explores the use of video stream predictability as a notion of importance. Specifically, the vast visual history of the camera (e.g., life experiences captured by a head-mounted camera) is used to make predictions about what the camera will see next, and the accuracy of these predictions dictates what data is retained. (Highly predictable occurrences are judged to be less valuable to retain in the database.) In addition, this research is characterizing the structure of always-on egocentric video streams (What is the "working set" of a person's day? How much novel information is collected from day to day?), leveraging this structure to inform the design of new algorithms for video corpus analysis (data compression, accelerated retrieval), and exploring the design of specialized programming abstractions for authoring visual data understanding applications at scale.URL: http://graphics.cs.cmu.edu/projects/egocentricPrediction