RI: Small: Using Prediction to Build a Compact Visual Memex Memory for Rapid Analysis and Understanding of Egocentric Video Data

该项目开发了用于以自我为中心（第一人称）视频流分析的新数据驱动技术，该技术利用了在几天，几个月甚至几年内捕获的流中的结构和冗余，以显着减少这些数据集的大小，而不会丢失最有用的视觉信息。同时，研究团队正在开发并行编程框架，以简化这些视频分析算法的表达和加速。 虽然这项研究的重点是核心算法和系统的设计，但成功意味着能够开发新的应用程序（在导航，个人辅助，健康/行为监测等领域），这些应用程序使用相机的广泛视觉历史来智能地解释连续的视觉数据源并立即响应观察到的输入。 本研究的另一个成果是收集和组织了一个以自我为中心的视频数据库，其核心思想是识别和利用日常生活中的冗余。虽然维护一个摄像机所看到的所有视频的易于分析的表示是不容易的，但有可能识别并为未来的应用程序提供对最重要的视觉信息的快速访问。挑战在于确定哪些视觉数据是最重要的。 这项工作探讨了使用视频流的可预测性作为一个重要的概念。具体地说，相机的巨大视觉历史（例如，由头戴式摄像机捕获的生活经历）被用于预测摄像机接下来将看到什么，并且这些预测的准确性决定了保留什么数据。（高度可预测的事件被认为在数据库中保留的价值较低。）此外，这项研究还描述了以自我为中心的视频流的结构（什么是一个人一天的“工作集”？每天收集多少新信息？），利用这种结构为视频语料库分析（数据压缩，加速检索）的新算法的设计提供信息，并探索大规模创作视觉数据理解应用程序的专业编程抽象的设计。http://graphics.cs.cmu.edu/projects/egocentricPrediction

This project develops new data-driven techniques for egocentric (first-person) video stream analysis that exploit the structure and redundancy in streams captured over days, months, and even years, to significantly reduce the size of these datasets without losing the most useful visual information. Simultaneously, the research team is developing parallel programming frameworks that simplify expression and acceleration of these video analysis algorithms at scale. While the focus of this research is the design of core algorithms and systems, success stands to enable the development of new classes of applications (in domains such as navigation, personal assistance, health/behavior monitoring) that use the extensive visual history of a camera to intelligently interpret continuous visual data sources and immediately respond to the observed input. A further output of this research is the collection and organization of a large egocentric video database from the life of a single individual.The core idea of this research is to identify and exploit redundancy in everyday life. While it is not tractable to maintain an easily analyzable representation of all video ever seen by a camera, it is likely possible to identify and provide future applications fast access to the most important visual information. The challenge is to determine what visual data is the most important. This work explores the use of video stream predictability as a notion of importance. Specifically, the vast visual history of the camera (e.g., life experiences captured by a head-mounted camera) is used to make predictions about what the camera will see next, and the accuracy of these predictions dictates what data is retained. (Highly predictable occurrences are judged to be less valuable to retain in the database.) In addition, this research is characterizing the structure of always-on egocentric video streams (What is the "working set" of a person's day? How much novel information is collected from day to day?), leveraging this structure to inform the design of new algorithms for video corpus analysis (data compression, accelerated retrieval), and exploring the design of specialized programming abstractions for authoring visual data understanding applications at scale.URL: http://graphics.cs.cmu.edu/projects/egocentricPrediction