EAGER: Smart Space-Time Sampling for Recovering and Recognizing Dynamic Scenes

传统的动态场景被捕获为在规则的时空网格处采样的视频帧。然而，对于许多计算机视觉任务，这种均匀采样可能是低效的（例如，低光、高速运动）或不必要的（例如，运动/变化/事件检测）。该项目探讨了非均匀的自适应采样方案，该方案利用了时空体积的底层结构（例如，稀疏性、时间相干性、统计先验）。这些采样方案是通过相机中新颖的可编程像素编码曝光和光圈来实现的。所捕获的信息丰富的编码投影的空间-时间体积用于视频重建或直接作为运动/事件检测的功能。除了更高的成像效率和更高的重建结果的信噪比之外，该方法还提供了用于视频监控的数据安全性和隐私保护方面的益处，因为解码捕获的图像需要编码模式和字典的知识。这项研究在监控、机器视觉检测和高速成像方面有许多应用。该技术正在交通监控和事故检测的交通成像中进行测试。一个交通场景和事件的高速视频数据库正在被捕获，并计划在完成后在线发布。除了视频之外，该技术方法还可以适用于其他高维信号，例如光场或光传输矩阵。

Traditional, dynamic scenes are captured as video frames sampled at regular space-time grids. For many computer vision tasks, however, this uniform sampling may be either inefficient (e.g., low light, high-speed motion) or unnecessary (e.g., motion/change/event detection). This project explores non-uniform, adaptive sampling schemes that exploit the underlying structures of space-time volumes (e.g., sparsity, temporal coherence, statistical priors). These sampling schemes are implemented with novel programmable pixel-wised coded exposure and aperture in cameras. The captured information-rich coded projections of space-time volumes are used for video reconstruction or directly as features for motion/event detection. In addition to higher efficiency in imaging and higher signal-to-noise ratio in reconstructed results, the method also provides benefits in data security and privacy protection for video surveillance because decoding the captured images requires the knowledge of coded patterns and dictionaries. This research has many applications in surveillance, machine vision inspection, and high-speed imaging. The developed technology is being tested in transportation imaging for traffic monitoring and accident detection. A database of high-speed videos of traffic scenes and events is being captured and plan to be released online when it is finished. In addition to videos, the technical approach can also be applicable to other high-dimensional signals such as light fields or light transport matrices.