Visualization and summarization of Omnidirectional surveillance Video by Spatio-temporal Indexing

对于全方位视觉传感器提供的监控视频，本研究项目的目标是建立一个新的媒体系统，该系统能够通过协调时间和空间表示来可视化整个内容，并通过收集重要场景来总结视频。为此，我们进行了以下研究课题，并获得了有价值的知识和见解。1）高视觉全向视觉传感器处理工具的开发介绍了一种具有高视觉摄像头的高分辨率全向传感器，并实现了实时视频转换工具，例如生成透视图像，全景图像和变焦图像。2）事件检测和时空索引设计了基于背景减除的人体区域检测算法和基于人体腿部和头部坐标的二维位置估计算法。将IN、OUT、MOVE和STAY看作人的事件，实现了基于事件的时空索引。3）监控视频可视化与摘要原型系统的构建利用基于事件的时空索引技术，构建了一个监控视频可视化与摘要原型系统。为了使观众容易理解发生了什么事件，系统提供了时间线，空间地图，速度图和视频浏览。此外，该系统的有效性证明了它应用于实际的便利店环境。

For surveillance video given from an omnidirectional vision sensor, the goal of this research project is to build a new media system that is capable of visualizing the whole contents by coordinating temporal and spatial representation and summarizing the video by collecting significant scenes. For this purpose, we performed the following research topics and have obtained valuable knowledge and insights.1)Development of processing tools for high-vision omnidirectional vision sensorsA high-resolution ominidirectional sensor with a high-vision camera was introduced and real-time video transformation tools, e.g. generation of perspective images, panoramic images and zoom images were implemented. Also a model of an indoor environment where the sensor was located at its center was constructed.2)Event detection and spatio-temporal indexingAlgorithms for detecting the region of a person with background subtraction and for estimating the two-dimensional position of him/her based on coordinates of his/her legs and head were designed. Viewing IN, OUT, MOVE and STAY as events of the person, we realized event based spatio-temporal indexing. It is confirmed that the spatio-temporal indexing is suitable for surveillance applications.3)Construction of a prototype system for visualization and summarization of surveillance videoWe constructed a prototype system by taking advantage of event based spatio-temporal indexing. To make a viewer easy to understand what event has taken place, the system offers the time-line, spatial map, speed graph and video browsing. In addition, the effectiveness of this system was demonstrated by applying it to actual convenience store environments.