III-COR-Small: Developing Novel Mosaic Generation Methods for Object-Based Multimedia Information Systems

III-COR-Small：为基于对象的多媒体信息系统开发新颖的马赛克生成方法

• 批准号: 0812307
• 负责人: Ramazan Aygun
• 金额: $ 18.94万
• 依托单位: University of Alabama in Huntsville
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2008
• 资助国家: 美国
• 起止时间: 2008-10-01 至 2012-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0812307&HistoricalAwards=false
• 关键词: III; COR; Small; Developing; Novel

This project provides solutions to one of the important components of an interactive, object-based, semantic multimedia information retrieval system: "mosaic" generation. A mosaic can be considered as a static component (or background) of a scene that does not change over a sequence of frames and is obtained by computing the global motion between frames, warping according to the global motion, and then blending the frames. Mosaic generation plays an important role in many applications including object-based coding (where objects in a scene are also coded or compressed independent of regular rectangular frame coding), video compression, video indexing, object tracking, virtual environments, security surveillance, wide-area surveillance, panoramic video, traffic monitoring, object recognition, and human behavior analysis since these applications usually require the subtraction of actual scenes from the background (or the mosaic) to determine the foreground objects. Since traditional mosaic generation methods require object segmentation for videos containing moving objects, they are not suitable for real-time mosaic generation and especially for video encoders that require sprite coding (coding based on layering objects on top of a mosaic or a sprite). This project (i) develops mosaic generation solutions for larger domains of videos; (ii) generates mosaics for videos containing many shots by classifying video shots; and (iii) develops objective evaluation methods for mosaic generation by producing ground-truths. Sprite fusion method blends assertive and conservative sprites that are generated using the aligned frame differences without object segmentation thus eliminating the object occlusion problem especially for videos where the camera tracks an object. Since the sprite fusion method computes the global motion once for a pair of frames and does not require object segmentation, it suits well for real-time mosaic generation or sprite coding for video encoders. As the mosaic is constructed for a sequence, the global motion vectors are stored to be able to generate (or warp) any frame with respect to another frame thus allowing to provide multiple degrees-of-freedom for spatial interactions including up-down, left-right, move forward-backward, and rotation. The multiple degrees-of-freedom with the warping of the current (or active) scene forms the basis of interactive video reproduction by regenerating any frame from the mosaic and then overlaying the active (current scene) on top of the reproduced frame. The educational component of this project includes development of a new course on multimedia that appeals to any freshman students in order to grow interest in computer science. This course covers fundamental concepts in computer science including types of media, color models, storage devices, multimedia authoring, and internet. A multimedia workshop is planned for K-12 students. The results of this research project will be disseminated via Internet (http://www.cs.uah.edu/~raygun/projects/mosaics.htm), including a video database system (video sets and any truth annotation for it) to share test data with other researchers and to encourage open metrics-based evaluations across research institutions. This project will foster the development of photo-realistic visualization systems using interactive video reproduction, with a wide range of applications.

这个项目提供了一个交互式的，基于对象的，语义多媒体信息检索系统的重要组成部分之一的解决方案：“马赛克”的生成。 马赛克可以被认为是场景的静态分量（或背景），其在帧序列上不改变，并且通过计算帧之间的全局运动、根据全局运动进行扭曲、然后混合帧来获得。 镶嵌生成在包括基于对象的编码在内的许多应用中起着重要的作用（其中场景中的对象也独立于常规矩形帧编码而被编码或压缩），视频压缩，视频索引，对象跟踪，虚拟环境，安全监视，广域监视，全景视频，交通监视，对象识别，和人类行为分析，因为这些应用通常需要从背景（或马赛克）中减去实际场景以确定前景对象。由于传统的马赛克生成方法需要对包含移动对象的视频进行对象分割，因此它们不适合实时马赛克生成，特别是不适合需要子画面编码（基于在马赛克或子画面之上分层对象的编码）的视频编码器。该项目（i）为更大范围的视频开发马赛克生成解决方案;（ii）通过对视频镜头进行分类，为包含许多镜头的视频生成马赛克;（iii）通过产生地面实况，为马赛克生成开发客观评估方法。Sprite融合方法融合了使用对齐的帧差生成的断言和保守的Sprite，而没有对象分割，从而消除了对象遮挡问题，特别是对于摄像机跟踪对象的视频。由于子画面融合方法只对一对帧计算一次全局运动，不需要对象分割，因此非常适合视频编码器的实时马赛克生成或子画面编码。当为序列构建马赛克时，存储全局运动矢量以能够相对于另一帧生成（或扭曲）任何帧，从而允许为空间交互提供多个自由度，包括上下、左右、前后移动和旋转。具有当前（或活动）场景的扭曲的多个自由度通过从马赛克中重新生成任何帧并且然后将活动（当前场景）覆盖在再现帧的顶部上来形成交互式视频再现的基础。 该项目的教育部分包括开发一个新的多媒体课程，吸引任何新生，以提高对计算机科学的兴趣。本课程涵盖计算机科学的基本概念，包括媒体类型，颜色模型，存储设备，多媒体创作和互联网。计划为K-12学生举办一个多媒体讲习班。 这一研究项目的结果将通过因特网（http：//www.cs.uah.edu/martraygun/projects/mosaics.htm）传播，包括一个录像数据库系统（录像集和任何真相注释），以便与其他研究人员分享试验数据，并鼓励各研究机构进行公开的基于计量的评价。该项目将促进开发具有广泛应用的使用交互式视频再现的照片级真实感可视化系统。