Similarity Based Retrieval From Video and Pictorial Databases

本研究的目的是开发一个系统，用于从视频和图片数据库中进行基于相似性的检索。该系统是基于一种表达查询语言，称为层次时态逻辑（HTL），表示时间和图像内容的视频数据。然后，使用多维相似性度量将HTL谓词与数据库进行匹配。该方法包括发展的代表性和预处理方法的图形特征提取通用对象和通用运动。提取的特征包括形状和运动的光谱特征、局部和全局颜色直方图以及纹理。视频中的关节运动被分解成基本成分使用运动谱特征（MSS）。在频域中使用基于仿射不变谱特征（AISS）和扩展匹配（EXM）的索引来识别通用对象。运动和对象表示，然后利用多维超区域表示来构建HTL原子谓词的字典。还计划建立扩充词典的在线学习能力。视频镜头和场景分割的改进方法也有待开发。本研究的结果将提供一个有效的手段，从视频和图片数据库的查询检索。该系统在数字图书馆和电视电影行业有着广泛的应用前景。

The objective of this research is to develop a system for similarity based retrieval from video and pictorial databases. The system is based on an expressive query language called Hierarchical Temporal Logic (HTL) that represents the temporal and pictorial content of video data. HTL predicates are then matched with the database using multidimensional similarity measures. The approach includes development of representation and preprocessing methods for pictorial feature extraction both for generic objects and generic motion. The features extracted include spectral signatures of shape and motion, local and global color histograms and texture. Articulated motion in video is decomposed into basic components using Motion Spectral Signatures (MSS). Generic objects are recognized using Affine Invariant Spectral Signatures (AISS) and Expansion Matching (EXM) based indexing in the frequency domain. Motion and object representations are then utilized to construct a dictionary of HTL atomic predicates with multidimensional hyper-regional representation. An on-line learning capability for expanding the dictionary is also planned. Improved methods for video shot and scene segmentation are to be developed as well. The results of this research will provide an efficient means for query retrieval from video and pictorial databases. Such a system will have many applications in digital libraries and in TV movie industries.