Similarity Based Retrieval From Video and Pictorial Databases

本研究的目的是开发一个基于相似度的视频和图片数据库检索系统。该系统基于一种称为层次时态逻辑（html）的表达性查询语言，该语言表示视频数据的时态和图形内容。然后使用多维相似性度量将html谓词与数据库进行匹配。该方法包括对通用对象和通用运动的图像特征提取的表示和预处理方法的开发。提取的特征包括形状和运动的光谱特征、局部和全局颜色直方图以及纹理。利用运动谱特征（MSS）将视频中的关节运动分解为基本分量。利用仿射不变谱特征（AISS）和基于扩展匹配（EXM）的频域索引来识别一般目标。然后利用运动和对象表示构造具有多维超区域表示的html原子谓词字典。此外，还计划开发在线学习功能，以扩充词典。改进的视频拍摄和场景分割方法也有待开发。研究结果将为视频和图片数据库的查询检索提供一种有效的方法。该系统将在数字图书馆和电视电影行业中得到广泛应用。

The objective of this research is to develop a system for similarity based retrieval from video and pictorial databases. The system is based on an expressive query language called Hierarchical Temporal Logic (HTL) that represents the temporal and pictorial content of video data. HTL predicates are then matched with the database using multidimensional similarity measures. The approach includes development of representation and preprocessing methods for pictorial feature extraction both for generic objects and generic motion. The features extracted include spectral signatures of shape and motion, local and global color histograms and texture. Articulated motion in video is decomposed into basic components using Motion Spectral Signatures (MSS). Generic objects are recognized using Affine Invariant Spectral Signatures (AISS) and Expansion Matching (EXM) based indexing in the frequency domain. Motion and object representations are then utilized to construct a dictionary of HTL atomic predicates with multidimensional hyper-regional representation. An on-line learning capability for expanding the dictionary is also planned. Improved methods for video shot and scene segmentation are to be developed as well. The results of this research will provide an efficient means for query retrieval from video and pictorial databases. Such a system will have many applications in digital libraries and in TV movie industries.