Perceptual content description language and interface for supporting visual semantic computing

视觉数据的快速增长已经吸引了认知科学、计算机视觉、信息检索、数据库和机器学习的显著研究兴趣，以开发用于处理基于其语义内容的大型图像和视频数据库以及视觉数据流的新方法。视觉语义计算（VSC）领域汇集了那些关注将人类意图（通常是精心制定的）与图像和视频联系起来的学科。各种视觉信息系统的用户可能以不同的方式感知数据，但共同的需求是基于他们自己对数据的感知来处理和查询数据的能力。 设计能够充分理解图像以实现有效操作的系统，例如搜索大型数据库，仍然是一个非常具有挑战性的问题。例如，在一个示例中，图像检索系统不仅仅是图像相似性引擎。除了图像匹配之外，它还应该解决索引问题，以实现快速搜索;考虑先验信息，可以用来对某些图像进行比其他图像更强的加权;以及探索用户在检索循环中的存在。尽管近年来不断努力，最重要的挑战仍然是弥合人类对视觉数据的感知与计算机所看到的特征之间的语义鸿沟。本研究的目标是以对称的方式解决这一挑战，强调基于感知的内容表示和操作。1)探讨人类视觉和认知研究中视觉注意、模块化、感知和记忆的关键理论和假设。2)以连贯和协作的方式复制一些关键功能，以提取感知特征，特别是感知分区和基于感知的特征。3)基于1）和2），开发了感知内容描述语言（PCDL）的框架。4)发展一种系统的方法来构建视觉语义分类（VST）和VST和PCDL之间的映射，其中基于感知像性的描述与先前的语义词相关联。 这种新的方法将促进基于内容的图像/视频检索，索引，注释，操作和识别的VSC应用程序开发。

The rapid growth of visual data has attracted a significant research interest in cognition science, computer vision, information retrieval, databases and machine learning to develop new methods for handling large image and video databases and visual data streams based on its semantic content. The field of Visual Semantic Computing (VSC) brings together those disciplines concerned with connecting human intention (often vaguely-formulated) with images and video. The users of various visual information systems may perceive the data in different ways, but a common demand is the ability to process and query the data based on their own perception of the data. Designing systems that understand images well enough to enable effective operations, such as search of large databases remains a very challenging problem. E.g., an image retrieval system is more than an image similarity engine. In addition to image matching, it should address the problems of indexing to enable fast searches; accounting for prior information, which can be used to weigh some images more strongly than others; and exploring the user's presence in the retrieval loop. Despite the sustained efforts in recent years, the paramount challenge still remains on bridging the semantic gap between human's perception on visual data and the features seen by the computer. The goal of this research is to tackle this challenge in a symmetrical way emphasizing perception based content representation and manipulation. 1) Investigate the key theories and hypotheses from human vision and cognition research in visual attention, modularity, perception, and memory. 2) Duplicate some of the key functionalities in coherent and collaborative way for extracting perceptual features, in particular Perceptual Partitioning and Grouping based. 3) Develop a frame work of Perceptual Content Description Language (PCDL) based on 1) and 2). 4) Develop a systematic approach for constructing Visual Semantic Taxonomy (VST) and mapping between VST and PCDL in that the perceptual iconic based descriptions are linked to semantic words in prior. This new approach will facilitate VSC application development for content based image/video retrieval, indexing, annotation, manipulation and recognition.