Collaborative Research: Simultaneous Contour Grouping and Medial Axis Estimation

随着图像和视频数量的快速增长，图像和视频信息提取的主要瓶颈是图像和视频的分析（索引）和检索。目前的图像和视频搜索引擎是基于文本描述的，因为在处理大量的图像和视频时，视觉线索的水平太低，无法提供有用的检索结果。例如，如果一个人提交了一个查询图像，并请求查找类似的图像，那么她将关注查询图像中的某个对象或一组对象。因此，相似性的含义是由包含相似对象的图像给出的。因此，图像（和视频）中对象的提取是基于内容的图像/视频检索（CBIR）真正取得进展的关键因素。然而，目标提取是计算机视觉中尚未解决的问题。这一事实导致了大量方法的发展，这些方法试图在没有对象提取的情况下进行CBIR。然而，尽管这些方法在一些受限的应用领域可能是成功的，在这种情况下，低级特征可能足以取代对象提取，但它们在通用的CBIR中还没有成功。ppi认为，解决目标提取问题将导致CBIR的突破。因此，pi建议从图像中提取目标。针对CV中的目标提取问题，已经进行了大量的尝试，但都没有提供满意的解决方案。为什么我们的方法会提供一个好的解决方案？pi提出的一种新的方法和计算框架为目标提取的突破提供了坚实的证据。在认知和几何建模方面，pi建议使用更高层次的形状相似性知识和中级层次的局部和全局对称知识作为对象提取的认知动机约束。约束是必要的，因为目标提取是一个已知的不适定逆问题。人类的视觉系统很好地解决了这个问题，我们正在接近于完全理解这是如何完成的。在计算方面，pi提出了一个同时估计中轴和轮廓的新框架。该方法的灵感来自于机器人地图领域的SLAM （Simultaneous Localization and Mapping）方法。基于粒子滤波的SLAM计算是近年来机器人测绘领域的突破性解决方案。SLAM计算迭代机器人在现有部分地图（轨迹估计）中的定位过程，然后根据新的观测结果和估计的轨迹更新地图。pi将中间轴视为虚拟机器人的轨迹，将部分边界视为与中间轴相关的边缘段组成的地图。初步结果中的pi证明了该框架的首次成功应用。项目网址：http://knight.cis.temple.edu/~shape/

With the ever faster growing number of images and videos, the main bottleneck in extracting the information contained in them is their analysis (indexing) and retrieval. Nowadays image and video search engines are based on textual descriptions, since visual cues are at too low level to provide useful retrieval results when dealing with a large variety of images and videos. For example, if a human submits a query image with the request to find similar images, she focuses on a certain object or a group of objects in the query image. Thus, the meaning of similarity is given by the images that contain similar objects. Therefore, extraction of objects in images (and videos) is a key factor for true progress in content based image/video retrieval (CBIR). However, object extraction belongs to unsolved problems in Computer Vision (CV). This fact led to the development of a huge number of approaches that try to do CBIR without object extraction. However, although such approaches may be successful in some restricted application domains, in which case low level features may be sufficient to replace object extraction, they have not been successful in general purpose CBIR. The PIs believe solving the object extraction problem will lead to a breakthrough in CBIR. Therefore, the PIs propose to work on object extraction in images. There have been a large number of attempts to solve the object extraction problem in CV, and none provided a satisfactory solution. Why will our approach provide a good solution? A new methodology and a computation framework proposed by the PIs provide solid evidence that the breakthrough in object extraction is possible. On the cognitive and geometric modeling side, the PIs propose to use a higher level knowledge of shape similarity and a mid level knowledge of local and global symmetry as cognitively motivated constraints for object extraction. Constraints are essential because object extraction is known to be an ill-posed inverse problem. The human visual system solves this problem very well and we are getting close to a full understanding of how this is done. On the computational side, the PIs propose a new framework for a simultaneous estimation of medial axes and the contours. The proposed approach is inspired by the SLAM (Simultaneous Localization and Mapping) approaches in the field of robot mapping. Recent breakthrough solutions in robot mapping are based on the SLAM computation with particle filters. SLAM computation iterates over the processes of localization of the robot in the existing partial map (trajectory estimation), followed by a map update based on new observations and the estimated trajectory. The PIs treat the medial axis as trajectory of a virtual robot and the partial boundary as the map that is composed of edge segments associated with the medial axis. A first successful application of this framework is demonstrated by the PIs in the preliminary results.Project URL: http://knight.cis.temple.edu/~shape/