CAREER: Learning Models for Scalable Content-Based Image Retrieval

该项目致力于设计机器学习算法，使基于内容的图像检索在网络规模的照片集合。本研究将影像撷取视为一个二元分类问题：决定哪些资料库影像与使用者提供的照片相同。通过将分类器约束为传统文本搜索引擎支持的模型，可以实现大集合的效率和可扩展性，这些文本搜索引擎在数十亿文档的数据库中执行实时搜索。为了实现基于高层次相似性概念的搜索，研究团队开发了自动定位输入照片中最相关的内容区域的方法，并从中提取出语义强大的分类器，将外观线索与强大的几何约束相结合。这些算法从用户提供的标签中学习，这些标签指示类似视觉内容的存在，但不指示其位置，因此需要最少量的人类监督。本研究还探讨了如何使用这种高级形式的相似图像搜索来组织个人照片，提供语义注释，并支持基于内容的图片聚类。 此外，这项工作提供了广泛的计算机视觉问题，包括对象检测，视觉显着性和基于内容的照片聚类的技术进步。此外，研究小组正在收集一个前所未有的大图像数据集，以评估开发的图像检索系统，并提供给社区。通过相关课程和课外活动，研究自然与教育和外联相结合，旨在吸引学生进入这一领域，并鼓励跨学科合作。

This project addresses the design of machine learning algorithms enabling content-based image retrieval in Web-scale collections of photos. This research formulates image retrieval as a binary classification problem: decide which database images are the "same" as the user-provided photo. Efficiency and scalability to large collections are achieved by constraining the classifiers to be models supported by traditional text-search engines, which perform real-time search in databases of several billion documents. In order to implement search based on high-level notions of similarity, the research team develops methods to automatically localize the most content-relevant regions in the input photo and to extract from them semantically powerful classifiers combining appearance cues with robust geometric constraints. The algorithms learn from user-provided labels indicating the presence but not the location of similar visual content, thus requiring a minimal amount of human supervision. This research investigates also how this advanced form of similar-image search can be used to organize personal photos, provide semantic annotations, and support content-based clustering of pictures. Furthermore, this work provides technical advances in a wide range of computer vision problems including object detection, visual saliency, and content-based clustering of photos. Moreover, the research team is collecting an unprecedentedly large image data set to evaluate the developed image retrieval system and to be available to the community. Research is naturally integrated with education and outreach by means of related courses and out-of-classroom activities aimed at attracting students to this field and at encouraging interdisciplinary collaborations.