EAGER: Social Networks Based Concept Learning in Images

在概念层面上对可视化数据进行简单、快速和直观的搜索的需求是普遍的。该项目将探索一种基于网络分析的新方法来搜索视觉数据，最终导致视觉搜索引擎，使搜索图像像今天的关键字搜索一样容易。实现这一目标需要新的视觉概念机器学习方法。这个项目提出了一个正式的框架，统一了社交网络和语义概念学习的想法，使多个语义概念可以学习高信心。具体来说，该方法利用概念之间的层次共现相关性作为线索，以帮助检测单个视觉概念。鲁棒性的来源是学习同现模式，类似于社交网络中的社区结构，以及它们随时间的改进。概念共现检测的成功将简化自动标注的个人图像数据的管理。利用语义上组织的个人内容，可以学习用户的偏好以提供他/她在线消费的各种内容的个性化。这将对各种应用产生广泛影响，从信息技术到物理、生命和社会科学，再到情报机构和新闻机构。数字数据的取证分析将大大加快，因为人类不必筛选大量数据。将这些技术扩展到视频、音乐和多模态数据也将为内容消费提供类似的便利。研究团队提供了一个环境，将教育和劳动力发展与研究相结合，并招募和保留了一批多样化的学生。除了研究活动之外，还将在教育和公共宣传方面采取新的举措。该项目开发了一种变革性的方法来系统地探索语义视觉概念的获取和细化。首先，它发现的层次共现模式的概念作为潜在的社区结构的共现网络。同现模式在更高层次的语义上扮演着类似于底层场景概念的角色。其次，提出了一种选择视觉上一致的语义概念的方法。由于概念的视觉复杂性各不相同，每个概念的视觉语义相关性进行了研究，通过定量测量的概念内的视觉变化和视觉距离的其他概念，使它们可以更可靠地建模，更容易检测。第三，该项目引入了一种新的图像内容描述符称为概念签名，可以记录语义概念和相应的置信度值推断从低级别的功能。最后，该项目提出了可扩展性的技术，通过开发开源技术和软件工具来处理和评估大型数据库的性能。将通过项目网站（http：//vislab.ucr.edu/RESEARCH/VisualSemanticConcepts/VSC.php）、定期发布软件工具和在IEEE/ACM主要会议上提供教程/讲习班，广泛传播成果。

The need for easy, quick, and intuitive search of visual data at the conceptual level is universal. This project will explore a novel network analysis-based approach to searching visual data, ultimately leading to visual search engines that would make searching images as easy as searching by keywords is today. Accomplishing this requires new approaches to machine learning of visual concepts. This project proposes a formal framework that unifies the ideas from social networks and semantic concept learning so multiple semantic concepts can be learned with high confidence. Specifically, the approach utilizes hierarchical co-occurrence correlation among concepts as cues to help the detection of individual visual concepts. The sources for robustness are the learning of co-occurrence patterns, similar to community structures in a social network, and their refinement over time. The success of concept co-occurrence detection will simplify management of personal image data with automatic tagging. With semantically organized personal content, the preferences of a user can be learned to provide personalization of various contents that he/she consumes online. This will have a broad impact for diverse applications ranging from information technology to physical, life and social sciences to intelligence organizations to news bureaus. Forensics analysis of digital data would be greatly speeded up as humans do not have to sift through large amount of data. Extension of the techniques to video, music, and multi-modal data would also provide similar ease for content consumption. The research team provides an environment integrating education and workforce development with research, and with recruiting and retaining a diverse group of students. Complementing the research activities will be new initiatives in education and public outreach. The project develops a transformative approach to explore the acquisition and refinement of semantic visual concepts systematically. First, it discovers the hierarchical co-occurrence patterns of concepts as underlying community structures in the co-occurrence network. The co-occurrence patterns play roles similar to underlying scene concepts at a higher level of semantics. Second, it proposes an approach for selecting visually-consistent-semantic concepts. Since concepts vary in their visual complexity, visual-semantic relatedness of each concept is investigated by quantitatively measuring the within-concept visual variability and the visual distances to the other concepts such that they can be modeled more reliably and detected more easily. Third, the project introduces a novel image content descriptor called concept signature that can record both the semantic concept and the corresponding confidence value inferred from low-level features. Finally, the project proposes techniques for scalability to handle and evaluate the performance on large databases by developing open source techniques and software tools. The results will be broadly disseminated through the project website (http://vislab.ucr.edu/RESEARCH/VisualSemanticConcepts/VSC.php), via regular releases of software tools and offering tutorials/workshops at major IEEE/ACM conferences.