CAREER: Telling the Story of a Visual World: Event Classification and Integrated Image Understanding

从视觉世界中获得意义的能力，如识别对象、场景以及在语义上有意义的活动和事件，是人工智能的基石。在计算机视觉中，最近在物体和场景级别的识别方面取得了非常重要的进展。但这样的任务往往是在没有对现场进行完整和连贯的描述的情况下执行的。此外，目前很少有算法能够进一步解释图像的高级语义，如事件或活动。该项目的目标是通过对单个未知图像的综合图像理解来实现事件分类。该项目旨在通过使用大量真实世界的数据(如来自互联网的数据)开发适合于训练算法的复杂学习框架，推动图像综合和描述性理解的前沿。高精度的性能、最少的人工监督、灵活性和可扩展的学习将是这一努力的重点。该项目？S理论框架将计算机视觉的多个领域联系在一起，为机器学习领域提供了有趣的模型表示，并将更多语义驱动的视觉识别问题与自然语言处理领域联系起来。研究结果对视障人士的图像理解技术、大型数字图书馆的图像自动标注以及下一代图像检索引擎，以及语言学生和医学患者(如失语症、中风等)的翻译、教育和康复技术都具有重要意义。S长期教育计划的重点是将最新的视觉计算和认知研究直接带入课堂和整个社区，重点是接触到代表性不足的学生群体。

The ability to make meaning out of a visual world, such as recognizing objects, scenes and semantically meaningful activities and events, is a cornerstone of artificial intelligence. In computer vision, very important progress has been made recently in object and scene level recognition. But such tasks are often performed without an integrated and coherent description of the scene. Moreover, very few current algorithms are capable of further interpreting higher level semantic meanings of an image such as an event or activity. The goal of this project is to achieve event classification via an integrated image understanding given a single unknown image.This project aims to push the frontier of integrated and descriptive understanding of images through the development of sophisticated learning frameworks suitable for training algorithms by using a large amount of real-world data such as the ones from the Internet. High accuracy performance, minimal human supervision, flexibility and scalable learning will be the focus of this endeavor. This project?s theoretical framework ties together several areas of computer vision, offers interesting model representations for the machine learning field, and connects more semantically driven visual recognition problem with the natural language processing field. The results are vital for image understanding technology for the visually- impaired; automatic annotation of images for large digital library as well as the next generation of image retrieval engines; and translation, education and rehabilitation technology for language students and medical patients (such as aphasia, stroke, etc.). The project?s long-term educational plan focuses on bringing the latest visual computation and cognition research directly into the classroom and the community at large, with an emphasis on reaching the underrepresented groups of students.