RI: Medium: Integrating Humans and Computers for Image and Video Understanding

在这个项目中，研究团队探索了几个研究挑战，以利用图像、视频和观看这些视觉图像的人之间的关系。探索领域包括：1)行为实验，以更好地理解人类观众和图像之间的关系；2)开发用于图像和视频理解的人机协作系统，该系统利用自动计算机视觉算法结合人类观众的主动和被动线索；3)使用我们的协作模型实现检索和收集组织应用程序。从b谷歌到微软再到Facebook，通过这些非常成功的公司的基础设施，数以十亿计的图片和数以百万计的视频现在可以在网上获得。这种丰富的视觉数据为交流和社区创造了大量的机会，并使我们世界的社会结构更加紧密。在网络图像爆炸式增长的同时，监视用户的摄像头也在不断增加，从笔记本电脑上随处可见的网络摄像头，到我们随身携带的手机摄像头。这种用户观看行为的记录，特别是他们的眼睛、身体运动或描述，可以为人们如何与图像或视频交互提供巨大的见解，并可以为构建更有效的视觉应用程序（如图像或视频检索）提供信息。此外，了解人们对图像或视频的识别、关注或描述是实现以人为中心的图像理解的高水平目标的必要步骤，这将对许多不同领域产生研究益处，包括计算机视觉和行为科学。

In this project, the research team explores several research challenges to exploit the relationship between images, video, and the people viewing this visual imagery. Areas of exploration include: 1) behavioral experiments to better understand the relationship between human viewers and imagery, 2) development of human-computer collaborative systems for image and video understanding that utilize automatic computer vision algorithms in conjunction with active and passive cues from human viewers, and 3) implementing retrieval and collection organization applications using our collaborative models.Billions of images and millions of videos are now available online via the infrastructure of amazingly successful companies from Google to Microsoft to Facebook. This wealth of visual data is creating considerable opportunities for communication and community, and tightening the social fabric of our world. In parallel to this explosion in online imagery, there is also an increasing proliferation of cameras viewing the user, from the ever present webcams peering out at us from our laptops, to cell phone cameras carried in our pockets wherever we go. This record of a user's viewing behavior, particularly of their eye, body movements, or descriptions, can provide enormous insight into how people interact with images or video, and can inform construction of more effective visual applications such as image or video retrieval. In addition, understanding what people recognize, attend to, or describe about an image or video is a necessary step toward high level goals of human centric image understanding that will have research benefits to many diverse fields, including computer vision and behavioral science.