RI: Medium: Integrating Humans and Computers for Image and Video Understanding

在这个项目中，研究小组探讨了几个研究挑战，以利用图像，视频和观看这种视觉图像的人之间的关系。 探索的领域包括：1）行为实验，以更好地理解人类观众和图像之间的关系，2）开发用于图像和视频理解的人机协作系统，其利用自动计算机视觉算法结合来自人类观众的主动和被动线索，和3）使用我们的协作模型实现检索和收集组织应用程序。现在可以使用数十亿张图像和数百万个视频通过从谷歌到微软再到Facebook等非常成功的公司的基础设施在线。这些丰富的视觉数据为沟通和社区创造了相当多的机会，并收紧了我们世界的社会结构。 与在线图像的爆炸式增长相平行的是，观察用户的摄像头也越来越多，从笔记本电脑上一直存在的网络摄像头到我们随身携带的手机摄像头。用户的观看行为的这种记录，特别是他们的眼睛、身体运动或描述，可以提供人们如何与图像或视频交互的巨大洞察力，并且可以为更有效的视觉应用（例如图像或视频检索）的构建提供信息。 此外，了解人们对图像或视频的认知、关注或描述是实现以人为中心的图像理解的高层次目标的必要步骤，这将对许多不同的领域产生研究效益，包括计算机视觉和行为科学。

In this project, the research team explores several research challenges to exploit the relationship between images, video, and the people viewing this visual imagery. Areas of exploration include: 1) behavioral experiments to better understand the relationship between human viewers and imagery, 2) development of human-computer collaborative systems for image and video understanding that utilize automatic computer vision algorithms in conjunction with active and passive cues from human viewers, and 3) implementing retrieval and collection organization applications using our collaborative models.Billions of images and millions of videos are now available online via the infrastructure of amazingly successful companies from Google to Microsoft to Facebook. This wealth of visual data is creating considerable opportunities for communication and community, and tightening the social fabric of our world. In parallel to this explosion in online imagery, there is also an increasing proliferation of cameras viewing the user, from the ever present webcams peering out at us from our laptops, to cell phone cameras carried in our pockets wherever we go. This record of a user's viewing behavior, particularly of their eye, body movements, or descriptions, can provide enormous insight into how people interact with images or video, and can inform construction of more effective visual applications such as image or video retrieval. In addition, understanding what people recognize, attend to, or describe about an image or video is a necessary step toward high level goals of human centric image understanding that will have research benefits to many diverse fields, including computer vision and behavioral science.