RII Track-2 FEC: The Visual Experience Database: A Large-Scale Point-of-View Video Database for Vision Research

当前识别视觉内容的人工智能（AI）系统需要数百万个训练示例才能达到良好的性能。然而，用于训练此类系统的数据库通常是从互联网上获取照片和视频，因此不能代表人类日常看到的内容。这给人工智能系统带来了实质性的偏差，可能对自动驾驶汽车等基于人工智能的应用产生严重影响。该项目由贝茨学院、内华达大学里诺分校和北达科他州立大学合作，将创建视觉体验数据库（VED），该数据库包含240多个小时的视频，这些视频来自不同的观察者，他们从事日常活动，如购物、吃饭或走路。随着这些视频，我们将跟踪每个观察者的头部和眼睛的位置，以了解人们如何看待世界，以及如何随着环境，年龄和任务的变化。我们的目标是使这个数据库对所有人开放和访问。拥有使用数据库的计算机技能是可访问性的关键，因此我们将发布一套使用数据库的软件工具，以及实施基本计算机编程技能的夏季讲习班，以培养为各种科学职业做好准备的劳动力。通过向公众开放数据库，我们将使科学家、历史学家、甚至艺术家都能从这个丰富的资源中受益。人类视觉神经科学和计算机视觉的进展都受到代表性视觉数据可用性的限制。然而，目前可用的图像和电影数据库并不能代表典型的第一人称视觉体验。这个项目由贝茨学院、内华达大学里诺分校和北达科他州立大学法戈分校合作，将创建视觉体验数据库（VED），这是一个超过240小时的第一人称视频数据库，包括眼睛和头部跟踪。我们将记录不同年龄（5-70岁）的人在三个地理位置不同的地点从事共同的日常活动，如购物、吃饭或散步。有了这些数据，我们将能够评估观察者如何采样他们的视觉环境，以及凝视模式如何随着环境、年龄和任务而变化。此外，这些数据可以用作下一代计算机视觉系统的训练数据。为了培养具有处理大数据所需技能的劳动力，我们将开设一个大数据技能暑期讲习班，为本科生和研究生提供计算机编程和计算素养的基本技能，为这个项目做出贡献，并为他们从事各种STEM职业做好准备。VED将广泛应用于多个学术团体（认知科学、神经科学、计算机视觉，可能还有数字人文和艺术）。通过创建一个代表人类共同经验的数据库，我们绕过了现有数据集的许多偏见，这将提高计算机视觉算法的效率。通过使这些数据完全开放，我们将使所有人都能获得这些领域的进展。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Current artificial intelligence (AI) systems that recognize visual content require millions of training examples to achieve good performance. However, the databases used to train such systems often take photos and videos from the internet, and thus do not represent the content that humans see on a daily basis. This introduces substantial biases into the AI systems that can have serious implications for AI-based applications such as self-driving cars. This project, a collaboration between Bates College, the University of Nevada, Reno, and North Dakota State University, Fargo will create the Visual Experience Database (VED), a database of over 240 hours of video shot from the perspective of a diverse set of observers engaged in common, everyday activities such as shopping, eating, or walking. Along with these videos, we will track each observer's head and eye position in order to understand how people look at the world, and how this changes with environment, age, and task. Our goal is to make this database open and accessible to all. Having the computer skills to use the database is key to accessibility, so we will be releasing a suite of software tools for using the database, as well as implement a summer workshop in basic computer programming skills to grow a workforce that is prepared for a variety of scientific occupations. By making the database open to the public, we will enable scientists, historians, and even artists to benefit from this rich resource. Progress in both human visual neuroscience and computer vision are limited by the availability of representative visual data. However, currently available image and movie databases are not representative of typical first-person visual experience. This project, a collaboration between Bates College, the University of Nevada, Reno, and North Dakota State University, Fargo, will create the Visual Experience Database (VED), a database of over 240 hours of first-person video complete with eye- and head-tracking. We will record from people of diverse ages (5-70 years) across three geographically distinct sites as they engage in common, everyday activities such as shopping, eating, or walking. With these data, we will be able to assess how observers sample their visual environments, and how gaze patterns change with environment, age, and task. Further, these data can be used as training data for next-generation computer vision systems. In order to develop a workforce with the skills necessary to work with big data, we will teach a Big Data Skills Summer Workshop to provide undergraduate and graduate students with the basic skills of computer programming and computational literacy to make contributions to this project and to prepare them for a variety of STEM occupations. The VED will be of broad use across several academic communities (cognitive science, neuroscience, computer vision, and possibly digital humanities and art). By creating a database that represents common, human experiences, we bypass the many biases of extant datasets, which will increase the efficacy of computer vision algorithms. By making these data fully open, we will enable advances in these fields to be accessible to all.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.