RII Track-2 FEC: The Visual Experience Database: A Large-Scale Point-of-View Video Database for Vision Research

目前识别视觉内容的人工智能(AI)系统需要数百万个训练样本才能达到良好的性能。然而，用于训练这类系统的数据库通常从互联网上拍摄照片和视频，因此并不代表人类每天看到的内容。这给人工智能系统带来了严重的偏见，可能会对自动驾驶汽车等基于人工智能的应用产生严重影响。该项目由贝茨学院、内华达大学雷诺分校和北达科他州立大学法戈分校合作，将创建视觉体验数据库(VED)，这是一个从从事共同日常活动(如购物、饮食或散步)的不同观察者的角度拍摄的240多小时视频的数据库。除了这些视频，我们还将跟踪每个观察者的头部和眼睛位置，以了解人们如何看待世界，以及这种看法如何随着环境、年龄和任务的变化而变化。我们的目标是使这个数据库对所有人开放和访问。拥有使用数据库的计算机技能是无障碍访问的关键，因此我们将发布一套使用数据库的软件工具，并实施基本计算机编程技能夏季研讨会，以培养为各种科学职业做好准备的劳动力。通过将数据库向公众开放，我们将使科学家、历史学家甚至艺术家能够从这一丰富的资源中受益。人类视觉神经科学和计算机视觉的进步都受到具有代表性的视觉数据可用性的限制。然而，目前可用的图像和电影数据库并不能代表典型的第一人称视觉体验。这个项目是由贝茨学院、内华达大学雷诺分校和北达科他州立大学法戈分校合作完成的，将创建视觉体验数据库(VED)，该数据库包含超过240小时的第一人称视频，包括眼睛和头部跟踪。我们将记录来自三个不同地理位置的不同年龄(5-70岁)的人，他们参与购物、饮食或散步等共同的日常活动。有了这些数据，我们将能够评估观察者如何采样他们的视觉环境，以及凝视模式如何随着环境、年龄和任务的变化而变化。此外，这些数据可以用作下一代计算机视觉系统的训练数据。为了培养一支拥有大数据工作所需技能的劳动力队伍，我们将开设大数据技能暑期研讨会，为本科生和研究生提供计算机编程和计算能力的基本技能，以便为该项目做出贡献，并为各种STEM职业做好准备。VED将在几个学术社区(认知科学、神经科学、计算机视觉，可能还有数字人文和艺术)中广泛使用。通过创建一个代表常见人类经验的数据库，我们绕过了现有数据集的许多偏见，这将提高计算机视觉算法的效率。通过将这些数据完全公开，我们将使所有人都能够获得这些领域的进展。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Current artificial intelligence (AI) systems that recognize visual content require millions of training examples to achieve good performance. However, the databases used to train such systems often take photos and videos from the internet, and thus do not represent the content that humans see on a daily basis. This introduces substantial biases into the AI systems that can have serious implications for AI-based applications such as self-driving cars. This project, a collaboration between Bates College, the University of Nevada, Reno, and North Dakota State University, Fargo will create the Visual Experience Database (VED), a database of over 240 hours of video shot from the perspective of a diverse set of observers engaged in common, everyday activities such as shopping, eating, or walking. Along with these videos, we will track each observer's head and eye position in order to understand how people look at the world, and how this changes with environment, age, and task. Our goal is to make this database open and accessible to all. Having the computer skills to use the database is key to accessibility, so we will be releasing a suite of software tools for using the database, as well as implement a summer workshop in basic computer programming skills to grow a workforce that is prepared for a variety of scientific occupations. By making the database open to the public, we will enable scientists, historians, and even artists to benefit from this rich resource. Progress in both human visual neuroscience and computer vision are limited by the availability of representative visual data. However, currently available image and movie databases are not representative of typical first-person visual experience. This project, a collaboration between Bates College, the University of Nevada, Reno, and North Dakota State University, Fargo, will create the Visual Experience Database (VED), a database of over 240 hours of first-person video complete with eye- and head-tracking. We will record from people of diverse ages (5-70 years) across three geographically distinct sites as they engage in common, everyday activities such as shopping, eating, or walking. With these data, we will be able to assess how observers sample their visual environments, and how gaze patterns change with environment, age, and task. Further, these data can be used as training data for next-generation computer vision systems. In order to develop a workforce with the skills necessary to work with big data, we will teach a Big Data Skills Summer Workshop to provide undergraduate and graduate students with the basic skills of computer programming and computational literacy to make contributions to this project and to prepare them for a variety of STEM occupations. The VED will be of broad use across several academic communities (cognitive science, neuroscience, computer vision, and possibly digital humanities and art). By creating a database that represents common, human experiences, we bypass the many biases of extant datasets, which will increase the efficacy of computer vision algorithms. By making these data fully open, we will enable advances in these fields to be accessible to all.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.