CAREER: Bootstrapping Recognition from Little Data in New Domains

该奖项的部分资金来自《2021年美国救援计划法案》(公法117-2)。在当前的计算机视觉技术下，人们可以很容易地开发出能够识别数千个物体的软件，无论是猫、车还是鸟。然而，这样的系统必须在数百万张图像的大数据集上进行训练，这些图像已经被人类注释员煞费苦心地标记。对于许多应用程序领域，很难创建如此大的带标签的数据集，例如显微镜，在这些领域中，注释员将需要经过高度培训。收集大型数据集也可能与隐私问题相冲突，可能需要昂贵的管理来消除种族、性别或其他类型的偏见。最后，收集大型数据集的高昂成本使许多人无法使用计算机视觉技术。这个项目开发了识别系统的技术，可以成功地识别困难的视觉概念，而不需要任何大型数据集。能够处理有限训练数据的识别系统将解锁许多下游应用，特别是在专门领域，并将使所有人都能获得计算机视觉技术的进步。该项目团队将使所有人都可以使用这些识别系统，为来自代表性不足社区的高中生组织一个研讨会，并开发一种新的计算机视觉课程，专注于计算机视觉技术的广泛应用。为了从少量数据(少至1-2个标记图像和约1000个未标记图像)构建识别系统，该项目探索了两种受人类视觉启发的策略。首先，与目前针对每个问题领域单独训练的识别系统不同，人类学习在其丰富的先前视觉经验的背景下执行新的视觉任务(如分析显微镜图像)。这个项目类似地设计了视觉学习者，通过利用过去跨多个领域的视觉任务的记忆，从有限的数据中学习新的识别任务。第二，不同于目前只能从有标签的图像中学习的系统，人类通过丰富的互动和与专家教师的来回交流来学习。在这一观察的启发下，该项目建立了可以通过向专家提出详细问题来学习的系统。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This award is funded in part under the American Rescue Plan Act of 2021 (Public Law 117-2).With current computer vision technology, one can easily build software that can recognize thousands of objects, be they cats, cars or birds. However, such systems must be trained on large datasets of millions of images which have been painstakingly labeled by human annotators. Such large, labeled datasets are difficult to create for many application domains, such as microscopy, where annotators will need to be highly trained. Collecting large datasets may also run afoul of privacy concerns, and may need expensive curation to remove racial, gender or other kinds of bias. Finally, the steep cost of collecting large datasets makes computer vision technology inaccessible to many. This project develops technologies for recognition systems that can successfully identify difficult visual concepts without needing any large datasets. Recognition systems that can work with limited training data will unlock many downstream applications, especially in specialized domains, and will make advances in computer vision technology accessible to all. The project team will make these recognition systems broadly accessible to all, organize a workshop for high school students from underrepresented communities, and develop a new computer vision curriculum that focuses on broad applications of computer vision technology.To build recognition systems from little data (as few as 1-2 labeled images and around 1000 unlabeled images), this project explores two strategies inspired from human vision. First, unlike current recognition systems that are trained in isolation for each problem domain, humans learn to perform new visual tasks (such as analyzing microscopy images) in the context of their vast prior visual experience. This project similarly designs visual learners that learn new recognition tasks from limited data by leveraging a memory of past visual tasks across multiple domains. Second, unlike current systems that learn only from labeled images, humans learn through rich interactions and back-and-forth with expert teachers. Inspired by this observation, this project builds systems that can learn by asking detailed questions of experts.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.