CAREER: Bootstrapping Recognition from Little Data in New Domains

该奖项部分由2021年美国救援计划法案（公法117-2）资助。利用目前的计算机视觉技术，人们可以很容易地构建出能够识别数千种物体的软件，无论是猫、汽车还是鸟类。然而，这样的系统必须在数以百万计的图像的大数据集上进行训练，这些图像已经被人类注释者煞费苦心地标记了。对于许多应用领域，如显微镜，这样的大型标记数据集很难创建，在这些领域，注释者需要经过高度训练。收集大型数据集也可能与隐私问题相冲突，并且可能需要昂贵的管理来消除种族、性别或其他类型的偏见。最后，收集大型数据集的高昂成本使得许多人无法使用计算机视觉技术。该项目开发的识别系统技术可以在不需要任何大型数据集的情况下成功识别困难的视觉概念。能够使用有限训练数据的识别系统将解锁许多下游应用，特别是在专业领域，并将使所有人都能获得计算机视觉技术的进步。项目团队将使这些识别系统广泛地向所有人开放，为来自代表性不足社区的高中生组织一个研讨会，并开发一个新的计算机视觉课程，重点关注计算机视觉技术的广泛应用。为了从少量数据（少至1-2张标记图像和约1000张未标记图像）构建识别系统，该项目探索了两种受人类视觉启发的策略。首先，与目前针对每个问题领域进行单独训练的识别系统不同，人类在其丰富的先前视觉经验的背景下学习执行新的视觉任务（例如分析显微镜图像）。这个项目同样设计了视觉学习者，通过利用跨多个领域的过去视觉任务的记忆，从有限的数据中学习新的识别任务。其次，与目前只从标记图像中学习的系统不同，人类通过丰富的互动和与专家教师的反复交流来学习。受这一观察结果的启发，该项目构建了可以通过向专家询问详细问题来学习的系统。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This award is funded in part under the American Rescue Plan Act of 2021 (Public Law 117-2).With current computer vision technology, one can easily build software that can recognize thousands of objects, be they cats, cars or birds. However, such systems must be trained on large datasets of millions of images which have been painstakingly labeled by human annotators. Such large, labeled datasets are difficult to create for many application domains, such as microscopy, where annotators will need to be highly trained. Collecting large datasets may also run afoul of privacy concerns, and may need expensive curation to remove racial, gender or other kinds of bias. Finally, the steep cost of collecting large datasets makes computer vision technology inaccessible to many. This project develops technologies for recognition systems that can successfully identify difficult visual concepts without needing any large datasets. Recognition systems that can work with limited training data will unlock many downstream applications, especially in specialized domains, and will make advances in computer vision technology accessible to all. The project team will make these recognition systems broadly accessible to all, organize a workshop for high school students from underrepresented communities, and develop a new computer vision curriculum that focuses on broad applications of computer vision technology.To build recognition systems from little data (as few as 1-2 labeled images and around 1000 unlabeled images), this project explores two strategies inspired from human vision. First, unlike current recognition systems that are trained in isolation for each problem domain, humans learn to perform new visual tasks (such as analyzing microscopy images) in the context of their vast prior visual experience. This project similarly designs visual learners that learn new recognition tasks from limited data by leveraging a memory of past visual tasks across multiple domains. Second, unlike current systems that learn only from labeled images, humans learn through rich interactions and back-and-forth with expert teachers. Inspired by this observation, this project builds systems that can learn by asking detailed questions of experts.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.