CAREER: Foundations of Small Data

该奖项全部或部分由2021年美国救援计划法案（公法117-2）资助。深度学习是一种技术，其中构建人工神经网络，模仿生物大脑中神经元的工作。这种技术推动了当今广泛的任务，例如，在手机上打字时预测下一个单词，用照片中的人名标记照片，转录语音等。构建人工网络需要从这些任务中的每一个收集大量数据。但是，随着我们试图将深度学习应用于越来越多样化的任务，从每个任务中收集如此大量的数据变得越来越困难。例如，一些语言或方言的使用者比英语或西班牙语少得多，因此它们的数据更加稀缺。这种数据稀缺在临床科学等领域更为严重。该项目的目标是开发理论和计算工具，使人工神经网络即使在数据很少的情况下也能很好地工作。该项目的教育和推广目标包括：（a）为研究生和本科生开发新课程;（B）指导在计算机科学、物理学和工程学等既定学科工作的学员;（c）在大费城地区的高中、高等教育机构和工业界培养机器学习生态系统。这个项目将发展对学习任务的基本理解。它将研究典型的学习任务如何具有某种有效的低维结构，使深度网络能够有效地学习这些任务。它试图描述适合典型任务的预测模型的函数空间的几何特征，以了解学习一项任务何时有助于或无助于减少学习另一项任务所需的数据量。它旨在利用这种几何结构来构建贝叶斯先验，自动适应可用数据的数量。预计这种方法将减少训练所需的标记数据量高达1000倍。该理论将用于开发转移、多任务和持续学习的新方法，以及能够准确诊断阿尔茨海默病的工具。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).Deep Learning is a technique where one builds an artificial neural network that mimics the working of neurons in the biological brain. This technique drives a wide range of tasks today, e.g., predicting the next word while typing on the phone, tagging photos with the names of people in it, transcribing speech, etc. Building the artificial network requires collecting a large amount of data from each of these tasks. But as we seek to apply deep learning to more and more diverse tasks, it is becoming difficult to collect such large amounts of data from every task. For example, a number of languages or dialects have much fewer speakers than English or Spanish, and so their data is more scarce. This data scarcity is even more acute in domains such as the clinical sciences. The goal of this project is to develop theoretical and computational tools that enable artificial neural networks to work well even with few data. Educational and outreach goals of this project include (a) development of new curricula for graduate and undergraduate students, (b) mentoring trainees who work across established disciplines such as computer science, physics and engineering, and (c) fostering an ecosystem for machine learning across high-schools, higher-educational institutions and industry in the Greater Philadelphia region.In order to achieve these goals, this project will develop a foundational understanding of learning tasks. It will study how typical learning tasks have a certain effective low-dimensional structure that enables deep networks to learn such tasks efficiently. It seeks to characterize the geometry of the function space of predictive models fitted on typical tasks to understand when learning one task helps, or does not help, reduce the amount of data required to learn another task. It aims to exploit this geometry to build Bayesian priors that automatically adapt to the amount of available data. It is expected that such methods will reduce the amount of labeled data required for training by up to 1000 times. This theory will be used to develop new methods for transfer, multi-task and continual learning, and tools that enable accurate diagnosis of Alzheimer’s Disease.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.