CIF: RI: Medium: Design principles and theory for data augmentation

泛化，或将知识从一种环境转移到另一种环境的能力，是人类智能的一个标志。然而，在人工智能(AI)中，在一种设置下训练的模型在新设置下测试时往往会失败，即使这种变化很小或难以察觉。为了构建更具通用性的人工智能，大多数现代方法采用某种形式的数据增强(DA)，即对数据应用转换以创建虚拟样本，然后将其添加到数据集。由此产生的新示例的合成似乎在人工智能中建立了有用的属性，例如对某些自然转换的不变性或抵抗力，以及对新任务和现有任务中的噪声的稳健性。尽管DA程序有希望和性能，但它们大多是以特别方式应用的，需要在逐个数据集的基础上进行设计和测试。缺乏一套基本原则和理论来理解DA及其对模型训练和测试的影响。为了解决这个突出的挑战，研究人员将提供对DA对泛化的影响的准确理解，并利用这种理解来设计可以跨多个应用和领域使用的新颖的扩充。在这个项目中，研究人员提出了一个原则性的数学框架，以1)理解DA何时帮助以及何时DA可能损害学习，2)理解DA诱导的结构并表征是什么构成了高质量的扩充，以及3)提供新颖、系统和可扩展的设计原则来在我们缺乏先验知识的新领域中扩充数据。这些设计原则将极大地扩大DA的适用性和前景，使其从计算机视觉扩展到新的领域(例如，神经数据、图表和表格数据)，在这些领域，原则性的增强仍然未知。这个项目的特别焦点将是DA在神经活动中的应用，在这方面，增强已经显示出在大脑和行为之间建立更普遍的联系的前景。这项研究还将为DA在促进现代机器学习中的公平性、问责制和透明度方面的作用开出处方。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Generalization, or the ability to transfer knowledge from one context to the next, is a hallmark of human intelligence. In artificial intelligence (AI), however, models trained in one setting often fail when tested in a new setting, even if the shift is minor or imperceptible. To build more generalizable AI, most modern methods employ some form of data augmentation (DA), which applies transformations to the data to create virtual samples that are then added to the dataset. The resulting synthesis of new examples appears to build helpful properties in AI such as invariance or resistance to change to certain natural transformations, and robustness to new tasks as well as noise in existing tasks. Despite the promise and performance of DA procedures, they are mostly applied in an ad-hoc manner and need to be designed and tested on a dataset by dataset basis. A set of fundamental principles and theory to understand DA and its impact on model training and testing is lacking. To address this outstanding challenge, the investigators will provide a precise understanding of the impact of DA on generalization, and leverage this understanding to design novel augmentations that can be used across multiple applications and domains.In this project, the investigators propose a principled mathematical framework to 1) understand when DA helps and when DA could potentially hurt learning, 2) understand the structure induced by DA and characterize what makes high-quality augmentations, and 3) provide novel, systematic, and scalable design principles for augmenting data in new domains where we lack prior knowledge to guide us. These design principles will significantly broaden the applicability and promise of DA from computer vision to new domains (e.g., neural data, graphs and tabular data) where principled augmentations are still not known. Of special focus in this project will be applications of DA to neural activity, where augmentations have shown promise in building a more generalizable link between the brain and behavior. This research will also yield prescriptions for the role of DA in advancing fairness, accountability and transparency in modern machine learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.