CAREER: Brain-inspired Methods for Continual Learning of Large-scale Vision and Language Tasks

该研究项目的目标是创建深度神经网络，使其在广泛的环境中表现出色，能够随着时间的推移从新数据中学习，并且对数据集偏差具有鲁棒性。深度神经网络现在可以像人类一样完成一些任务，比如识别人脸、识别物体和其他感知任务。然而，现有的方法有局限性，包括当数据结构化时无法有效地随着时间的推移学习而不忘记过去的信息，通过多次循环数据来缓慢学习，以及扩大已有的数据集偏差，从而导致对数据较少的群体的错误预测。为了克服这些问题，该研究项目旨在结合哺乳动物在清醒和睡眠时发生的记忆系统所启发的记忆巩固过程。在这个项目中开发的新方法可能会导致机器学习系统，1)更节能，2)可以在低功率的移动设备和机器人上学习，3)可以克服数据集的偏见。此外，一个重要的教育组成部分包括培训下一代科学家和工程师，通过新的课程和项目部署安全、可靠、经过良好测试的机器学习系统。在更大的技术细节上，该项目将为神经网络开发新的测量方法，以1)测试偏差，2)评估鲁棒概念的获取，以及3)研究经过时间训练的神经网络的前向迁移。有人提出了一种新的受大脑启发的算法，这种算法可以在线学习，但随后会有一段停机时间，在这段时间里，它们会进行更高水平的记忆巩固，这是神经科学对人类和其他哺乳动物醒-睡周期中发生的神经活动的研究结果所提供的信息。提出的算法是基于大脑的互补学习系统的记忆形成，存储和检索。在大规模增量图像分类任务以及涉及多模态场景理解和抽象推理的任务中对模型进行了评估。这项研究将为其他人提供构建模块，用于创建新的算法和应用程序。所有代码和数据集都将公开提供。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The goal of this research project is to create deep neural networks that excel in a broad set of circumstances, are capable of learning from new data over time, and are robust to dataset bias. Deep neural networks can now perform some tasks as well as humans, such as identifying faces, recognizing objects, and other perception tasks. However, existing approaches have limitations, including the inability to effectively learn over time when data is structured without forgetting past information, learning slowly by looping over data many times, and amplification of pre-existing dataset bias which results in erroneous predictions for groups with less data. To overcome these problems, this research project aims to incorporate memory consolidation processes inspired by the mammalian memory system that occur both when animals are awake and asleep. The new methods developed in this project could lead to machine learning systems that 1) are more power efficient, 2) can learn on low-powered mobile devices and robots, and 3) can overcome bias in datasets. In addition, a significant educational component involves training the next generation of scientists and engineers in deploying machine learning systems that are safe, reliable, and well tested via new courses and programs.In greater technical detail, this project will develop new measures for neural networks to 1) test for biases, 2) assess the acquisition of robust concepts, and 3) study forward transfer in neural networks trained over time. New brain-inspired algorithms are proposed that learn online but then have downtime periods in which they engage in greater levels of memory consolidation, which are informed by findings in neuroscience for the neural activities that occur during the wake-sleep cycles of humans and other mammals. The proposed algorithms are based on the brain's complementary learning systems for memory formation, storage, and retrieval. The models are evaluated on large-scale incremental image classification tasks as well as tasks involving multi-modal scene understanding and abstract reasoning. This research will provide building blocks that others can use to create new algorithms and applications. All code and datasets will be made publicly available.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.