CAREER: Brain-inspired Methods for Continual Learning of Large-scale Vision and Language Tasks

该研究项目的目标是创建深度神经网络，该网络在广泛的环境中表现出色，能够随着时间的推移从新数据中学习，并且对数据集偏差具有鲁棒性。深度神经网络现在可以像人类一样执行一些任务，例如识别人脸，识别物体和其他感知任务。然而，现有的方法具有局限性，包括当数据被结构化而不忘记过去的信息时，无法随着时间的推移有效地学习，通过多次循环数据来缓慢地学习，以及放大预先存在的数据集偏差，这导致对具有较少数据的组的错误预测。为了克服这些问题，该研究项目旨在将受哺乳动物记忆系统启发的记忆巩固过程纳入动物清醒和睡眠时都会发生的记忆巩固过程。该项目中开发的新方法可能会导致机器学习系统1）更节能，2）可以在低功耗的移动的设备和机器人上学习，3）可以克服数据集中的偏见。此外，一个重要的教育组成部分涉及培训下一代科学家和工程师部署机器学习系统，这些系统是安全的，可靠的，并通过新的课程和计划进行了良好的测试。在更详细的技术细节中，该项目将开发神经网络的新措施，以1）测试偏差，2）评估稳健概念的获得，3）研究随时间训练的神经网络中的前向传递。新的受大脑启发的算法被提出，它们可以在线学习，但随后会有停机时间，在此期间它们会进行更高水平的记忆巩固，这是由神经科学中关于人类和其他哺乳动物在觉醒-睡眠周期期间发生的神经活动的发现所告知的。所提出的算法是基于大脑的互补学习系统的记忆形成，存储和检索。这些模型在大规模增量图像分类任务以及涉及多模态场景理解和抽象推理的任务上进行了评估。这项研究将提供其他人可以用来创建新算法和应用程序的构建块。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The goal of this research project is to create deep neural networks that excel in a broad set of circumstances, are capable of learning from new data over time, and are robust to dataset bias. Deep neural networks can now perform some tasks as well as humans, such as identifying faces, recognizing objects, and other perception tasks. However, existing approaches have limitations, including the inability to effectively learn over time when data is structured without forgetting past information, learning slowly by looping over data many times, and amplification of pre-existing dataset bias which results in erroneous predictions for groups with less data. To overcome these problems, this research project aims to incorporate memory consolidation processes inspired by the mammalian memory system that occur both when animals are awake and asleep. The new methods developed in this project could lead to machine learning systems that 1) are more power efficient, 2) can learn on low-powered mobile devices and robots, and 3) can overcome bias in datasets. In addition, a significant educational component involves training the next generation of scientists and engineers in deploying machine learning systems that are safe, reliable, and well tested via new courses and programs.In greater technical detail, this project will develop new measures for neural networks to 1) test for biases, 2) assess the acquisition of robust concepts, and 3) study forward transfer in neural networks trained over time. New brain-inspired algorithms are proposed that learn online but then have downtime periods in which they engage in greater levels of memory consolidation, which are informed by findings in neuroscience for the neural activities that occur during the wake-sleep cycles of humans and other mammals. The proposed algorithms are based on the brain's complementary learning systems for memory formation, storage, and retrieval. The models are evaluated on large-scale incremental image classification tasks as well as tasks involving multi-modal scene understanding and abstract reasoning. This research will provide building blocks that others can use to create new algorithms and applications. All code and datasets will be made publicly available.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.