CAREER: Brain-inspired Methods for Continual Learning of Large-scale Vision and Language Tasks

职业：持续学习大规模视觉和语言任务的受大脑启发的方法

• 批准号: 2047556
• 负责人: Christopher Kanan
• 金额: $ 55万
• 依托单位: Rochester Institute of Tech
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-12-01 至 2023-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2047556&HistoricalAwards=false
• 关键词: CAREER; Brain; inspired; Methods; Continual

The goal of this research project is to create deep neural networks that excel in a broad set of circumstances, are capable of learning from new data over time, and are robust to dataset bias. Deep neural networks can now perform some tasks as well as humans, such as identifying faces, recognizing objects, and other perception tasks. However, existing approaches have limitations, including the inability to effectively learn over time when data is structured without forgetting past information, learning slowly by looping over data many times, and amplification of pre-existing dataset bias which results in erroneous predictions for groups with less data. To overcome these problems, this research project aims to incorporate memory consolidation processes inspired by the mammalian memory system that occur both when animals are awake and asleep. The new methods developed in this project could lead to machine learning systems that 1) are more power efficient, 2) can learn on low-powered mobile devices and robots, and 3) can overcome bias in datasets. In addition, a significant educational component involves training the next generation of scientists and engineers in deploying machine learning systems that are safe, reliable, and well tested via new courses and programs.In greater technical detail, this project will develop new measures for neural networks to 1) test for biases, 2) assess the acquisition of robust concepts, and 3) study forward transfer in neural networks trained over time. New brain-inspired algorithms are proposed that learn online but then have downtime periods in which they engage in greater levels of memory consolidation, which are informed by findings in neuroscience for the neural activities that occur during the wake-sleep cycles of humans and other mammals. The proposed algorithms are based on the brain's complementary learning systems for memory formation, storage, and retrieval. The models are evaluated on large-scale incremental image classification tasks as well as tasks involving multi-modal scene understanding and abstract reasoning. This research will provide building blocks that others can use to create new algorithms and applications. All code and datasets will be made publicly available.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该研究项目的目标是创建深度神经网络，该网络在广泛的环境中表现出色，能够随着时间的推移从新数据中学习，并且对数据集偏差具有鲁棒性。深度神经网络现在可以像人类一样执行一些任务，例如识别人脸，识别物体和其他感知任务。然而，现有的方法具有局限性，包括当数据被结构化而不忘记过去的信息时，无法随着时间的推移有效地学习，通过多次循环数据来缓慢地学习，以及放大预先存在的数据集偏差，这导致对具有较少数据的组的错误预测。为了克服这些问题，该研究项目旨在将受哺乳动物记忆系统启发的记忆巩固过程纳入动物清醒和睡眠时都会发生的记忆巩固过程。该项目中开发的新方法可能会导致机器学习系统1）更节能，2）可以在低功耗的移动的设备和机器人上学习，3）可以克服数据集中的偏见。此外，一个重要的教育组成部分涉及培训下一代科学家和工程师部署机器学习系统，这些系统是安全的，可靠的，并通过新的课程和计划进行了良好的测试。在更详细的技术细节中，该项目将开发神经网络的新措施，以1）测试偏差，2）评估稳健概念的获得，3）研究随时间训练的神经网络中的前向传递。人们提出了新的大脑启发算法，可以在线学习，但随后会有停机时间，在此期间它们会进行更高水平的记忆巩固，这是由神经科学对人类和其他哺乳动物醒-睡周期期间发生的神经活动的发现所提供的信息。所提出的算法是基于大脑的互补学习系统的记忆形成，存储和检索。这些模型在大规模增量图像分类任务以及涉及多模态场景理解和抽象推理的任务上进行了评估。这项研究将提供其他人可以用来创建新算法和应用程序的构建块。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

OccamNets: Mitigating Dataset Bias by Favoring Simpler Hypotheses

• DOI: 10.48550/arxiv.2204.02426
• 发表时间: 2022-04
• 期刊: Euphytica
• 影响因子: 1.9
• 作者: Robik Shrestha;Kushal Kafle;Christopher Kanan

Online Continual Learning for Embedded Devices

• DOI: 10.48550/arxiv.2203.10681
• 发表时间: 2022-03
• 期刊: ArXiv
• 作者: Tyler L. Hayes;Christopher Kanan

How efficient are today’s continual learning algorithms?

当今持续学习算法的效率如何？

• 发表时间: 2023
• 期刊: CVPR Workshop on Continual Learning in Computer Vision (CLVISION
• 作者: Harun, M.Y.;Gallardo, J.;Hayes, T.L.;Kanan, C.
• 通讯作者: Kanan, C.

BiasedMNIST

有偏见的MNIST

• DOI: 10.35009/cfccis-ay07
• 发表时间: 2022
• 期刊: European Conference on Computer Vision (ECCV
• 作者: Shrestha, Robik;Kafle, Kushal;Kanan, Christopher
• 通讯作者: Kanan, Christopher

Detecting Out-Of-Context Objects Using Graph Context Reasoning Network

使用图上下文推理网络检测脱离上下文的对象

• 发表时间: 2022
• 期刊: IJCAI
• 作者: Acharya, M;Roy, A;Koneripalli, K;Jha, Susmit;Kanan, C;Divakaran, A
• 通讯作者: Divakaran, A