III: Small: Modular structures in the brain and artificial learningsystems: emergence and function

深度学习已经取得了长足的进步，人工神经网络在对专门任务进行了详尽的训练后，表现优于人类。然而，作为生物大脑的标志，快速学习和灵活的智力仍然遥不可及。在这项提议中，我们试图了解生物大脑的一个重要特征，这将是更好和更具解释力的人工智能的关键：存在以丰富的方式组合在一起的模块化架构，以解决具有某些共享子结构的多个问题。我们将研究模块体系结构的三个方面：1)如何通过对神经元可以延伸到多远以及它可以制造多少突触的简单和局部限制，在大脑中观察到的模块结构可能会出现；2)在已知神经电路和功能的背景下，神经电路中的模块组织在解决任务方面的稳健性和学习速度有什么用处；3)如何将我们在1)-2)中了解到的导致快速模块化的模块体系结构和原则引入人工神经网络，以提高机器智能的学习速度和灵活性。因此，我们寻求更好地了解大脑，在神经科学和人工智能之间建立更强有力的对话，并利用由此产生的见解来改进机器智能。这笔拨款将为学生和博士后研究员提供对工业、政府和科学有浓厚兴趣的前沿领域的培训机会，我们将重点培训这些领域的高技能和多样化的劳动力。合成能力，即通过重新组合较简单的子功能来学习和执行复杂认知功能的能力，是智力的基础和基础。这种能力的基础是大脑中模块化结构的存在。模块化结构赋予了固有的优势，例如增加了对扰动的稳定性和更快的学习。我们将研究自然和人工系统中模块化出现的机制。现有的模块化模型主要基于自上而下的监督学习，这需要大量的学习时间和数据。我们的中心假设是，大脑中连接性的局部限制提供了强烈的先前对模块化的偏见，这些导致模块化结构和改进的功能的快速出现。首先，我们将使用理论和计算工具来模拟低水平的约束，以探索它们如何驱动在大脑中观察到的神经回路的模块化，包括内嗅觉皮质中的网格细胞。其次，我们将通过研究直接受到内嗅觉-海马体电路启发的具有MEC样子网络的大容量记忆结构的特性，来分析模块化组织在生物系统中所赋予的优势。第三，我们将结合我们开发的模块化机制和对生物结构电路的理解，将类似的优势引入人工学习系统，创建人工神经网络，实现针对复杂现实世界任务的稳健模块化解决方案。因此，我们的工作将有助于表征生物和人工网络如何自发地模块化，以支持稳健和高效的推理和学习。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Deep learning has made great strides, with artificial neural networks outperforming humans after training exhaustively on specialized tasks. However, rapid learning and flexible intelligence, hallmarks of biological brains, remain out of reach. In this proposal, we seek to understand an important feature of biological brains that will be critical for better and more interpretable artificial intelligence: the existence of modular architectures that are combined in rich ways to solve multiple problems with some shared sub-structure. We will study three aspects of modular architecture: 1) How observed modular structures in the brain might arise with minimal training, through simple and local constraints on how far a neuron can extend and how many synapses it can make; 2) What is the utility of a modular organization in neural circuits in solving tasks, in terms of robustness and the speed of learning, in the context of known neural circuits and function; 3) How the modular architectures and principles that lead to rapid modularization we learn about in 1)-2) can be imported into artificial neural networks to improve the learning speed and flexibility of machine intelligence. Thus, we seek to better understand brains, build a stronger dialogue between neuroscience and artificial intelligence, and use the resulting insights to improve machine intelligence. The grant will provide training opportunities to students and postdoctoral fellows in cutting-edge areas of strong interest for industry, government, and science, and we will focus on training a highly skilled and diverse workforce in these areas.Compositionality, i.e., the ability to learn and perform complex cognitive function by re-combining simpler sub-functions, is fundamental to the capabilities of the mind and is the basis for general intelligence. Underlying this ability is the presence of modular structures in the brain. Modular structures confer inherent advantages, such as increased stability to perturbations and faster learning. We will investigate mechanisms for the emergence of modularity in natural and artificial systems. Existing models for modularity are based primarily on top-down supervised learning, which requires large amounts of learning time and data. Our central hypothesis is that local constraints on connectivity in the brain provide strong prior biases towards modularization, and these lead to the rapid emergence of modular structure and improved function. First, we will use theoretical and computational tools to model low-level constraints to probe how they may drive modularization in neural circuits observed in the brain, including grid cells in the entorhinal cortex. Second, we will analyze the advantages conferred by modular organization in biological systems by studying the properties of high-capacity memory architectures directly inspired by the entorhinal-hippocampal circuit, with mEC-like modular subnetworks. Third, we will combine our developed modularization mechanisms and understanding of biological architectural circuitry to import similar advantages into artificial learning systems, creating artificial neural networks that achieve robust modular solutions to complex real-world tasks. Thus, our work will help to characterize how biological and artificial networks may spontaneously modularize to support robust and efficient inference and learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.