III: Small: Modular structures in the brain and artificial learningsystems: emergence and function

深度学习已经取得了长足的进步，人工神经网络在专门任务上进行了详尽的训练后，表现优于人类。然而，快速学习和灵活的智力，生物大脑的标志，仍然遥不可及。在这个提议中，我们试图理解生物大脑的一个重要特征，这对于更好、更可解释的人工智能至关重要：模块化架构的存在，这些架构以丰富的方式组合在一起，用一些共享的子结构来解决多个问题。我们将研究模块化结构的三个方面：1）通过对神经元可以延伸多远以及可以形成多少突触的简单和局部限制，如何在最小的训练下产生大脑中观察到的模块化结构; 2）在已知神经回路和功能的背景下，在鲁棒性和学习速度方面，神经回路中的模块化组织在解决任务中的效用是什么; 3）如何将我们在1）-2）中学习的导致快速模块化的模块化架构和原理导入人工神经网络，以提高机器智能的学习速度和灵活性。因此，我们寻求更好地理解大脑，在神经科学和人工智能之间建立更强有力的对话，并利用由此产生的见解来提高机器智能。该补助金将为学生和博士后研究员提供对工业、政府和科学感兴趣的前沿领域的培训机会，我们将专注于在这些领域培养高技能和多样化的劳动力。通过重新组合较简单的子功能来学习和执行复杂认知功能的能力是心智能力的基础，并且是一般智力的基础。这种能力的基础是大脑中模块化结构的存在。模块化结构具有固有的优势，例如增加对扰动的稳定性和更快的学习。我们将研究自然和人工系统中模块化出现的机制。现有的模块化模型主要基于自上而下的监督学习，这需要大量的学习时间和数据。我们的中心假设是，大脑中连接的局部约束提供了对模块化的强烈先验偏见，这些导致模块化结构和功能的快速出现。首先，我们将使用理论和计算工具来模拟低水平的约束，以探索它们如何驱动大脑中观察到的神经回路的模块化，包括内嗅皮层中的网格细胞。其次，我们将通过研究直接受到内鼻海马电路启发的高容量存储器架构的特性，分析生物系统中模块化组织所带来的优势，具有类似MEC的模块化子网络。第三，我们将联合收割机结合我们开发的模块化机制和对生物结构电路的理解，将类似的优势引入人工学习系统，创建人工神经网络，为复杂的现实世界任务提供强大的模块化解决方案。因此，我们的工作将有助于描述生物和人工网络如何自发地模块化，以支持强大而有效的推理和learning.This奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估的支持。

Deep learning has made great strides, with artificial neural networks outperforming humans after training exhaustively on specialized tasks. However, rapid learning and flexible intelligence, hallmarks of biological brains, remain out of reach. In this proposal, we seek to understand an important feature of biological brains that will be critical for better and more interpretable artificial intelligence: the existence of modular architectures that are combined in rich ways to solve multiple problems with some shared sub-structure. We will study three aspects of modular architecture: 1) How observed modular structures in the brain might arise with minimal training, through simple and local constraints on how far a neuron can extend and how many synapses it can make; 2) What is the utility of a modular organization in neural circuits in solving tasks, in terms of robustness and the speed of learning, in the context of known neural circuits and function; 3) How the modular architectures and principles that lead to rapid modularization we learn about in 1)-2) can be imported into artificial neural networks to improve the learning speed and flexibility of machine intelligence. Thus, we seek to better understand brains, build a stronger dialogue between neuroscience and artificial intelligence, and use the resulting insights to improve machine intelligence. The grant will provide training opportunities to students and postdoctoral fellows in cutting-edge areas of strong interest for industry, government, and science, and we will focus on training a highly skilled and diverse workforce in these areas.Compositionality, i.e., the ability to learn and perform complex cognitive function by re-combining simpler sub-functions, is fundamental to the capabilities of the mind and is the basis for general intelligence. Underlying this ability is the presence of modular structures in the brain. Modular structures confer inherent advantages, such as increased stability to perturbations and faster learning. We will investigate mechanisms for the emergence of modularity in natural and artificial systems. Existing models for modularity are based primarily on top-down supervised learning, which requires large amounts of learning time and data. Our central hypothesis is that local constraints on connectivity in the brain provide strong prior biases towards modularization, and these lead to the rapid emergence of modular structure and improved function. First, we will use theoretical and computational tools to model low-level constraints to probe how they may drive modularization in neural circuits observed in the brain, including grid cells in the entorhinal cortex. Second, we will analyze the advantages conferred by modular organization in biological systems by studying the properties of high-capacity memory architectures directly inspired by the entorhinal-hippocampal circuit, with mEC-like modular subnetworks. Third, we will combine our developed modularization mechanisms and understanding of biological architectural circuitry to import similar advantages into artificial learning systems, creating artificial neural networks that achieve robust modular solutions to complex real-world tasks. Thus, our work will help to characterize how biological and artificial networks may spontaneously modularize to support robust and efficient inference and learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.