Computations and Unsupervised Learning in Recurrent Neural Networks

循环神经网络中的计算和无监督学习

• 批准号: 7313129
• 负责人: DEAN V BUONOMANO
• 金额: $ 7.25万
• 依托单位: UNIVERSITY OF CALIFORNIA LOS ANGELES
• 依托单位国家: 美国
• 财政年份: 2007
• 资助国家: 美国
• 起止时间: 2007-07-03 至 2009-06-30
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/7313129
• 关键词: Accounting; Animal Vocalization; Attention; Autistic Disorder; Be++ element; Behavior; Behavior Control; Beryllium; Biological Neural Networks; Brain; Cerebral cortex; Chromosome Pairing; Class; Classification; Code; Cognition; Complex; Computer information processing; Data; Discrimination; Disease; Dyslexia; Educational process of instructing; Epilepsy; Equilibrium; Fire - disasters; Fragile X Syndrome; Goals; Human; Inhibitory Synapse; Lead; Learning; Light; Modeling; Nature; Neurons; Neurosciences; Numbers; Output; Pattern; Play; Population; Process; Property; Rate; Reading; Recurrence; Research; Role; Sensory Process; Space Perception; Speech; Staging; Stimulus; Synapses; Synaptic plasticity; System; Thinking; Time; Training; computational neuroscience; experience; feeding; inhibitory neuron; large scale simulation; man; millimeter; nervous system disorder; neuromechanism; novel; relating to nervous system; response

DESCRIPTION (provided by applicant): The human brain in general, and the cerebral cortex in particular, remains the most sophisticated computational system known to man. Elucidating the mechanisms underlying the cerebral cortex's ability to process information is critical for understanding both normal cortical processing, and a myriad of neurological disorders produced by abnormal cortical function. The current research begins with the assumption that a necessary step towards understanding cortical processing will be to elucidate two related problems: temporal processing and neural dynamics. First, temporal processing (the decoding of temporal information) is of fundamental importance for most forms of sensory processing, perhaps most notably speech. Yet the neural mechanisms underlying even simple forms of temporal computations are not known. Second, neural dynamics (evolving changes in the spatial-temporal patterns of neural activity) is thought to play a pivotal role in many forms of neural computation, including temporal processing. However, given the inherently complex and highly nonlinear nature of neural activity in recurrent networks composed of spiking neurons, we do not yet understand (1) how dynamics emerges and is controlled (e.g. to avoid epileptic activity), and (2) how dynamics is tuned in an experience- dependent manner to allow learning. It is generally assumed that synaptic plasticity (as well as plasticity at other loci) ultimately underlies the control and behavior of cortical networks. However, while a number of forms of plasticity have provided powerful learning rules when implemented into feed-forward networks, the implementation of effective learning rules in recurrent networks has proven largely intractable. In the current proposal we will examine whether multiple learning rules in parallel, operating synergistically, can lead to effective learning in recurrent networks. We will pay particular attention to the simultaneous control of excitatory and inhibitory synapses, which is hypothesized to be critical in recurrent networks. The proposal consists of two Aims. In the first Aim we will use a set of synaptic learning rules to train a large-scale recurrent network to develop stable and robust responses to spoken phonemes - in other words to encode complex stimuli in a spatial-temporal pattern of spikes. The second Aim will use a novel learning rule to teach output units to respond selectively to classes of phonemes - that is, to decode the patterns of the cortical network. If progress is made in this direction the research described here will enhance not only our understanding of normal cortical processing, but in the understanding of abnormal cortical states that arise in neurological disorders characterized by abnormal temporal processing and neural dynamics; such diseases include autism, Fragile X, dyslexia, and epilepsy. Behavior and cognition are ultimately an emergent property of the dynamic interaction of hundreds of thousands of neurons embedded in complex networks. While significant progress has been made towards understanding cellular and synaptic properties in isolation, elucidating how the activity of hundreds of thousands of neurons underlie cortical computations remains an elusive and fundamental goal in neuroscience. The research described here will use large-scale simulations of cortical networks to define the synaptic learning rules that allow computations to emerge from recurrent cortical networks. Specifically, the ability of artificial neural networks to learn to discriminate spoken phonemes will be examined. Understanding how computations emerge from massive networks of interconnected elements is a necessary step towards understanding normal cortical function, as well as the pathological cortical abnormalities observed in a myriad of neurological disorders.

描述（由申请人提供）：一般来说，人类大脑，特别是大脑皮层，仍然是人类已知的最复杂的计算系统。阐明大脑皮层处理信息能力的机制对于理解正常的皮层处理和由异常皮层功能产生的无数神经障碍都是至关重要的。当前的研究始于这样的假设：理解皮质处理的必要一步是阐明两个相关问题：时间处理和神经动力学。首先，时间处理（时间信息的解码）对于大多数形式的感觉处理都是至关重要的，也许最明显的是语音。然而，即使是简单形式的时间计算背后的神经机制也是未知的。其次，神经动力学（神经活动的时空模式的演变变化）被认为在许多形式的神经计算中起着关键作用，包括时间处理。然而，考虑到由尖峰神经元组成的递归网络中神经活动的固有复杂性和高度非线性性质，我们还不了解（1）动态如何出现和控制（例如，以避免癫痫活动），以及（2）如何以经验依赖的方式调整动态以允许学习。一般认为，突触可塑性（以及其他部位的可塑性）最终是皮层网络控制和行为的基础。然而，虽然许多形式的可塑性在前馈网络中实现时提供了强大的学习规则，但在递归网络中实现有效的学习规则在很大程度上被证明是棘手的。在当前的提案中，我们将研究并行的多个学习规则，协同操作，是否可以导致循环网络中的有效学习。我们将特别注意兴奋性和抑制性突触的同时控制，这被假设为是关键的复发网络。该提案包括两个目标。在第一个目标中，我们将使用一组突触学习规则来训练一个大规模的递归网络，以发展对口语音素的稳定和鲁棒的反应-换句话说，以时空模式的尖峰编码复杂的刺激。第二个目标将使用一种新的学习规则来教输出单元选择性地对音素类别做出反应-也就是说，解码皮层网络的模式。如果在这个方向上取得进展，这里描述的研究将不仅提高我们对正常皮层处理的理解，而且提高我们对异常皮层状态的理解，这些异常皮层状态出现在以异常时间处理和神经动力学为特征的神经系统疾病中;这些疾病包括自闭症，脆性X染色体，阅读障碍和癫痫。行为和认知最终是嵌入复杂网络中的数十万神经元动态交互的一种涌现特性。虽然在孤立地理解细胞和突触特性方面取得了重大进展，但阐明数十万神经元的活动如何构成皮层计算的基础仍然是神经科学中一个难以捉摸的基本目标。这里描述的研究将使用大规模的皮层网络模拟来定义突触学习规则，这些规则允许计算从递归皮层网络中出现。具体来说，人工神经网络学习辨别口语音素的能力将被检查。理解计算是如何从大量相互连接的元素网络中产生的，是理解正常皮层功能以及在无数神经系统疾病中观察到的病理性皮层异常的必要步骤。