The gradient clusteron: A model neuron that learns to solve classification tasks via dendritic nonlinearities, structural plasticity, and gradient descent.
The gradient clusteron: A model neuron that learns to solve classification tasks via dendritic nonlinearities, structural plasticity, and gradient descent.
复制标题
DOI：
10.1371/journal.pcbi.1009015
复制
发表时间：
2021-05
影响因子：
4.3
通讯作者：
Segev I
中科院分区：
文献类型：
--
作者：
Moldwin T;Kalmenson M;Segev I
神经元树突上的突触簇被认为在模式识别中起着重要的作用。树突状分支上的相邻突触可以通过非线性电压依赖性机制（如NMDA受体）以协同、合作的方式相互作用。受到NMDA受体的启发，单分支clusteron学习算法利用位置依赖性乘法非线性来解决分类任务，方法是在学习过程中随机打乱模型树突上“表现不佳”突触的位置（“结构可塑性”），最终导致具有相关活动的突触在树突上彼此相邻。我们提出了一种替代模型，梯度clusteron，或G-clusteron，它使用一个解析推导的梯度下降规则，突触是“吸引”或“排斥”彼此在输入和位置相关的方式。我们通过在MNIST手写数字数据集上进行测试来证明该算法的分类能力，并表明，当使用softmax激活函数时，G-clusteron在所有与所有MNIST任务上的准确性（~85%）接近逻辑回归（~93%）。除了位置更新规则，我们还推导出一个学习规则的突触权重的G-clusteron（“功能可塑性”），并表明，一个G-clusteron，利用权重更新规则可以达到约89%的准确度MNIST任务。我们还表明，一个G-clusteron的重量和位置更新规则可以学习解决XOR问题，从任意的初始条件。人工神经网络（ANN）已成为人工智能和机器学习中最强大的工具之一，使计算机能够解决图像识别等复杂任务。受大脑的启发，ANN由简单的神经元样单元组成，这些单元执行突触输入的加权和。人工神经元可以通过修改每个输入的权重来“学习”。然而，生物神经元要复杂得多。真实的神经元中的突触可以由于电压依赖性机制（例如NMDA受体）而以距离依赖性方式彼此非线性地相互作用。我们创建了一个模型神经元，称为Gradient clusteron（G-clusteron），它可以使用距离依赖的非线性来学习通过使其突触相互吸引或排斥来解决分类任务。我们测试了G-clusteron从MNIST数据集中识别手写数字的能力，并表明它可以通过更新其突触位置来实现高水平的分类准确率（约85%）。我们还推导出一个规则，用于更新的G-簇的突触权重，并表明，具有更新其突触权重和突触位置的能力，允许G-簇来解决“异或”（XOR）的问题，这是著名的不可能的线性神经元。
Synaptic clustering on neuronal dendrites has been hypothesized to play an important role in implementing pattern recognition. Neighboring synapses on a dendritic branch can interact in a synergistic, cooperative manner via nonlinear voltage-dependent mechanisms, such as NMDA receptors. Inspired by the NMDA receptor, the single-branch clusteron learning algorithm takes advantage of location-dependent multiplicative nonlinearities to solve classification tasks by randomly shuffling the locations of “under-performing” synapses on a model dendrite during learning (“structural plasticity”), eventually resulting in synapses with correlated activity being placed next to each other on the dendrite. We propose an alternative model, the gradient clusteron, or G-clusteron, which uses an analytically-derived gradient descent rule where synapses are "attracted to" or "repelled from" each other in an input- and location-dependent manner. We demonstrate the classification ability of this algorithm by testing it on the MNIST handwritten digit dataset and show that, when using a softmax activation function, the accuracy of the G-clusteron on the all-versus-all MNIST task (~85%) approaches that of logistic regression (~93%). In addition to the location update rule, we also derive a learning rule for the synaptic weights of the G-clusteron (“functional plasticity”) and show that a G-clusteron that utilizes the weight update rule can achieve ~89% accuracy on the MNIST task. We also show that a G-clusteron with both the weight and location update rules can learn to solve the XOR problem from arbitrary initial conditions. Artificial neural networks (ANNs) have become among the most powerful tools in artificial intelligence and machine learning, enabling computers to solve complex tasks like image recognition. Inspired by the brain, ANNs are composed of simple neuron-like units that perform a weighted sum of their synaptic inputs. Artificial neurons can “learn” by modifying the weight of each input. Biological neurons, however, are more complex. Synapses in a real neuron can interact nonlinearly with each other in a distance-dependent manner due to voltage-dependent mechanisms, such as the NMDA receptor. We created a model neuron, called the Gradient clusteron (G-clusteron) which can use distance-dependent nonlinearities to learn to solve classification tasks by making its synapses attract or repel each other. We tested the G-clusteron on its ability to recognize handwritten digits from the MNIST dataset and showed that it can achieve a high level of classification accuracy (~85%) just by updating its synaptic locations. We also derive a rule for updating the synaptic weights of the G-clusteron and show that having the ability to update both its synaptic weights and synaptic locations allows the G-clusteron to solve the “exclusive or” (XOR) problem, which is famously impossible for a linear neuron.
登录
查看更多内容
影响因子：
7.7
作者：
Bartol TM;Bromer C;Kinney J;Chirillo MA;Bourne JN;Harris KM;Sejnowski TJ
通讯作者：
Sejnowski TJ
DOI：
10.1126/science.aao0862
发表时间：
2018-06-22
期刊：
Science (New York, N.Y.)
影响因子：
--
作者：
El-Boustani S;Ip JPK;Breton-Provencher V;Knott GW;Okuno H;Bito H;Sur M
通讯作者：
Sur M
DOI：
10.1073/pnas.1803274115
发表时间：
2018-07-17
影响因子：
11.1
作者：
Hiratani, Naoki;Fukai, Tomoki
通讯作者：
Fukai, Tomoki
影响因子：
3.5
作者：
Hawkins J;Ahmad S
通讯作者：
Ahmad S
影响因子：
4.3
作者：
Behabadi BF;Polsky A;Jadi M;Schiller J;Mel BW
通讯作者：
Mel BW