Neural sampling machine with stochastic synapse allows brain-like learning and inference.
Neural sampling machine with stochastic synapse allows brain-like learning and inference.
复制标题
DOI：
10.1038/s41467-022-30305-8
复制
发表时间：
2022-05-11
影响因子：
16.6
通讯作者：
中科院分区：
文献类型：
--
作者：
许多现实世界中关键任务的应用程序都需要从嘈杂的数据和实时决策中持续的在线学习，并具有定义的置信度。神经网络的脑启发的概率模型可以明确处理数据的不确定性，并可以随时进行自适应学习。但是，它们在紧凑，低功率硬件中的实现仍然是一个挑战。在这项工作中，我们引入了一种新型的硬件结构，可以通过利用突触连接中的随机性来实现一种新的称为神经采样机（NSM）的随机神经网络（NSM）。我们通过将铁电场效应晶体管（FEFET）基于两端的随机选择器元件配对，通过将铁电场效应晶体管（FEFET）配对通过配对硅杂化随机突触。我们表明，绝缘体和金属状态之间选择器的随机切换特征类似于NSM的乘法突触噪声。我们执行网络级别的模拟，以突出随机NSM提供的显着特征，例如执行自主体重归一化，以进行连续的在线学习和贝叶斯推论。我们表明，随机NSM不仅可以在标准MNIST数据集上使用98.25％的精度执行高度精确的图像分类，而且还可以估计预测的不确定性（以预测的熵来衡量），当MNIST数据集的数字旋转时。构建可以支持神经科学启发模型的概率硬件平台可以增强当前人工智能（AI）的学习和推理能力。
神经采样机可以利用噪声进行学习。在这里，Dutta等。呈现由铁电晶体管组成的混合随机突触，并结合了实现神经样品机所需的乘数突触噪声的随机选择器。
Many real-world mission-critical applications require continual online learning from noisy data and real-time decision making with a defined confidence level. Brain-inspired probabilistic models of neural network can explicitly handle the uncertainty in data and allow adaptive learning on the fly. However, their implementation in a compact, low-power hardware remains a challenge. In this work, we introduce a novel hardware fabric that can implement a new class of stochastic neural network called Neural Sampling Machine (NSM) by exploiting the stochasticity in the synaptic connections for approximate Bayesian inference. We experimentally demonstrate an in silico hybrid stochastic synapse by pairing a ferroelectric field-effect transistor (FeFET)-based analog weight cell with a two-terminal stochastic selector element. We show that the stochastic switching characteristic of the selector between the insulator and the metallic states resembles the multiplicative synaptic noise of the NSM. We perform network-level simulations to highlight the salient features offered by the stochastic NSM such as performing autonomous weight normalization for continual online learning and Bayesian inferencing. We show that the stochastic NSM can not only perform highly accurate image classification with 98.25% accuracy on standard MNIST dataset, but also estimate the uncertainty in prediction (measured in terms of the entropy of prediction) when the digits of the MNIST dataset are rotated. Building such a probabilistic hardware platform that can support neuroscience inspired models can enhance the learning and inference capability of the current artificial intelligence (AI). Neural sampling machines make use of noise to perform learning. Here, Dutta et al. present a hybrid stochastic synapse composed out of a ferroelectric transistor combined with a stochastic selector exhibiting multiplicative synaptic noise required for implementing a neural sample machine.
登录
查看更多内容
影响因子：
4.3
作者：
Dutta, Sourav;Schafer, Clemens;Datta, Suman
通讯作者：
Datta, Suman
影响因子：
3.1
作者：
Burr, Geoffrey W.;Shelby, Robert M.;Hwang, Hyunsang
通讯作者：
Hwang, Hyunsang
影响因子：
3.4
作者：
Jerry, Matthew;Dutta, Sourav;Datta, Suman
通讯作者：
Datta, Suman
影响因子：
4
作者：
Cha, Euijun;Park, Jaehyuk;Hwang, Hyunsang
通讯作者：
Hwang, Hyunsang
影响因子：
--
作者：
BERGLUND, CN;GUGGENHEIM, HJ
通讯作者：
GUGGENHEIM, HJ