Training a Probabilistic Graphical Model With Resistive Switching Electronic Synapses
Training a Probabilistic Graphical Model With Resistive Switching Electronic Synapses
复制标题
DOI：
10.1109/ted.2016.2616483
复制
发表时间：
2016-12-01
影响因子：
3.1
通讯作者：
Wong, Hon-Sum Philip
中科院分区：
文献类型：
--
作者：
Eryilmaz, Sukru Burc;Neftci, Emre;Wong, Hon-Sum Philip
当前深度学习和数据挖掘的大规模实现需要数千个处理器，大量的片外内存，并消耗千兆焦耳的能量。新的存储技术，如纳米级双端电阻开关存储设备，提供了一种紧凑、可扩展和低功耗的替代方案，允许在片上以细粒度分布式并行架构进行并发处理和存储。在这里，我们报告了首次使用电阻存储器器件来实现和训练受限玻尔兹曼机（RBM），这是一种生成概率图形模型，是深度网络中无监督学习的关键组成部分。我们通过实验演示了一个45突触的RBM，该RBM由90个电阻性相变存储器（PCM）元件实现，该元件采用了一种生物变体的对比发散算法，实现了Hebbian和anti-Hebbian权重更新。与未训练的情况相比，电阻式PCM器件在训练超过30次的缺失像素模式补全任务中显示出两倍到十倍的错误率降低。PCM器件的编程能耗为6.1 nJ / epoch，比传统的处理器-存储器系统低150倍。我们分析和讨论了PCM模拟存储器件中学习性能对周期变化和渐进电平数量的依赖。
Current large-scale implementations of deep learning and data mining require thousands of processors, massive amounts of off-chip memory, and consume giga-joules of energy. New memory technologies, such as nanoscale two-terminal resistive switching memory devices, offer a compact, scalable, and low-power alternative that permits on-chip colocated processing and memory in fine-grain distributed parallel architecture. Here, we report the first use of resistive memory devices for implementing and training a restricted Boltzmann machine (RBM), a generative probabilistic graphical model as a key component for unsupervised learning in deep networks. We experimentally demonstrate a 45-synapse RBM realized with 90 resistive phase change memory (PCM) elements trained with a bioinspired variant of the contrastive divergence algorithm, implementing Hebbian and anti-Hebbian weight updates. The resistive PCM devices show a twofold to tenfold reduction in error rate in a missing pixel pattern completion task trained over 30 epochs, compared with untrained case. Measured programming energy consumption is 6.1 nJ per epoch with the PCM devices, a factor of similar to 150 times lower than the conventional processor-memory systems. We analyze and discuss the dependence of learning performance on cycle-to-cycle variations and number of gradual levels in the PCM analog memory devices.