Illusion of large on-chip memory by networked computing chips for neural network inference
Illusion of large on-chip memory by networked computing chips for neural network inference
复制标题
DOI：
10.1038/s41928-020-00515-3
复制
发表时间：
2021-01
影响因子：
34.3
通讯作者：
R. Radway;Andrew Bartolo;Paul C. Jolly;Zainab F. Khan;B. Le;Pulkit Tandon;Tony F. Wu;Yunfeng Xin;E. Vianello;P. Vivet;E. Nowak;H. Wong;M. Aly;E. Beigné;Mary Wootters;S. Mitra
中科院分区：
文献类型：
--
作者：
R. Radway;Andrew Bartolo;Paul C. Jolly;Zainab F. Khan;B. Le;Pulkit Tandon;Tony F. Wu;Yunfeng Xin;E. Vianello;P. Vivet;E. Nowak;H. Wong;M. Aly;E. Beigné;Mary Wootters;S. Mitra
用于深度神经网络（DNN）推理的硬件通常会受到片上存储器不足的影响，因此需要访问单独的仅存储器芯片。这种片外存储器访问在能量和执行时间方面产生相当大的成本。在片上存储器中装配整个DNN是具有挑战性的，特别是由于该技术的物理尺寸。在这里，我们报告了一个DNN推理系统-称为错觉-它由联网的计算芯片组成，每个芯片都包含一定数量的本地片上存储器和快速唤醒和关闭机制。一个八芯片的Illusion系统硬件实现的能量和执行时间分别为没有片外存储器的理想单芯片的3.5%和2.5%。Illusion是灵活和可配置的，可为各种DNN类型和大小实现接近理想的能量和执行时间。我们的方法是专为片上非易失性存储器与弹性永久写入故障，但适用于几种存储器技术。详细的模拟还表明，我们的硬件结果可以扩展到64芯片的错觉系统。
Hardware for deep neural network (DNN) inference often suffers from insufficient on-chip memory, thus requiring accesses to separate memory-only chips. Such off-chip memory accesses incur considerable costs in terms of energy and execution time. Fitting entire DNNs in on-chip memory is challenging due, in particular, to the physical size of the technology. Here, we report a DNN inference system—termed Illusion—that consists of networked computing chips, each of which contains a certain minimal amount of local on-chip memory and mechanisms for quick wakeup and shutdown. An eight-chip Illusion system hardware achieves energy and execution times within 3.5% and 2.5%, respectively, of an ideal single chip with no off-chip memory. Illusion is flexible and configurable, achieving near-ideal energy and execution times for a wide variety of DNN types and sizes. Our approach is tailored for on-chip non-volatile memory with resilience to permanent write failures, but is applicable to several memory technologies. Detailed simulations also show that our hardware results could be scaled to 64-chip Illusion systems.