Energy-Efficient Deep Neural Networks with Mixed-Signal Neurons and Dense-Local and Sparse-Global Connectivity
Energy-Efficient Deep Neural Networks with Mixed-Signal Neurons and Dense-Local and Sparse-Global Connectivity
复制标题
具有混合信号神经元以及密集局部和稀疏全局连接的节能深度神经网络
DOI：
10.1145/3394885.3431614
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
Sen, Shreyas
中科院分区：
文献类型：
--
作者：
Chatterjee, Baibhab;Sen, Shreyas
神经形态计算已经变得非常受欢迎，因为它能够比传统的冯诺依曼计算机更好地解决某些类别的学习任务。数据密集型分类和模式识别问题一直是神经形态工程师特别感兴趣的问题，因为这些问题为深度神经网络（DNN）提供了复杂的用例，这些深度神经网络（DNN）的动机来自人类大脑的架构，并采用以分层方式组织的密集连接的神经元和突触。然而，随着这些系统变得越来越大，以处理越来越多的数据和更高维度的功能，设计往往成为连接约束。为了解决这个问题，计算被分成多个核心/岛，称为处理引擎（PE）。如今，这些PE之间的通信是通过耗电的片上网络（NoC）进行的，因此这些岛的最佳分布沿着节能计算和通信策略在降低神经形态计算机的总体能量方面变得极其重要，神经形态计算机目前比生物人脑高几个数量级。在本文中，我们广泛地分析了基于混合信号神经元/突触的岛的大小的选择，在系统级分类误差的允许范围内的3-8位分辨率，由神经元中的模拟非理想性（噪声和失配）确定，并提出了涉及本地和全局通信的策略，以减少系统级的能量消耗。AC耦合混合信号神经元的非理想性比DC耦合神经元低10倍，而岛的数量的选择被证明是网络的函数，受模拟到数字转换（反之亦然）的电源在岛的接口。在一个岛的最大层数进行了分析，并提出了一个全球总线为基础的稀疏连接，它消耗的数量级低于竞争的电力线通信技术的功率。
Neuromorphic Computing has become tremendously popular due to its ability to solve certain classes of learning tasks better than traditional von-Neumann computers. Data-intensive classification and pattern recognition problems have been of special interest to Neuromorphic Engineers, as these problems present complex use-cases for Deep Neural Networks (DNNs) which are motivated from the architecture of the human brain, and employ densely connected neurons and synapses organized in a hierarchical manner. However, as these systems become larger in order to handle an increasing amount of data and higher dimensionality of features, the designs often become connectivity constrained. To solve this, the computation is divided into multiple cores/islands, called processing engines (PEs). Today, the communication among these PEs are carried out through a power-hungry network-on-chip (NoC), and hence the optimal distribution of these islands along with energy-efficient compute and communication strategies become extremely important in reducing the overall energy of the neuromorphic computer, which is currently orders of magnitude higher than the biological human brain. In this paper, we extensively analyze the choice of the size of the islands based on mixed-signal neurons/synapses for 3-8 bit-resolution within allowable ranges for system-level classification error, determined by the analog non-idealities (noise and mismatch) in the neurons, and propose strategies involving local and global communication for reduction of the system-level energy consumption. AC-coupled mixed-signal neurons are shown to have 10X lower non-idealities than DC-coupled ones, while the choice of number of islands are shown to be a function of the network, constrained by the analog to digital conversion (or viceversa) power at the interface of the islands. The maximum number of layers in an island is analyzed and a global bus-based sparse connectivity is proposed, which consumes orders of magnitude lower power than the competing powerline communication techniques.