SHF: Medium: A Technology-Architecture-Algorithm Co-Design Exploration of Scalable Spiking Neural Networks (SNNs)

SHF：Medium：可扩展尖峰神经网络 (SNN) 的技术-架构-算法协同设计探索

• 批准号: 1955815
• 负责人: Chitaranjan Das
• 金额: $ 100万
• 依托单位: Pennsylvania State Univ University Park
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-06-01 至 2025-05-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1955815&HistoricalAwards=false
• 关键词: SHF; Medium; Technology; Architecture; Algorithm

Neuromorphic computing is an inspiring and ambitious problem because by understanding how the human nervous system is able to work efficiently using less than 20-Watts of power, we should be able to design intelligent computing systems that can solve complex problems like humans. While there have been significant advancements in the accuracy of artificial neural networks (ANNs), the quest for power efficiencies akin to biological systems remains elusive. Consequently, there has been significant interest in an approach called spiking neural networks (SNNs) which uses biologically-inspired event-driven spike communication. However, SNN design is still in its infancy, with need for exploring different neuromorphic computing models, learning algorithms, hardware substrates and designing a communication fabric that can scale to meet a strict power budget. This research seeks to explore the design space of scalable, low-power SNNs. The outcomes of this research could be highly valuable and will pave the way for an array of related research in designing intelligent machines. On the educational front, plans include the involvement of undergraduate and graduate students in this emerging research, where they will get exposure to cross-cutting topics in computer architecture, electronic devices, circuits, and applications. Female and minority students will be recruited to work in the project. Several Broadening Participation in Computing (BPC) activities such as summer camp for girls and collaboration with the Education Department to expose K-12 students to many areas of computer science will be pursued.The project seeks to take a comprehensive approach spanning device/circuit level innovations for designing extremely low-power spiking neurons and synapses, architectural level solutions for designing hierarchical communication networks, algorithm level optimizations to improve the accuracy of SNN models, and evaluation of the designs with different classes of applications. The proposed research consists of four research tasks. First, the team plans to investigate the design of neuron device primitives using Magnetic Tunnel Junction (MTJ)-based spintronic devices that are orders of magnitude more power efficient than existing technologies. Second, using these MTJ-based neuron tiles, the design space of a scalable on-chip interconnection fabric will be explored. Third, the research focuses on developing algorithmic solutions to enhance the accuracy of SNNs, and a hybrid solution to leverage the benefits of both SNNs and ANNs. Finally, the plan includes to develop a simulation platform to evaluate the performance and energy efficiency of different designs using diverse applications.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

神经形态计算是一个鼓舞人心且雄心勃勃的问题，因为通过了解人类神经系统如何能够使用不到 20 瓦的功率高效工作，我们应该能够设计出能够像人类一样解决复杂问题的智能计算系统。尽管人工神经网络 (ANN) 的准确性取得了显着进步，但对类似于生物系统的功率效率的追求仍然难以实现。因此，人们对一种称为尖峰神经网络 (SNN) 的方法产生了浓厚的兴趣，该方法使用受生物学启发的事件驱动的尖峰通信。然而，SNN 设计仍处于起步阶段，需要探索不同的神经拟态计算模型、学习算法、硬件基础，并设计可扩展以满足严格功率预算的通信结构。本研究旨在探索可扩展、低功耗 SNN 的设计空间。这项研究的成果可能非常有价值，并将为设计智能机器的一系列相关研究铺平道路。在教育方面，计划包括让本科生和研究生参与这项新兴研究，他们将接触计算机体系结构、电子设备、电路和应用领域的交叉主题。 将招募女性和少数民族学生在该项目中工作。将开展多项扩大计算参与 (BPC) 活动，例如女孩夏令营以及与教育部合作，让 K-12 学生接触计算机科学的许多领域。该项目寻求采取一种全面的方法，涵盖设备/电路级创新来设计极低功耗尖峰神经元和突触，设计分层通信网络的架构级解决方案，算法级优化来提高 SNN 模型的准确性， 以及不同类别应用的设计评估。 拟议的研究包括四项研究任务。首先，该团队计划使用基于磁隧道结（MTJ）的自旋电子器件来研究神经元器件基元的设计，该器件的能效比现有技术高出几个数量级。 其次，使用这些基于 MTJ 的神经元块，将探索可扩展片上互连结构的设计空间。第三，研究重点是开发算法解决方案以提高 SNN 的准确性，以及混合解决方案以利用 SNN 和 ANN 的优势。最后，该计划包括开发一个模拟平台，以评估使用不同应用的不同设计的性能和能源效率。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Sparse Vector-Matrix Multiplication Acceleration in Diode-Selected Crossbars

二极管选择交叉开关中的稀疏向量矩阵乘法加速

• DOI: 10.1109/tvlsi.2021.3114186
• 发表时间: 2021
• 期刊: IEEE Transactions on Very Large Scale Integration (VLSI
• 作者: Jao, Nicholas;Ramanathan, Akshay Krishna;Sampson, John;Narayanan, Vijaykrishnan
• 通讯作者: Narayanan, Vijaykrishnan

Gesture-SNN: Co-optimizing accuracy, latency and energy of SNNs for neuromorphic vision sensors

Gesture-SNN：共同优化神经形态视觉传感器的 SNN 的准确性、延迟和能量

• DOI: 10.1109/islped52811.2021.9502506
• 发表时间: 2021
• 期刊: 2021 IEEE/ACM International Symposium on Low Power Electronics and Design (ISLPED
• 作者: Singh, Sonali;Sarma, Anup;Lu, Sen;Sengupta, Abhronil;Narayanan, Vijaykrishnan;Das, Chita R.
• 通讯作者: Das, Chita R.

Structured in Space, Randomized in Time: Leveraging Dropout in RNNs for Efficient Training

• 发表时间: 2021-06
• 作者: Anup Sarma;Sonali Singh;Huaipan Jiang;Rui Zhang;M. Kandemir;C. Das

Reconfigurable perovskite nickelate electronics for artificial intelligence

用于人工智能的可重构钙钛矿镍酸盐电子器件

• DOI: 10.1126/science.abj7943
• 发表时间: 2022
• 期刊: Science
• 影响因子: 56.9
• 作者: Zhang, Hai-Tian;Park, Tae Joon;Islam, A. N.;Tran, Dat S.;Manna, Sukriti;Wang, Qi;Mondal, Sandip;Yu, Haoming;Banik, Suvo;Cheng, Shaobo
• 通讯作者: Cheng, Shaobo

Exploring the Connection Between Binary and Spiking Neural Networks

• DOI: 10.3389/fnins.2020.00535
• 发表时间: 2020-06-24
• 期刊: FRONTIERS IN NEUROSCIENCE
• 影响因子: 4.3
• 作者: Lu, Sen;Sengupta, Abhronil
• 通讯作者: Sengupta, Abhronil