SHF: Small: Methods and Architectures for Optimization and Hardware Acceleration of Spiking Neural Networks

SHF：小型：尖峰神经网络优化和硬件加速的方法和架构

• 批准号: 2310170
• 负责人: Peng Li
• 金额: $ 59.93万
• 依托单位: University of California-Santa Barbara
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-09-15 至 2026-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2310170&HistoricalAwards=false
• 关键词: SHF; Small; Methods; Architectures; Optimization

Artificial intelligence is a powerful cross-cutting technology and is expected to promote broad advancements in science and technology as well as foster social benefits. To this end, exploring novel computational principles inspired by the brain may offer promising new avenues to enable artificial intelligence. This project is positioned to address key challenges in designing and engineering brain-inspired spiking neural models. As such, it may lead to methods, tools, and hardware system designs that will ultimately support new generations of software- and hardware-based artificial intelligence systems with potentially significantly improved performance and efficiency. This project will produce educational materials to be integrated into undergraduate- and graduate-level curricula on artificial intelligence and hardware system design, thereby providing workforce training opportunities in these areas of importance. The principal investigator will actively recruit undergraduate, underrepresented, and female students for research participation and training while partnering with various outreach programs. The results of this award may be derived in a variety of forms, including algorithms, software design tools, and hardware architectures and implementations that will be disseminated in broad research and industrial communities through publications, workshops, talks, and research collaborations. Engagement with US high-tech industries and other research organizations will be sought to broaden the impact of this work, promote potential technology transfer into practice, and offer additional mentoring and training of students under diverse industrial and research settings.Deep learning based on conventional non-spiking artificial neural networks (ANNs) has achieved great success in many application domains in recent years. Nevertheless, the conventional ANNs cannot immediately explore temporal codes and lack energy-efficient event-based processing. On the other hand, it is believed that attaining near-human-level intelligence requires computing paradigms that better mimic biological brains. As such, spiking neural networks (SNNs) offer a complementary biologically-plausible approach to facilitating future artificial intelligence systems. However, there are key roadblocks to a wider adoption of spiking neural networks. SNNs are much harder to train than conventional ANNs. There is a general lack of insights and systemic approaches for designing computationally-powerful SNNs, particularly SNNs with recurrent connections. Hardware acceleration of SNNs is hampered by complex data dependencies across both time and space, and unstructured firing sparsity. This work will start out by developing much needed accurate SNN training methods that can robustly learn precise temporal behavior and jointly tune spike count and spike timing. Scalable architectural design of recurrent SNNs and novel automated spiking neural structural optimization methods will be developed to support the design of computationally powerful SNNs. To enable energy-efficient high-throughput hardware acceleration, dedicated SNN hardware accelerator architectures that minimize expensive data movements and facilitate parallel processing in both space and time will be designed. Application-independent spike coding, spike compression, and architectures exploring unstructured firing sparsity will be investigated for SNN hardware acceleration. High-performance SNN hardware accelerators will be demonstrated on field-programmable gate-array devices.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

人工智能是一项强大的交叉技术，预计将推动科学技术的广泛进步，并促进社会效益。为此，探索受大脑启发的新计算原理可能会为实现人工智能提供有前途的新途径。该项目旨在解决设计和设计大脑启发的棘波神经模型方面的关键挑战。因此，它可能导致最终支持新一代基于软件和硬件的人工智能系统的方法、工具和硬件系统设计，并可能显著提高性能和效率。该项目将编写教育材料，纳入关于人工智能和硬件系统设计的本科生和研究生课程，从而在这些重要领域提供劳动力培训机会。首席调查员将积极招募本科生、代表性不足的学生和女性学生参与研究和培训，同时与各种外展计划合作。该奖项的结果可能以多种形式获得，包括算法、软件设计工具、硬件架构和实现，这些将通过出版物、研讨会、演讲和研究合作在广泛的研究和工业社区中传播。将寻求与美国高科技行业和其他研究组织的接触，以扩大这项工作的影响，促进潜在的技术转化为实践，并在不同的产业和研究环境下为学生提供额外的指导和培训。近年来，基于传统的非尖峰人工神经网络(ANN)的深度学习在许多应用领域取得了巨大成功。然而，传统的人工神经网络不能立即探索时间代码，并且缺乏基于事件的节能处理。另一方面，人们认为，要达到接近人类水平的智能，需要更好地模拟生物大脑的计算范例。因此，尖峰神经网络(SNN)提供了一种互补的生物学上看似合理的方法来促进未来的人工智能系统。然而，要更广泛地采用尖峰神经网络，还有一些关键障碍。与传统的神经网络相比，SNN的训练难度要大得多。在设计计算能力强大的SNN，特别是具有循环连接的SNN方面，普遍缺乏洞察力和系统的方法。SNN的硬件加速受到时间和空间上复杂的数据依赖以及非结构化激发稀疏性的阻碍。这项工作将从开发急需的准确SNN训练方法开始，这些方法可以稳健地学习精确的时间行为，并联合调整尖峰计数和尖峰计时。递归神经网络的可扩展结构设计和新的自动尖峰神经结构优化方法将被开发来支持计算能力强大的神经网络的设计。为了实现高能效的高吞吐量硬件加速，将设计专用的SNN硬件加速器架构，以最大限度地减少昂贵的数据移动，并在空间和时间上促进并行处理。独立于应用程序的尖峰编码、尖峰压缩和探索非结构化激发稀疏性的体系结构将被研究用于SNN硬件加速。高性能SNN硬件加速器将在现场可编程门阵列设备上展示。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。