SHF: Small: Methods and Architectures for Optimization and Hardware Acceleration of Spiking Neural Networks

人工智能是一项强大的跨领域技术，有望促进科学技术的广泛进步，并促进社会效益。为此，探索受大脑启发的新型计算原理可能为实现人工智能提供有希望的新途径。该项目定位于解决设计和工程大脑激发的尖峰神经模型的关键挑战。因此，它可能会导致方法、工具和硬件系统设计，最终将支持新一代基于软件和硬件的人工智能系统，并可能显著提高性能和效率。该项目将制作教材，整合到人工智能和硬件系统设计的本科和研究生课程中，从而在这些重要领域提供劳动力培训机会。首席研究员将积极招募本科生、未被充分代表的学生和女学生参与研究和培训，同时与各种外展项目合作。该奖项的成果可能以多种形式产生，包括算法、软件设计工具、硬件架构和实现，这些成果将通过出版物、研讨会、讲座和研究合作在广泛的研究和工业社区中传播。将寻求与美国高科技产业和其他研究组织的合作，以扩大这项工作的影响，促进潜在的技术转化为实践，并在不同的工业和研究环境下为学生提供额外的指导和培训。近年来，基于传统无尖峰人工神经网络的深度学习在许多应用领域取得了巨大成功。然而，传统的人工神经网络不能立即探索时间代码，并且缺乏高效的基于事件的处理。另一方面，人们认为要获得接近人类水平的智能，需要更好地模拟生物大脑的计算范式。因此，脉冲神经网络（snn）为促进未来的人工智能系统提供了一种互补的生物学上可行的方法。然而，在更广泛地采用脉冲神经网络的过程中，仍存在一些关键障碍。snn比传统的人工神经网络更难训练。对于设计计算能力强大的snn，特别是具有循环连接的snn，普遍缺乏洞察力和系统方法。snn的硬件加速受到跨时间和空间的复杂数据依赖以及非结构化发射稀疏性的阻碍。这项工作将从开发急需的精确SNN训练方法开始，这些方法可以鲁棒地学习精确的时间行为，并联合调整尖峰计数和尖峰时间。循环snn的可扩展架构设计和新颖的自动尖峰神经结构优化方法将被开发出来，以支持计算能力强大的snn的设计。为了实现高能效的高吞吐量硬件加速，将设计专用SNN硬件加速器架构，以最大限度地减少昂贵的数据移动，并促进空间和时间上的并行处理。应用无关的尖峰编码、尖峰压缩和探索非结构化发射稀疏性的架构将被研究用于SNN硬件加速。高性能SNN硬件加速器将在现场可编程门阵列设备上进行演示。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Artificial intelligence is a powerful cross-cutting technology and is expected to promote broad advancements in science and technology as well as foster social benefits. To this end, exploring novel computational principles inspired by the brain may offer promising new avenues to enable artificial intelligence. This project is positioned to address key challenges in designing and engineering brain-inspired spiking neural models. As such, it may lead to methods, tools, and hardware system designs that will ultimately support new generations of software- and hardware-based artificial intelligence systems with potentially significantly improved performance and efficiency. This project will produce educational materials to be integrated into undergraduate- and graduate-level curricula on artificial intelligence and hardware system design, thereby providing workforce training opportunities in these areas of importance. The principal investigator will actively recruit undergraduate, underrepresented, and female students for research participation and training while partnering with various outreach programs. The results of this award may be derived in a variety of forms, including algorithms, software design tools, and hardware architectures and implementations that will be disseminated in broad research and industrial communities through publications, workshops, talks, and research collaborations. Engagement with US high-tech industries and other research organizations will be sought to broaden the impact of this work, promote potential technology transfer into practice, and offer additional mentoring and training of students under diverse industrial and research settings.Deep learning based on conventional non-spiking artificial neural networks (ANNs) has achieved great success in many application domains in recent years. Nevertheless, the conventional ANNs cannot immediately explore temporal codes and lack energy-efficient event-based processing. On the other hand, it is believed that attaining near-human-level intelligence requires computing paradigms that better mimic biological brains. As such, spiking neural networks (SNNs) offer a complementary biologically-plausible approach to facilitating future artificial intelligence systems. However, there are key roadblocks to a wider adoption of spiking neural networks. SNNs are much harder to train than conventional ANNs. There is a general lack of insights and systemic approaches for designing computationally-powerful SNNs, particularly SNNs with recurrent connections. Hardware acceleration of SNNs is hampered by complex data dependencies across both time and space, and unstructured firing sparsity. This work will start out by developing much needed accurate SNN training methods that can robustly learn precise temporal behavior and jointly tune spike count and spike timing. Scalable architectural design of recurrent SNNs and novel automated spiking neural structural optimization methods will be developed to support the design of computationally powerful SNNs. To enable energy-efficient high-throughput hardware acceleration, dedicated SNN hardware accelerator architectures that minimize expensive data movements and facilitate parallel processing in both space and time will be designed. Application-independent spike coding, spike compression, and architectures exploring unstructured firing sparsity will be investigated for SNN hardware acceleration. High-performance SNN hardware accelerators will be demonstrated on field-programmable gate-array devices.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.