NN-Thunder: HW/SW Codesign for Accelerating DNNs with Heterogeneous Beyond-von Neumann Architectures

点击翻译按钮获取中文摘要

Deep learning and deep neural networks (DNNs) have been adopted in many application areas and have a great potential to reshape the future of humankind. However, recent studies have demonstrated that classical von Neumann architectures are inherently inefficient when it comes to deep learning because such heavy workloads result in an extraordinary shift towards data-centric computing. Under such scenarios, a significant amount of energy is inevitably spent in moving a massive amount of data back and forth between processing elements and memory blocks. In this project, we envision a heterogeneous underlying HW, that combines various types of accelerators from both von Neumann and beyond-von Neumann architectures, offering a wide range of tradeoffs between computation accuracy, consumed energy, latency, and area footprint. Our project intends to address the limitation and improvement of beyond-von Neumann architectures w.r.t. performance and energy efficiency. The lack of fundamental exploration of suitable neural networks on beyond-von Neumann architectures hinders the possibility to exploit the full potential of such accelerators. In this regard, binarized neural networks (BNNs) offer the possibility of ultra-efficient hardware implementation and outstanding synergy with novel beyond-von Neumann architectures that are built from emerging beyond-CMOS devices. This project proposal plans to explore different means w.r.t. modeling, design, and optimization, to fully unleash the power of beyond-von Neumann neural network accelerators with the following goals: (1) cross-layer modeling for emerging technologies and abstractions of LiM (Logic-in-Memory) and PiM (Processing-in-Memory) for BNNs, (2) hardware-aware NN optimization for beyond-von Neumann architecture, and (3) HW/SW Codesign and Optimization. All in all, realizing a heterogeneous HW architecture is a key for future deep learning. It enables an ultra-efficient execution of hybrid-precision neural networks through HW/SW codesign and effectively allows the possibility to explore different novel tradeoffs.