SHF: Medium: Time Based Deep Neural Networks: An Integrated Hardware-Software Approach

深度学习硬件和算法的最新进展为计算机提供了前所未有的人类智能水平，用于自动驾驶汽车、患者诊断和治疗、语音处理、战略游戏和教育等应用。传统的深度学习算法依赖于连接到云端的强大计算机，这会产生大量的通信开销，需要大量的计算资源，并危及隐私和安全。专家们达成了一个强烈的共识，即深度学习的下一个前沿将是在移动的平台上运行的高效神经网络处理器。该项目旨在开发一种紧凑、低功耗的替代传统深度学习计算硬件的方案，专门针对边缘设备。所提出的方法是基于一种新的计算概念，称为基于时间的电路，它可以提供一个类似的推理性能水平，只有一小部分的功耗相比，传统的方法。在整个项目中，研究人员将考虑将新的神经网络计算方法转移到工业中。新的基于时间的深度学习计算方法将被纳入明尼苏达大学电气工程和计算机科学系的研究生和本科生课程以及K-12推广活动中。该项目将专注于在资源受限的移动的平台上实现深度学习应用的硬件和软件技术。在硬件方面，该团队将展示一个原型低功耗深度神经网络处理器，其中卷积、池化和激活函数等内部操作完全在时域中执行。在软件方面，该团队将开发修剪、近似和混合方法，这些方法可以有效降低深度神经网络的复杂性，同时对整体推理准确性的影响最小。该项目的一个独特之处在于硬件和软件团队之间的持续互动，以提供首个针对边缘设备的完全基于时间的深度神经网络引擎。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Recent advancements in deep learning hardware and algorithms are providing computers with unprecedented levels of human-like intelligence for applications such as self-driving cars, patient diagnosis and treatment, speech processing, strategy games, and education. Traditional deep learning algorithms rely on powerful computers tethered to the cloud which incurs a large communication overhead, requires extensive computing resources, and compromises privacy and security. There is a strong consensus among experts that the next frontier in deep learning will be highly-efficient neural network processors running on mobile platforms. This project aims at developing a compact and low power alternative to conventional deep learning computing hardware, specifically targeted for edge devices. The proposed approach is based on a novel computing concept called time-based circuits, which can deliver a similar level of inference performance at only a fraction of the power consumption compared to traditional methods. Throughout the project, the investigators will consider transferring the new neural network computing methods to industry. The new time-based deep learning computation methods will be incorporated into the graduate and undergraduate curricula, as well as K-12 outreach activities, of the electrical engineering and computer science departments at the University of Minnesota.This project will focus on both hardware and software techniques for enabling deep learning applications on resource-constrained mobile platforms. On the hardware side, the team will demonstrate a prototype low-power deep neural network processor where internal operations such as convolution, pooling, and activation functions are performed entirely in the time domain. On the software side, the team will develop pruning, approximation, and hybrid approaches that can effectively reduce the complexity of deep neural networks with minimal impact on the overall inference accuracy. A unique aspect of this project is the continual interaction between the hardware and software groups to deliver the first fully time-based deep neural network engine targeted for edge devices.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.