CNS Core: Small: Harnessing Weight Repetition for Efficient Deep Neural Network Inference on General-Purpose Platforms

社会正在见证深度神经网络(DNN)在日常生活的方方面面的使用爆炸式增长，包括健康、金融、娱乐和交通。DNN通过执行DNN推理来使用，DNN推理使用输入(例如，图像)查询DNN以获得答案(例如，分类)。社会每天都依赖于推理，它在从云服务器到个人电脑的各种设备上运行。该项目的目标是开发新的方法，使推理在这些设备上高效(快速，低功耗)。技术方法是探索如何将一种称为权重重复的新现象应用于中央处理器(CPU)和图形处理器(GPU)等通用设备。其思想是，当DNN权重重复时，可以简化DNN推理操作。第一个项目推力将开发高效率的重量重复感知软件内核，可以在未经修改的硬件上运行。第二个推力将开发新的训练技术，以共同设计具有权重重复感知核的DNN。最后，第三个推力将探索在前两个推力中可以进行哪些点的硬件改进以进一步提高效率。通过证明重量重复在通用设备上的有效性，该项目将释放软件、算法和硬件方面的创新。该项目还将放大相关但正交的技术可能带来的改进，例如权重量化和权重稀疏。为了支持跨堆栈方法，该项目将培训一批新的学生和研究人员，他们可以跨高性能软件、硬件和DNN训练算法工作，建立共同设计的机器学习堆栈，并在未来将所学到的经验应用于其他需要跨层解决方案的高影响问题。该项目将在项目结束后在伊利诺伊大学托管至少3年的面向公众的网站上存储所有出版物、代码和数据集。这些信息将通过商业网站提供。到这些网站的链接将反映在http://cwfletcher.net/weightrepetition.This奖上，该奖项反映了国家科学基金会的法定使命，并已被认为值得支持，通过使用基金会的智力优势和更广泛的影响审查标准进行评估。

Society is witnessing an explosion in the use of Deep Neural Networks (DNNs) across all facets of daily life including health, finances, entertainment and transportation. DNNs are used by performing DNN inference, which queries the DNN with an input (for example, an image) to get an answer (for example, a classification). Society relies on inference every day, where it is run on devices ranging from cloud servers to personal computers. The goal of this project is to develop new ways to make inference efficient (fast, low power) on these devices.The technical approach is to explore how a new phenomenon, called weight repetition, can be applied to general-purpose devices such as Central Processing Units (CPUs) and Graphical Processing Units (GPUs). The idea is, when a DNN weight is repeated, DNN inference operations can be simplified. The first project thrust will develop high-efficiency weight repetition-aware software kernels that can run on un-modified hardware. The second thrust will develop novel training techniques to co-design the DNN with the weight repetition-aware kernels. Finally, the third thrust will explore what point hardware modifications can be made to further improve efficiency in the first two thrusts.By proving weight repetition's effectiveness on general-purpose devices, this project will unlock innovation in software, algorithms and hardware. The project will also amplify the improvement possible from related, but orthogonal, techniques such as weight quantization and weight sparsity. To support the cross-stack approach, the project will train a new class of students and researchers who can work across high-performance software, hardware and DNN training algorithms to build co-designed Machine Learning stacks and, in the future, apply the lessons learned to other high-impact problems that require cross-layer solutions.The project will store all publications, code and data-sets on public-facing websites, hosted at the University of Illinois for at least 3 years after the end of the project. This information will be made available via commercial websites. Links to these websites will be mirrored at http://cwfletcher.net/weightrepetition.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.