CNS Core: Small: Harnessing Weight Repetition for Efficient Deep Neural Network Inference on General-Purpose Platforms

CNS 核心：小型：利用权重重复在通用平台上进行高效深度神经网络推理

• 批准号: 1909999
• 负责人: Christopher Fletcher
• 金额: $ 50万
• 依托单位: University of Illinois at Urbana-Champaign
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-10-01 至 2023-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1909999&HistoricalAwards=false
• 关键词: CNS; Core; Small; Harnessing; Weight

Society is witnessing an explosion in the use of Deep Neural Networks (DNNs) across all facets of daily life including health, finances, entertainment and transportation. DNNs are used by performing DNN inference, which queries the DNN with an input (for example, an image) to get an answer (for example, a classification). Society relies on inference every day, where it is run on devices ranging from cloud servers to personal computers. The goal of this project is to develop new ways to make inference efficient (fast, low power) on these devices.The technical approach is to explore how a new phenomenon, called weight repetition, can be applied to general-purpose devices such as Central Processing Units (CPUs) and Graphical Processing Units (GPUs). The idea is, when a DNN weight is repeated, DNN inference operations can be simplified. The first project thrust will develop high-efficiency weight repetition-aware software kernels that can run on un-modified hardware. The second thrust will develop novel training techniques to co-design the DNN with the weight repetition-aware kernels. Finally, the third thrust will explore what point hardware modifications can be made to further improve efficiency in the first two thrusts.By proving weight repetition's effectiveness on general-purpose devices, this project will unlock innovation in software, algorithms and hardware. The project will also amplify the improvement possible from related, but orthogonal, techniques such as weight quantization and weight sparsity. To support the cross-stack approach, the project will train a new class of students and researchers who can work across high-performance software, hardware and DNN training algorithms to build co-designed Machine Learning stacks and, in the future, apply the lessons learned to other high-impact problems that require cross-layer solutions.The project will store all publications, code and data-sets on public-facing websites, hosted at the University of Illinois for at least 3 years after the end of the project. This information will be made available via commercial websites. Links to these websites will be mirrored at http://cwfletcher.net/weightrepetition.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

社会正在见证深度神经网络（DNN）在日常生活的各个方面的爆炸式使用，包括健康，财务，娱乐和交通。 DNN通过执行DNN推理来使用，DNN推理使用输入（例如，图像）查询DNN以获得答案（例如，分类）。 社会每天都依赖于推理，它在从云服务器到个人电脑的各种设备上运行。 该项目的目标是开发新的方法，使这些设备上的推理效率（快速，低功耗）。技术方法是探索如何将一种称为权重重复的新现象应用于通用设备，如中央处理器（CPU）和图形处理器（GPU）。 这个想法是，当DNN权重重复时，DNN推理操作可以简化。 第一个项目的重点是开发可以在未经修改的硬件上运行的高效权重重复感知软件内核。 第二个目标是开发新的训练技术，以协同设计DNN和权重重复感知内核。 最后，第三个重点将探索在前两个重点中，可以对硬件进行哪些改进以进一步提高效率。通过在通用设备上证明重量重复的有效性，该项目将开启软件、算法和硬件的创新。 该项目还将扩大相关但正交的技术（如权重量化和权重稀疏）的可能改进。 为了支持跨栈方法，该项目将培训一批新的学生和研究人员，他们可以跨高性能软件、硬件和DNN训练算法工作，以构建共同设计的机器学习栈，并在未来将学到的经验教训应用于其他需要跨层解决方案的高影响力问题。该项目将在面向公众的网站上存储所有出版物、代码和数据集，项目结束后，在伊利诺伊大学主持至少3年。 这些信息将通过商业网站提供。 这些网站的链接将反映在http://cwfletcher.net/weightrepetition.This奖项反映了NSF的法定使命，并被认为值得通过使用基金会的知识价值和更广泛的影响审查标准进行评估的支持。

项目成果

Speculative Taint Tracking (STT): A Comprehensive Protection for Speculatively Accessed Data

• DOI: 10.1145/3352460.3358274
• 发表时间: 2019-10
• 期刊: IEEE Micro
• 影响因子: 3.6
• 作者: Jiyong Yu;Mengjia Yan;Artem Khyzha;Adam Morrison;J. Torrellas;Christopher W. Fletcher