CNS Core: Small: CNN-Accelerator Co-Design

近年来，机器学习（ML）模型和加速器架构的自动化协同设计引起了业界和学术界的广泛关注。然而，大多数协同设计框架探索有限的搜索空间。此外，训练机器学习模型和模拟加速器性能在计算上是昂贵的。为了解决这些限制，本项目提出了一个卷积神经网络（CNN）和硬件加速器协同设计框架。它将包括两个新的基准测试子框架，CNNBench和AccelBench，这将极大地扩展CNN和CNN加速器的设计空间。CNNBench将使用一种新的搜索技术来收敛到最优的CNN架构。AccelBench将在广阔的设计空间中为各种加速器架构执行周期精确的模拟。在为给定应用程序设计加速器时，需要选择大量超参数的值。它们包括处理元素和片上缓冲区的数量和大小、数据流、主存大小和类型，以及许多特定领域模块（如稀疏感知计算和低精度设计）的数量和大小。类似地，在设计CNN时，许多超参数也会发挥作用：层数、卷积类型和大小、归一化类型、池化类型和大小、最终多层感知器头部的结构、激活函数、训练配方等等。同时，具有最高性能的CNN架构可能无法满足能源消耗和内存占用等资源限制。所提出的协同设计框架将在给定用户约束下有效地导航目标应用的联合cnn -加速器设计空间。该项目预计，在提供更高性能的同时，相对于目前的基线，该领域的导航效率至少会有一个数量级的提高。鉴于cnn -加速器协同设计问题对行业的重要性，拟议的协同设计框架可能会对其产生变革性影响。为了有效地转移研究成果，该项目计划继续我们积极的工业合作。本项目将招收研究生/本科生女/少数民族学生（PI迄今已指导15名女博士研究生，以及多名女/少数民族本科生和高中生）。该项目将确保参与者，包括通过外展参与的更广泛的公众，体验研究的跨学科性质。研究成果将包括在机器学习和嵌入式计算两门本科课程中。计划通过普林斯顿实验室学习计划向高中生推广。将通过发表论文、海报和研讨会向学术界和工业界广泛传播。此外，各种工具和模型将在网上分发。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Recently, automated co-design of machine learning (ML) models and accelerator architectures has attracted significant attention from both the industry and academia. However, most co-design frameworks explore a limited search space. Furthermore, training the ML model and simulating the accelerator performance are computationally expensive. To address these limitations, this project proposes a convolutional neural network (CNN) and hardware-accelerator co-design framework. It will consist of two new benchmarking sub-frameworks, CNNBench and AccelBench, that will explore vastly expanded design spaces of CNNs and CNN accelerators. CNNBench will use a new search technique to converge to the optimal CNN architecture. AccelBench will perform cycle-accurate simulations for a diverse set of accelerator architectures in a vast design space. Values for a large number of hyperparameters need to be chosen while designing an accelerator for a given application. They include the number and size of processing elements and on-chip buffers, dataflow, main memory size and type, and those for many domain-specific modules like sparsity-aware computation and reduced-precision designs. Similarly, many hyperparameters come into play while designing a CNN as well: the number of layers, convolution type and size, normalization type, pooling type and size, structure of the final multi-layer perceptron head, activation function, training recipe, and many more. At the same time, a CNN architecture that has the highest performance may not meet resource constraints like energy consumption and memory footprint. The proposed co-design framework will efficiently navigate the joint CNN-accelerator design space for the target application under given user constraints. THe project expects at least an order of magnitude efficiency improvement in navigating this space relative to current baselines while delivering higher performance. Given the importance of the CNN-accelerator co-design problem to the industry, the proposed co-design framework will likely have a transformative impact on it. To effectively transfer research results, the project plans to continue our active industrial engagements. The project will recruit graduate/undergraduate female/minority students (the PI has supervised 15 female Ph.D. students so far as well as many female/minority undergraduate and high-school students). The project will ensure that the participants, including the broader public engaged through outreach, experience the interdisciplinary nature of the research. The research outcomes will be included in two undergraduate courses on Machine Learning and Embedded Computing. Outreach to high-school students is planned through Princeton Laboratory Learning Program. Broad dissemination to the academic and industrial communities will be achieved through published papers, posters, and seminars. In addition, various tools and models will be distributed online.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.