Collaborative Research: SHF: Medium: Neural-Network-based Stochastic Computing Architectures with applications to Machine Learning

合作研究：SHF：中：基于神经网络的随机计算架构及其在机器学习中的应用

基本信息

批准号：
1953980
负责人：
Ahmed Louri
金额：
$ 60万
依托单位：
George Washington University
依托单位国家：
美国
项目类别：
Continuing Grant
财政年份：
2020
资助国家：
美国
起止时间：
2020-06-15 至 2025-05-31
项目状态：
未结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1953980&HistoricalAwards=false
关键词：
Collaborative Research SHF Medium Neural

项目摘要

Modern computing hardware is constrained by stringent requirements such as extremely small size, low power consumption, and high reliability. Consequently, unconventional computing methods, such as Stochastic Computing (SC), that directly address these issues are of increasing interest, especially for Machine Learning (ML) applications in Artificial Intelligence (AI). SC is a novel computation framework in which input data is continuously provided as a streams of bits; therefore, complex computations can then be computed by simple bit-wise operations on the streams. The main attraction of SC is that it enables very low-cost and low-power architectural implementations, especially for arithmetic operations using simple logic elements. This feature is very relevant to Neural Networks (NNs), because NNs require significant hardware resources, therefore consuming substantial power when processing big datasets for ML. Moreover, current NN architectures are difficult to configure to suit different applications, because the hardware is rather complex and not very flexible. Thus, as ML systems are reaching the fundamental limits of computation using NNs, SC has emerged as a plausible and practical solution to meet performance, energy and resilience requirements for massive parallelism and fast deployment of hardware to support AI with direct impact on technology and national economic growth. The goal of this project is to develop NN architectures that rely on different computational features for cross-cutting schemes (spanning hardware units, algorithms, and applications) aimed at designing such efficient SC-based NNs.The technical work pursued under this project exploits the main features of SC and proposes a sound research program with several novel concepts. The first novelty of this investigation is that it makes possible the design of SC NNs by focusing on architectural-level hardware targeting also important metrics for SC (such as reducing latency and improving accuracy, mostly in inference and training). The second novelty of this work is that it addresses fundamental issues in which simple SC hardware is utilized adaptively to data to sustain a high level of parallel computation in NNs; solutions revolve around a configurable bottom-up scheme in which initially low-level hardware (such as neurons and processing function units) are modularly employed in the NNs to support computation at higher levels. Novel memory organizations to remedy errors when SC is employed are also proposed; this also enhances application-dependent requirements. The third novelty is the provision of having both SC as well as conventional (binary) computation on one combined hardware implementation; this is an added benefit for optimizing computing performance just in case the SC does not meet the accuracy requirements of the application at hand. Therefore, this timely research is directed to the continued technical innovation for emerging computing systems and architectures with relevance to both the computing and ML communities and strong implications on advancements in society and the US computing industry-at-large; moreover, this project is strongly committed to Broadening Participation in Computing (BPC) and its success.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

现代计算硬件受到严格要求的约束，例如极小的尺寸，低功耗和高可靠性。因此，直接解决这些问题的非常规计算方法，例如随机计算（SC），尤其是在人工智能（AI）中的机器学习（ML）应用中。 SC是一个新颖的计算框架，其中连续提供输入数据作为位流；因此，可以通过流在流上的简单的位操作来计算复杂的计算。 SC的主要吸引力是它可以实现非常低成本和低功率的体系结构实现，尤其是对于使用简单的逻辑元素的算术操作。此功能与神经网络（NNS）非常相关，因为NNS需要大量的硬件资源，因此在处理ML的大数据集时会消耗大量功能。此外，当前的NN体系结构很难配置以适合不同的应用程序，因为硬件相当复杂，而且不是很灵活。因此，随着ML系统使用NNS达到了计算的基本限制，SC已成为一种合理且实用的解决方案，以满足大规模并行性和快速部署硬件的性能，能源和弹性要求，以支持AI直接影响AI，直接影响技术和国家经济增长。该项目的目的是开发NN体系结构，这些体系结构依靠不同的计算功能来进行跨剪切方案（跨越硬件单元，算法和应用程序），旨在设计基于SC的高效NN。该项目下的技术工作利用SC的主要功能并提出了多个新颖概念的SC研究计划。这项调查的第一个新颖性是，它可以通过专注于体系结构级硬件目标来设计SC NN的设计，这也可以用于SC的重要指标（例如降低延迟和提高准确性，主要是推理和培训）。这项工作的第二个新颖性是，它解决了基本问题，其中简单的SC硬件可适应数据以维持NNS中的高水平平行计算；解决方案围绕着可配置的自下而上方案，其中最初低级硬件（例如神经元和处理功能单元）在NNS中被模块化用于支持较高级别的计算。还提出了新的记忆组织来解决SC时的纠正错误；这也增强了依赖应用程序的要求。第三个新颖性是提供一个合并硬件实现的SC和常规（二进制）计算的规定；这是优化计算性能的额外好处，以防SC不符合当前应用程序的准确性要求。因此，这项及时的研究针对了与计算和ML社区相关的新兴计算系统和架构的持续技术创新，以及对社会和美国计算行业的进步的强烈影响；此外，该项目强烈致力于扩大计算的参与（BPC）及其成功。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子优点和更广泛的影响评估标准通过评估来获得支持的。

项目成果

期刊论文数量（11）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

Learning-Based Quality Management for Approximate Communication in Network-on-Chips

DOI：
10.1109/tcad.2020.3012235
发表时间：
2020-11
期刊：
IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
影响因子：
2.9
作者：
Yuechen Chen;A. Louri
通讯作者：
Yuechen Chen;A. Louri

A Technique for Approximate Communication in Network-on-Chips for Image Classification