SPX: Collaborative Research: Scalable Neural Network Paradigms to Address Variability in Emerging Device based Platforms for Large Scale Neuromorphic Computing

SPX：协作研究：可扩展神经网络范式，以解决基于新兴设备的大规模神经形态计算平台的可变性

• 批准号: 1919167
• 负责人: Yiyu Shi
• 金额: $ 34.41万
• 依托单位: University of Notre Dame
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1919167&HistoricalAwards=false
• 关键词: SPX; Collaborative; Research; Scalable; Neural

Future computer data centers are being flooded with workloads requiring high-levels of computation using power-hungry deep neural network (DNN) models. DNN accelerators based on processing in memory built with new storage devices can offer great energy efficiency and performance for data centers. One challenge faced by these accelerators is their poor stability. This is due to the physical limitations of the new storage devices. This project aims to address this issue by developing efficient approaches to neural networks. One impact of proposed research is to develop more powerful, scalable, and sustainable deep learning computing systems. This will result in new consumer, business, scientific and national security applications. It will affect the fields of big data and cloud computing. This project will lead to new results in Computer Engineering and in fields that are hungry for deep learning capabilities. It will expose students to cutting-edge knowledge and hands-on research opportunities and elevate their competence. It will increase their confidence in facing today's highly competitive global job market. The education impact includes course integration of research results and outreach activities. Special attention is given in this to including women and underrepresented minority groups.The goal of the proposed research is to address a key issue in existing processing-in-memory-based neural network accelerators built with emerging nonvolatile devices, which is the bad stability due to weight uncertainties induced by the device characteristics. To escalate the stability of these promising emerging accelerators in a scalable and sustainable manner for future data centers, the project will include four tasks: 1) the explicitly modeling of weight uncertainties, which may exhibit spatial correlations extracted from device non-idealities, as parameterized canonical distributions. 2) a statistical neural network paradigm, which can be easily integrated into existing convolutional neural network architectures by replacing their deterministic operations with the statistical counterparts operating on parameterized canonical distributions. 3) variability-aware neural network classifier inspired by error correction output codes and modern neural network architecture. 4) variability-aware input pre-processing without touching neural networks. These paradigms will be generic to different software and hardware platforms, and will be implemented and evaluated with a wide set of real-world applications including image classification, biomedical image segmentation, and drone target tracking.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

未来的计算机数据中心将充斥着需要使用耗电的深度神经网络（DNN）模型进行高级别计算的工作负载。DNN加速器基于新存储设备构建的内存处理，可以为数据中心提供更高的能效和性能。这些加速器面临的一个挑战是它们的稳定性差。这是由于新存储设备的物理限制。该项目旨在通过开发有效的神经网络方法来解决这个问题。拟议研究的一个影响是开发更强大，可扩展和可持续的深度学习计算系统。这将导致新的消费者，商业，科学和国家安全应用。它将影响大数据和云计算领域。该项目将为计算机工程和渴望深度学习能力的领域带来新的成果。它将使学生接触到前沿知识和实践研究机会，并提高他们的能力。这将增加他们面对当今竞争激烈的全球就业市场的信心。教育影响包括课程、研究成果和外联活动的整合。特别注意的是，在这方面，包括妇女和代表性不足的少数groups.The拟议研究的目标是解决一个关键问题，在现有的处理在内存中的基于神经网络加速器建立与新兴的非易失性设备，这是由于重量的不确定性引起的设备特性的稳定性差。为了以可扩展和可持续的方式为未来的数据中心提升这些有前途的新兴加速器的稳定性，该项目将包括四项任务：1）明确建模权重不确定性，这些不确定性可能表现出从设备非理想性中提取的空间相关性，作为参数化的正则分布。2)一种统计神经网络范例，可以通过将其确定性操作替换为对参数化规范分布进行操作的统计对应物来轻松集成到现有的卷积神经网络架构中。3)受纠错输出代码和现代神经网络架构启发的可变性感知神经网络分类器。4)不接触神经网络的可变性感知输入预处理。这些范例将通用于不同的软件和硬件平台，并将通过广泛的实际应用进行实施和评估，包括图像分类，生物医学图像分割和无人机目标跟踪。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Reliable and Robust RRAM-based Neuromorphic Computing

• DOI: 10.1145/3386263.3407579
• 发表时间: 2020-09
• 期刊: Proceedings of the 2020 on Great Lakes Symposium on VLSI
• 作者: Grace Li Zhang;Bing Li;Ying Zhu;Shuhang Zhang;Tianchen Wang;Yiyu Shi;Tsung-Yi Ho;Hai Li;Ulf Schlichtmann

Device-Circuit-Architecture Co-Exploration for Computing-in-Memory Neural Accelerators

• DOI: 10.1109/tc.2020.2991575
• 发表时间: 2019-10
• 期刊: IEEE Transactions on Computers
• 影响因子: 3.7
• 作者: Weiwen Jiang;Qiuwen Lou;Zheyu Yan;Lei Yang;J. Hu;X. Hu;Yiyu Shi

Uncertainty Modeling of Emerging Device based Computing-in-Memory Neural Accelerators with Application to Neural Architecture Search

• DOI: 10.1145/3394885.3431635
• 发表时间: 2021-01
• 期刊: 2021 26th Asia and South Pacific Design Automation Conference (ASP-DAC)
• 作者: Zheyu Yan;Da-Cheng Juan;X. Hu;Yiyu Shi

RADARS: Memory Efficient Reinforcement Learning Aided Differentiable Neural Architecture Search

RADARS：记忆高效强化学习辅助可微神经架构搜索

• DOI: 10.1109/asp-dac52403.2022.9712499
• 发表时间: 2022
• 期刊: IEEE/ACM Asia and South Pacific Design Automation Conference
• 作者: Yan, Zheyu;Jiang, Weiwen;Hu, Xiaobo Sharon;Shi, Yiyu
• 通讯作者: Shi, Yiyu

Statistical Training for Neuromorphic Computing using Memristor-based Crossbars Considering Process Variations and Noise

• DOI: 10.23919/date48585.2020.9116244
• 发表时间: 2020-03
• 期刊: 2020 Design, Automation & Test in Europe Conference & Exhibition (DATE)
• 作者: Ying Zhu;Grace Li Zhang;Tianchen Wang;Bing Li;Yiyu Shi;Tsung-Yi Ho;Ulf Schlichtmann