E2CDA: Type I: Collaborative Research: Energy-Efficient Artificial Intelligence with Binary RRAM and Analog Epitaxial Synaptic Arrays

E2CDA：I 型：协作研究：采用二进制 RRAM 和模拟外延突触阵列的节能人工智能

• 批准号: 1740225
• 负责人: Jae-sun Seo
• 金额: $ 57.91万
• 依托单位: Arizona State University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-09-15 至 2021-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1740225&HistoricalAwards=false
• 关键词: E2CDA; Type; Collaborative; Research; Energy

In recent years, deep learning and artificial neural networks have been very successful in large-scale recognition and classification tasks, some even surpassing human-level accuracy. However, state-of-the-art deep learning algorithms tend to present very large network models, which poses significant challenges for hardware, especially for memory. Emerging resistive devices have been proposed as an alternative solution for weight storage and parallel neural computing, but severe limitations still exist for applying resistive random access memories (RRAMs) for practical large-scale neural computing. This proposal targets on addressing limitations in resistive device based neural computing through novel device engineering, new bitcell designs, new neuron circuits, energy-aware architecture, and a new circuit-level benchmark simulator. A successful completion of this research is likely to have consequences to our society, enabling wide adoption of dense and energy-efficient intelligent hardware to power-/area-constrained local mobile/wearable devices. Furthermore, a self-learning chip that learns in near real-time and consumes very low-power can be integrated in smart biomedical devices, personalizing healthcare. This project will have a strong effort on integrating the research outcomes with education and outreach through summer outreach programs for high school students, undergraduate/graduate student training, and organization of tutorials and workshops at conferences for knowledge dissemination.The proposal will perform innovative and interdisciplinary research to address many limitations in today?s resistive device based neural computing and make a leap progress towards energy-efficient intelligent computing. Severe limitations of applying resistive random access memories (RRAMs) for practical large-scale neural computing include: (1) device-level non-idealities, e.g., non-linearity, variability, selector, and endurance, (2) inefficiency in representing negative weights and neurons, and (3) limited demonstration on simpler networks, instead of cutting-edge convolutional and recurrent neural networks. To address these limitations, novel technologies from devices to architectures will be investigated. First, new bitcell circuits will be designed for today's binary resistive devices, efficiently mapping XNOR functionality with (+1, -1) weights and neurons. Second, a novel epitaxial resistive device (EpiRAM) that exhibits many idealistic properties will be investigated, including linear programming for analog weights, suppressed variability, self-selectivity, and high endurance. Third, new neuron circuits will be explored for integration with new resistive devices for feedforward/feedback deep neural networks. Finally, new data-mapping techniques that efficiently map state-of-the-art deep neural networks onto the hardware framework with RRAM arrays will be developed, and the overall energy-efficiency will be verified with a new benchmark simulator ?NeuroSim?. With vertical innovations across material, device, circuit and architecture, tremendous potential and research needs will be pursued towards energy-efficient artificial intelligence in ubiquitous resource-constrained hardware systems.

近年来，深度学习和人工神经网络在大规模识别和分类任务中非常成功，有些甚至超过了人类水平的准确性。然而，最先进的深度学习算法往往呈现非常大的网络模型，这对硬件，特别是内存构成了重大挑战。新兴的电阻器件已经被提出作为权重存储和并行神经计算的替代解决方案，但是将电阻随机存取存储器（RRAM）应用于实际的大规模神经计算仍然存在严重的限制。该提案旨在通过新颖的器件工程、新的位单元设计、新的神经元电路、能量感知架构和新的电路级基准模拟器来解决基于电阻器件的神经计算的局限性。这项研究的成功完成可能会对我们的社会产生影响，使密集和节能的智能硬件能够广泛应用于功率/面积受限的本地移动的/可穿戴设备。此外，一种近实时学习且功耗极低的自学习芯片可以集成到智能生物医学设备中，从而实现个性化医疗保健。该项目将通过面向高中生的暑期拓展计划、本科生/研究生培训、在知识传播会议上组织辅导和研讨会，努力将研究成果与教育和推广相结合。该提案将进行创新和跨学科的研究，以解决当今的许多局限性。基于电阻器件的神经计算，向节能智能计算迈进了一大步。将电阻式随机存取存储器（RRAM）应用于实际大规模神经计算的严重限制包括：（1）设备级非理想性，例如，非线性、可变性、选择器和耐久性，（2）表示负权重和神经元的效率低下，以及（3）在更简单的网络上的有限演示，而不是尖端的卷积和递归神经网络。为了解决这些限制，将研究从设备到架构的新技术。首先，将为当今的二进制电阻器件设计新的位单元电路，有效地将XNOR功能与（+1，-1）权重和神经元映射。第二，一种新型的外延电阻器件（EpiRAM），表现出许多理想的属性将进行调查，包括线性编程模拟权重，抑制变异性，自我选择性，和高耐久性。第三，将探索新的神经元电路，用于与前馈/反馈深度神经网络的新电阻器件集成。最后，将开发新的数据映射技术，有效地将最先进的深度神经网络映射到具有RRAM阵列的硬件框架上，并使用新的基准模拟器验证整体能效。NeuroSim？随着材料、器件、电路和架构的垂直创新，将在无处不在的资源受限硬件系统中追求节能人工智能的巨大潜力和研究需求。

项目成果

Investigation of Read Disturb and Bipolar Read Scheme on Multilevel RRAM-Based Deep Learning Inference Engine

• DOI: 10.1109/ted.2020.2985013
• 发表时间: 2020-06-01
• 期刊: IEEE TRANSACTIONS ON ELECTRON DEVICES
• 影响因子: 3.1
• 作者: Shim, Wonbo;Luo, Yandong;Yu, Shimeng
• 通讯作者: Yu, Shimeng

Monolithically Integrated RRAM- and CMOS-Based In-Memory Computing Optimizations for Efficient Deep Learning

• DOI: 10.1109/mm.2019.2943047
• 发表时间: 2019-11-01
• 期刊: IEEE MICRO
• 影响因子: 3.6
• 作者: Yin, Shihui;Kim, Yulhwa;Seo, Jae-sun
• 通讯作者: Seo, Jae-sun

Benchmark of RRAM based Architectures for Dot-Product Computation

• DOI: 10.1109/apccas.2018.8605606
• 发表时间: 2018-10
• 期刊: 2018 IEEE Asia Pacific Conference on Circuits and Systems (APCCAS)
• 作者: Xiaochen Peng;Shimeng Yu

High-Throughput In-Memory Computing for Binary Deep Neural Networks With Monolithically Integrated RRAM and 90-nm CMOS

• DOI: 10.1109/ted.2020.3015178
• 发表时间: 2019-09
• 期刊: IEEE Transactions on Electron Devices
• 影响因子: 3.1
• 作者: Shihui Yin;Xiaoyu Sun;Shimeng Yu;Jae-sun Seo

Technological Benchmark of Analog Synaptic Devices for Neuroinspired Architectures

• DOI: 10.1109/mdat.2018.2890229
• 发表时间: 2019-05-01
• 期刊: IEEE DESIGN & TEST
• 影响因子: 2
• 作者: Chen, Pai-Yu;Yu, Shimeng
• 通讯作者: Yu, Shimeng