E2CDA: Type I: Collaborative Research: Energy-Efficient Artificial Intelligence with Binary RRAM and Analog Epitaxial Synaptic Arrays

近年来，深度学习和人工神经网络在大规模的识别和分类任务中非常成功，有些甚至超过了人类水平的准确率。然而，最先进的深度学习算法往往呈现非常大的网络模型，这对硬件，特别是对内存提出了巨大的挑战。新兴的阻性器件已被提出作为重量存储和并行神经计算的替代解决方案，但将阻性随机存取存储器(RRAM)应用于实际大规模神经计算仍存在严重限制。该提案旨在通过新的器件工程、新的Bitcell设计、新的神经元电路、能量感知架构和新的电路级基准模拟器来解决基于阻性器件的神经计算的局限性。这项研究的成功完成可能会对我们的社会产生影响，使高密度、高能效的智能硬件能够广泛应用于电力/区域受限的本地移动/可穿戴设备。此外，一种近乎实时学习且功耗非常低的自学习芯片可以集成到智能生物医学设备中，实现个性化医疗保健。该项目将通过高中生暑期推广计划、本科生/研究生培训以及在知识传播会议上组织教程和研讨会，大力将研究成果与教育和推广相结合。该计划将进行跨学科研究，以解决当今基于阻性设备的神经计算的许多局限性，并在节能智能计算方面取得飞跃进展。将阻性随机存取存储器(RRAM)应用于实际大规模神经计算的严重限制包括：(1)设备级的非理想性，例如，非线性、可变性、选择器和耐力，(2)表示负权重和神经元的效率低下，以及(3)在更简单的网络上的有限演示，而不是尖端的卷积和递归神经网络。为了解决这些限制，将研究从设备到架构的新技术。首先，新的Bitcell电路将为当今的二进制电阻设备设计，有效地将XNOR功能与(1，-1)权重和神经元映射。其次，将研究一种新型的外延电阻器件(EpiRAM)，它具有许多理想的特性，包括模拟重量的线性规划、抑制变异性、自选择性和高耐久性。第三，将探索与用于前馈/反馈深度神经网络的新的阻性器件集成的新的神经元电路。最后，将开发新的数据映射技术，将最先进的深度神经网络有效地映射到具有RRAM阵列的硬件框架上，并将使用新的基准模拟器“NeuroSim”验证整体能效。随着材料、设备、电路和架构的创新，研究需求将转向在无处不在的资源受限的硬件系统中进行节能处理。

In recent years, deep learning and artificial neural networks have been very successful in large-scale recognition and classification tasks, some even surpassing human-level accuracy. However, state-of-the-art deep learning algorithms tend to present very large network models, which poses significant challenges for hardware, especially for memory. Emerging resistive devices have been proposed as an alternative solution for weight storage and parallel neural computing, but severe limitations still exist for applying resistive random access memories (RRAMs) for practical large-scale neural computing. This proposal targets on addressing limitations in resistive device based neural computing through novel device engineering, new bitcell designs, new neuron circuits, energy-aware architecture, and a new circuit-level benchmark simulator. A successful completion of this research is likely to have consequences to our society, enabling wide adoption of dense and energy-efficient intelligent hardware to power-/area-constrained local mobile/wearable devices. Furthermore, a self-learning chip that learns in near real-time and consumes very low-power can be integrated in smart biomedical devices, personalizing healthcare. This project will have a strong effort on integrating the research outcomes with education and outreach through summer outreach programs for high school students, undergraduate/graduate student training, and organization of tutorials and workshops at conferences for knowledge dissemination.The proposal will perform interdisciplinary research to address many limitations in today's resistive device based neural computing and make a leap progress towards energy-efficient intelligent computing. Severe limitations of applying resistive random access memories (RRAMs) for practical large-scale neural computing include: (1) device-level non-idealities, e.g., non-linearity, variability, selector, and endurance, (2) inefficiency in representing negative weights and neurons, and (3) limited demonstration on simpler networks, instead of cutting-edge convolutional and recurrent neural networks. To address these limitations, novel technologies from devices to architectures will be investigated. First, new bitcell circuits will be designed for today's binary resistive devices, efficiently mapping XNOR functionality with (+1, -1) weights and neurons. Second, a novel epitaxial resistive device (EpiRAM) that exhibits many idealistic properties will be investigated, including linear programming for analog weights, suppressed variability, self-selectivity, and high endurance. Third, new neuron circuits will be explored for integration with new resistive devices for feedforward/feedback deep neural networks. Finally, new data-mapping techniques that efficiently map state-of-the-art deep neural networks onto the hardware framework with RRAM arrays will be developed, and the overall energy-efficiency will be verified with a new benchmark simulator "NeuroSim". With innovations across material, device, circuit and architecture, research needs will be pursued towards energy-efficient processing in ubiquitous resource-constrained hardware systems.