SPX: Collaborative Research: Scalable Neural Network Paradigms to Address Variability in Emerging Device based Platforms for Large Scale Neuromorphic Computing

点击翻译按钮获取中文摘要

Future computer data centers are being flooded with workloads requiring high-levels of computation using power-hungry deep neural network (DNN) models. DNN accelerators based on processing in memory built with new storage devices can offer great energy efficiency and performance for data centers. One challenge faced by these accelerators is their poor stability. This is due to the physical limitations of the new storage devices. This project aims to address this issue by developing efficient approaches to neural networks. One impact of proposed research is to develop more powerful, scalable, and sustainable deep learning computing systems. This will result in new consumer, business, scientific and national security applications. It will affect the fields of big data and cloud computing. This project will lead to new results in Computer Engineering and in fields that are hungry for deep learning capabilities. It will expose students to cutting-edge knowledge and hands-on research opportunities and elevate their competence. It will increase their confidence in facing today's highly competitive global job market. The education impact includes course integration of research results and outreach activities. Special attention is given in this to including women and underrepresented minority groups.The goal of the proposed research is to address a key issue in existing processing-in-memory-based neural network accelerators built with emerging nonvolatile devices, which is the bad stability due to weight uncertainties induced by the device characteristics. To escalate the stability of these promising emerging accelerators in a scalable and sustainable manner for future data centers, the project will include four tasks: 1) the explicitly modeling of weight uncertainties, which may exhibit spatial correlations extracted from device non-idealities, as parameterized canonical distributions. 2) a statistical neural network paradigm, which can be easily integrated into existing convolutional neural network architectures by replacing their deterministic operations with the statistical counterparts operating on parameterized canonical distributions. 3) variability-aware neural network classifier inspired by error correction output codes and modern neural network architecture. 4) variability-aware input pre-processing without touching neural networks. These paradigms will be generic to different software and hardware platforms, and will be implemented and evaluated with a wide set of real-world applications including image classification, biomedical image segmentation, and drone target tracking.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.