Collaborative Research: SHF: Medium: Neural-Network-based Stochastic Computing Architectures with applications to Machine Learning

点击翻译按钮获取中文摘要

Modern computing hardware is constrained by stringent requirements such as extremely small size, low power consumption, and high reliability. Consequently, unconventional computing methods, such as Stochastic Computing (SC), that directly address these issues are of increasing interest, especially for Machine Learning (ML) applications in Artificial Intelligence (AI). SC is a novel computation framework in which input data is continuously provided as a streams of bits; therefore, complex computations can then be computed by simple bit-wise operations on the streams. The main attraction of SC is that it enables very low-cost and low-power architectural implementations, especially for arithmetic operations using simple logic elements. This feature is very relevant to Neural Networks (NNs), because NNs require significant hardware resources, therefore consuming substantial power when processing big datasets for ML. Moreover, current NN architectures are difficult to configure to suit different applications, because the hardware is rather complex and not very flexible. Thus, as ML systems are reaching the fundamental limits of computation using NNs, SC has emerged as a plausible and practical solution to meet performance, energy and resilience requirements for massive parallelism and fast deployment of hardware to support AI with direct impact on technology and national economic growth. The goal of this project is to develop NN architectures that rely on different computational features for cross-cutting schemes (spanning hardware units, algorithms, and applications) aimed at designing such efficient SC-based NNs.The technical work pursued under this project exploits the main features of SC and proposes a sound research program with several novel concepts. The first novelty of this investigation is that it makes possible the design of SC NNs by focusing on architectural-level hardware targeting also important metrics for SC (such as reducing latency and improving accuracy, mostly in inference and training). The second novelty of this work is that it addresses fundamental issues in which simple SC hardware is utilized adaptively to data to sustain a high level of parallel computation in NNs; solutions revolve around a configurable bottom-up scheme in which initially low-level hardware (such as neurons and processing function units) are modularly employed in the NNs to support computation at higher levels. Novel memory organizations to remedy errors when SC is employed are also proposed; this also enhances application-dependent requirements. The third novelty is the provision of having both SC as well as conventional (binary) computation on one combined hardware implementation; this is an added benefit for optimizing computing performance just in case the SC does not meet the accuracy requirements of the application at hand. Therefore, this timely research is directed to the continued technical innovation for emerging computing systems and architectures with relevance to both the computing and ML communities and strong implications on advancements in society and the US computing industry-at-large; moreover, this project is strongly committed to Broadening Participation in Computing (BPC) and its success.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.