EAGER: III: Small: Green Granular Neural Networks with Fast FPGA-based Incremental Transfer Learning

点击翻译按钮获取中文摘要

Currently, popular machine learning systems running on the two main types of processing units -- central processing units (CPUs) and graphics processing units (GPUs). These units generate a lot of carbon dioxide (CO2) emissions and also waste energy because traditional training algorithms (e.g., gradient descent algorithms and genetic algorithms) take huge amount of time to optimize billions of hyperparameters, and because CPUs and GPUs are not energy efficient. Since the estimated CO2 emission amount is proportional to the total computational power, two major ways of effectively reducing CO2 emissions include developing novel high-speed non-traditional training algorithms to significantly reduce training times, and use novel computational hardware with low CO2 emissions. An urgent challenge is developing a novel machine learning system with high-speed non-traditional training algorithms running on green and energy efficient hardware to significantly reduce both CO2 emissions and energy consumption.A novel shallow software-hardware-based green granular neural network (GGNN) with new fast, field-programmable gate array (FPGA). FPGA-based incremental transfer learning algorithms will be developed to reduce both CO2 emissions and energy consumption more effectively than the CPU-GPU-based training algorithms. Since FPGA is a light-weight hardware with low CO2 emissions and low energy consumption, the FPGA is used to quickly solve a system of linear equations to directly calculate optimal values of hyperparameters of the shallow GGNN. The shallow GGNN with transferred granular knowledge can be trained incrementally and quickly to make efficient decisions for real-time green computing applications. A new GGNN tree will be developed to resolve the curse of dimensionality. In addition, the shallow GGNN is explainable because it can generate interpretable granular If-Then rules. This project is also useful for improving intelligent green computing education for relevant courses such as artificial intelligence, computational intelligence, data mining, and operating systems. Undergraduate students and graduate students including underrepresented students will do research works together. Students will be well trained to become future green machine learning workforce to develop novel software-hardware-based machine learning algorithms to greatly reduce both CO2 emissions and energy consumption.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.