CRII: CNS: A Systematic Multi-Task Learning Framework for Improving Deep Learning Efficiency on Edge Platforms

点击翻译按钮获取中文摘要

Multi-task learning is a subfield of machine learning in which the data is trained with a shared model to solve different tasks simultaneously. Multi-task learning highly reduces the number of parameters in the machine learning models and thus reduces the computational and storage requirements. For example, there are multiple tasks to be done in real-time in self-driving cars, including object detection and depth estimation. If these tasks can be trained on a single model with shared parameters, the model size and the inference time can be highly reduced. This project aims to further compress the model used for multi-task learning as the model size of a single deep neural network is still a critical challenge to many computation systems, especially for edge platforms. This project proposes an approach to learn the difficulty of every task and maintain the performance of the most difficult task when compressing a multi-task learning model. It increases the potential in the compression rate with acceptable performance for all the tasks as the performance of the most difficult task needs to be guaranteed to provide a satisfactory user experience. This project also designs an efficient multi-task federated learning approach for edge platforms. It improves the convergence rate of multi-task federated learning and reduces the communication costs in every iteration. Finally, this project proposes to solve an algorithm-hardware co-design problem to maximize the implementation efficiency of the compressed multi-task DNN models on edge platforms.The files of compressed DNN models and the ideas on efficient DNN training and implementation may be useful to researchers who focus on improving the computation efficiency of DNN models on edge platforms and other hardware platforms.This project will involve undergraduate and graduate students in the research. The research achievements of this project will be incorporated into a current senior-level undergraduate course, a new planned advanced-level graduate course, and seminars for both undergraduate and graduate students. There are also planned research demonstrations during the workshops and summer camps for the K-12 students.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.