Collaborative Research: RI: Small: Theoretical Foundations: The Advantage of Deep Learning over Traditional Shallow Learning Methods

点击翻译按钮获取中文摘要

Machine learning has been a primary driving force behind many current intelligent decision-making systems. Recently it shows a paradigm shift with increasing reliance on deep learning approaches, which have achieved unprecedented performance in various applications such as image processing, speech recognition, language translation, and game playing. Besides the empirical success, provable guarantees and insights into the principles behind the success have also become sought-after goals. However, the lack of adequate understanding is still limiting our capacity to fully exploit the potential of deep learning. This project aims to lay the foundations for supporting the practical trends, by understanding the advantages of deep learning over traditional learning methods, which is crucial for revealing key factors behind the practical success. The project will provide frameworks for proving performance guarantees and advantages of deep learning over traditional learning methods and enable the development of new deep learning methods that are more efficient and accessible. This project will develop a thorough and systematic approach for understanding the superior empirical performance of deep learning over traditional learning methods and use the obtained insights to design new learning methods. It will develop new theoretical models of properties on the labeling function of the data and the structure of the input leading to the practical success, and also provides frameworks for proving performance guarantees and advantages over shallow learning. It will also design new learning methods that explicitly exploit those properties and thus can be more efficient and accessible. This direction is still largely unexplored, despite significant recent research activities. The planned theoretical and algorithmic solutions are possible through an interdisciplinary mix of tools from machine learning, statistics, and optimization. The proposed program is grounded in the investigators' prior work that includes both theoretical results and empirical validation. If successful, the proposed research can be transformational for modern intelligent systems by laying the foundations for further development. It will also help to solve new theoretical problems from practice that are not adequately addressed by current theory and will have lasting impacts on machine learning and optimization.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.