CAREER: Computational Foundations of Modern Machine Learning

机器学习将在社会的各个领域发挥核心作用，包括医疗、交通、教育和商业。尽管有这种巨大的潜力，但在对这些现代应用至关重要的一些最基本的方面的理论理解方面仍存在重大差距。这些应用提出了复杂的要求，例如对学习算法的内存或空间限制，以及学习模型对数据分布变化的稳健性。然而，经典的学习理论主要关注更传统的检测指标，如学习算法的运行时间和模型在测试集上获得的平均误差。因此，这个项目的目标是重新审视学习理论的一些基础，并发展一种考虑到现代要求的理论，如记忆效率和健壮性。通过这样做，该项目不仅将帮助构建丰富的算法套件来满足实践中的这些要求，从而显著增加当前应用的范围；它还将发展洞察力，以指导对学习和计算的新理论角度的理解。更详细地说，该项目的目标是了解在这些当代要求下可以实现的基本限制和权衡，并利用这种理解来开发新的算法框架来满足这些要求。为了实现这一点，该项目将检查一些基本学习和优化问题的可用内存和最佳收敛速度之间是否存在内在的权衡。该项目旨在利用这些权衡来确定合适的问题类别，在这些问题类别中，可以达到内存密集型算法的收敛速度，但使用的内存要少得多。最后，该项目旨在建立一个超越经典培训/测试范式的原则性框架，以了解学习模型的泛化能力，并开发一个有效满足现代稳健性需求的工具箱。为了帮助将理论结果转化为实际情况，将开发开放源码软件，并将在基准数据集上评估算法。教育计划与项目的研究目标紧密结合，包括与高中生的外展活动以及与洛杉矶县教育办公室的合作。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Machine learning is poised to play a central role in various areas of society, including healthcare, transportation, education, and commerce. Despite this immense potential, there are significant gaps in the theoretical understanding of some of the most foundational aspects which are crucial for these modern applications. These applications pose complex requirements, such as memory or space constraints on the learning algorithm and robustness of the learned model to changes in the data distribution. Classical learning theory, however, mainly focuses on more traditionally examined metrics, such as the running time of the learning algorithm and the average error that the model obtains on a test set. Therefore, the goal of this project is to re-examine some of the foundations of learning theory and develop a theory that takes into account modern requirements such as memory efficiency and robustness. In doing this, the project will not only help build a rich algorithmic suite to meet these requirements in practice and hence significantly increase the scope of current applications; it will also develop insights that can guide the understanding of new theoretical angles on learning and computation.In more detail, the project’s objective is to understand the fundamental limits and trade-offs of what is achievable under these contemporary requirements and use this understanding to develop new algorithmic frameworks to meet the requirements. To achieve this, the project will examine if there are inherent trade-offs between the available memory and the best achievable convergence rate for a number of fundamental learning and optimization problems. The project aims to leverage these trade-offs to identify suitable classes of problems where it is possible to achieve the convergence rate of memory-intensive algorithms but with much less memory usage. Finally, the project aims to establish a principled framework that goes beyond the classical training/test paradigm to understand the generalization abilities of learned models, and to develop a toolbox that effectively addresses modern robustness demands. To aid in the translation of theoretical results to practical settings, open-source software will be developed, and algorithms will be evaluated on benchmark datasets. An educational plan is tightly integrated with the research objectives of the project, including outreach activities with high-school students and a collaboration with the Los Angeles County Office of Education.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.