CAREER: Foundations of Resource Efficient Machine Learning

当代机器学习技术往往是资源密集型的，通常需要高质量的数据集、昂贵的硬件或强大的计算能力。在从医疗保健到移动的计算的广泛应用领域中，这些关键资源都缺乏。能够最佳利用资源的新方法可以帮助释放这些领域数据科学革命的全部潜力。为了实现这一目标，该项目将开发理论基础的算法，以促进在特定于应用程序的资源限制下设计机器学习模型。该项目的成果将有助于使机器学习方法能够在更少的人工注释数据、更少的计算能力和更广泛的硬件平台上运行。为了展示跨学科的影响，所产生的算法将用于设计高效的水文模型，以帮助预测和管理水资源。该研究还将通过指导本科生、开发新的本科生和研究生课程以及通过公开在线平台直播讲座等方式与教育紧密结合。该项目旨在开发基础理论和算法，以指导统计和计算资源的有效利用。统计前沿的研究重点是数据，并将揭示数据量，标签质量和模型准确性之间的基本权衡。理解这些权衡将导致改进的损失函数和正则化技术的设计。在计算方面，将通过探索模型大小和准确性之间的相互作用来开发理论启发的模型压缩方案。其次，通过协同设计架构、压缩方案和损失函数的计算高效算法来识别最佳模型架构，从而增强模型性能。这些理论和算法研究将利用统计学习，优化，深度学习理论和高维概率的工具。预计该研究将为半监督学习，模型压缩，神经结构搜索，该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Contemporary machine learning techniques tend to be resource-intensive, often requiring good quality datasets, expensive hardware, or significant computing power. In a wide array of application domains, ranging from healthcare to mobile computing, these critical resources are lacking. Novel methodologies that enable the optimal utilization of resources can help unlock the full potential of the data science revolution for these domains. Towards this aim, this project will develop theoretically-grounded algorithms to facilitate the design of machine learning models under application-specific resource constraints. The outcomes of the project will help enable machine learning methods to operate with less human-annotated data, less computing power, and on a wider range of hardware platforms. To demonstrate interdisciplinary impact, the resulting algorithms will be employed in the design of efficient hydrological models which aid in predicting and managing water resources. The research will also be strongly coupled with education through the mentoring of undergraduate students, new undergraduate and graduate course development, and live broadcasts of the lectures over publicly accessible online platforms.This project aims to develop the foundational theories and algorithms to guide the efficient use of statistical and computational resources. The research on the statistical front focuses on the data and will uncover the fundamental tradeoffs between the data amount, label quality, and the model accuracy. Understanding these tradeoffs will lead to the design of improved loss functions and regularization techniques. On the computational front, theory-inspired model compression schemes will be developed by exploring the interplay between the model size and accuracy. Secondly, the model performance will be enhanced by identifying the optimal model architecture via computationally-efficient algorithms that co-design the architecture, compression scheme, and the loss function. These theoretical and algorithmic investigations will utilize tools from statistical learning, optimization, deep learning theory, and high-dimensional probability. The proposed research is expected to provide much-needed theoretical basis for poorly-understood heuristics in fields spanning semi-supervised learning, model compression, neural architecture search, and will guide the design of next-generation algorithms achieving the optimal resource tradeoffs.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.