RI: AF: Small: Optimizing probabilities for learning: sampling meets optimization

大规模机器学习和人工智能（AI）的方法在过去十年中对世界产生了重大影响，包括在工业和科学背景下。这些惊人的成功是由大量数据集的可用性，以及从这些数据集中提取有用信息和见解的适当模型和算法的结合所驱动的。本研究项目旨在通过利用采样和优化之间的相互作用，推进大规模机器学习和人工智能算法的方法论和理解。特别是，解决了两大挑战：第一，优化理论的工具和见解可以为采样方法开发更有效的设计和分析技术；其次，这些技术可用于设计和分析问题的优化方法，例如深度学习中出现的问题。该项目的成功研究成果可能会增加对采样和优化方法的理解，并促进其原则性设计。成功的结果在使用大规模采样和优化方法的大型和不断增长的应用中具有重大的实际影响潜力，包括计算机视觉，语音识别和自动驾驶汽车。这项研究将支持研究生的发展，将通过伯克利的大型研究生课程及其基于网络的课程材料进行传播，并有可能通过在部署的人工智能系统中应用所研究的方法，使更广泛的社区受益。该项目主要有三个技术方向。首先，它旨在通过证明下界来识别采样问题的固有难度。其次，它的目的是产生分析工具和设计方法的抽样算法基于一定的随机微分方程家族被称为朗格万扩散。这将使开发具有性能保证的采样算法成为可能。第三，从抽样技术的角度分析和设计非凸优化问题的随机梯度方法，如深度神经网络的参数优化。该项目的另一个成果将是组织一个关于采样和优化之间的接口的专题讲习班。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Methods for large-scale machine learning and artificial intelligence (AI) have had major impacts on the world over the past decade, including in both industrial and scientific contexts. These spectacular successes are driven by a combination of the availability of massive datasets, and appropriate models and algorithms for extracting useful information and insights from these datasets. This research project aims to advance the methodology and understanding of algorithms for large-scale machine learning and AI by exploiting the interplay between sampling and optimization. In particular, two grand challenges are addressed: first, the tools and insights of optimization theory can develop more effective design and analysis techniques for sampling methods; second, these techniques can be used to design and analyze optimization methods for problems such as those that arise in deep learning. Successful research outcomes of this project are likely to increase the understanding of methods used for sampling and for optimization, and to facilitate their principled design. Successful outcomes have a significant potential for practical impact in the large and growing set of applications where large-scale sampling and optimization methods are used, including computer vision, speech recognition, and self-driving cars. The research will support the development of graduate students, will be disseminated through large graduate courses at Berkeley and their web-based course materials, and has the potential to benefit the broader community through the application of the methods studied in deployed AI systems.The project has three main technical directions. First, it aims to identify the inherent difficulty of sampling problems by proving lower bounds. Second, it aims to produce analysis tools and design methodologies for sampling algorithms based on a certain family of stochastic differential equations known as a Langevin diffusion. This will enable the development of sampling algorithms with performance guarantees. Third, it will use the viewpoint of sampling techniques to analyze and design stochastic gradient methods for nonconvex optimization problems, such as the optimization of parameters in deep neural networks. An additional outcome of the project will be the organization of a workshop on the topic of the interface between sampling and optimization.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.