CAREER: Rich and Scalable Optimization for Modern Bayesian Nonparametric Learning

大规模数据分析已经成为学术界和工业界不可或缺的工具。当数据量非常大时，通常需要在模型的丰富性、灵活性和潜在预测能力与计算需求之间进行权衡。虽然统计学和机器学习的最新进展为我们提供了一套丰富的模型和工具，但由于可供分析的数据量庞大，其中许多模型和工具无法应用于当代应用。特别是贝叶斯非参数模型是一类丰富的模型，很大程度上局限于小尺度环境。与标准参数统计模型相比，这类方法不固定模型的复杂性，而是允许数据确定最终模型的复杂程度。虽然这些模型似乎非常适合大规模数据分析，但目前在贝叶斯非参数模型中进行推理的方法尚未被证明适用于大规模数据分析。就更广泛的影响而言，机器学习的应用不断出现在医学、工程和人文等领域；这项研究有可能影响这些领域的许多问题。此外，PI的研究在计算机视觉领域也有应用，而计算机视觉本身在自动驾驶和老年人护理方面也有新兴的影响。该项目还包括进一步整合计算机科学课程与统计学课程，旨在继续弥合这两个领域之间的差距。最后，该研究将向本科生和高中生介绍研究，并将产生用于非参数问题的新软件，可由机器学习领域以外的从业者应用。这个CAREER项目探索可扩展的优化方法，旨在使贝叶斯非参数模型适用于大规模设置。本项目的研究主要集中在三个主题上：1)可扩展非参数建模的小方差渐近性。该项目的这一部分旨在为非参数问题开发新的可扩展算法，这些算法可以应用于主题建模、图像分割和图像特征学习等问题。目标包括将小方差渐近技术扩展到新的贝叶斯非参数模型，改进渐近技术的理论基础，以及为几个问题开发大规模软件。非参数问题的新变分推理技术。项目的这一部分侧重于将变分推理方法扩展到新的设置。特别是，PI专注于将变分推理应用于伽马过程模型，并将为新兴的指数族完全随机测度类开发变分推理方法。3)大规模贝叶斯非参数的新应用。项目的这一部分使用前两部分的结果来探索以前无法实现的贝叶斯非参数模型的应用。应用包括大规模图像建模、社会网络分析和大规模文档分析。

Large-scale data analysis has become an indispensable tool throughout academia and industry. When the amount of data is very large, one often faces a tradeoff between the richness, flexibility, and potential predictive power of the models, and the computational requirements. While recent advances in statistics and machine learning provide us with a rich set of models and tools, many of these cannot be applied in contemporary applications due to the sheer volume of data available for analysis. In particular, Bayesian nonparametric models are a rich class of models which have largely been restricted to small-scale setting. This class of methods, in contrast to standard parametric statistical models, does not fix the complexity of the model, instead allowing the data to determine how complex the resulting models are. While these models appear well-suited for large-scale data analysis, current methods for inference in Bayesian nonparametric models have not been shown to work at scale. In terms of broader impacts, applications of machine learning continue to emerge in fields such as medicine, engineering, and the humanities; this research has the potential to impact a number of problems in these fields. Further, the PI's research has applications in the field of computer vision, which itself has emerging impacts in autonomous driving and eldercare. The project also includes an effort to further integrate coursework in computer science with coursework in statistics, aiming to continue to bridge the gap between the fields. Finally, the research will introduce undergraduate and high school students to research, and will also yield new software for nonparametric problems that can be applied by practitioners outside the machine learning field.This CAREER project explores scalable optimization methods aimed at making Bayesian nonparametric models applicable in large-scale settings. The research in this project is focused on three general themes:1) Small-variance asymptotics for scalable nonparametric modeling. This part of the project aims to develop new and scalable algorithms for nonparametric problems that can be applied to problems such as topic modeling, image segmentation, and image feature learning. The goals include extending the technique of small-variance asymptotics to new Bayesian nonparametric models, improving the theoretical underpinnings of the asymptotic techniques, and developing large-scale software for several problems.2) New variational inference techniques for nonparametric problems. This part of the project focuses on extending variational inference methods to new settings. In particular, the PI focuses on applying variational inference to gamma process models, and will develop variational inference methods for the emerging class of exponential-family completely random measures.3) New applications for large-scale Bayesian nonparametrics. This part of the project uses the results in the previous two parts to explore applications of Bayesian nonparametric models that were previously unattainable. Applications include large-scale image modeling, social network analysis, and large-scale document analysis.