CAREER: Rich and Scalable Optimization for Modern Bayesian Nonparametric Learning

大规模数据分析已经成为整个学术界和工业界不可或缺的工具。当数据量非常大时，人们经常面临模型的丰富性、灵活性和潜在的预测能力与计算要求之间的权衡。虽然统计学和机器学习的最新进展为我们提供了一套丰富的模型和工具，但由于可用于分析的数据量太大，其中许多模型和工具无法应用于当代应用。特别是，贝叶斯非参数模型是一类丰富的模型，在很大程度上被限制在小规模背景下。与标准的参数统计模型相比，这类方法并不固定模型的复杂性，而是允许数据确定结果模型的复杂程度。虽然这些模型似乎很适合大规模数据分析，但目前贝叶斯非参数模型中的推理方法尚未被证明在规模上有效。在更广泛的影响方面，机器学习在医学、工程和人文等领域的应用不断涌现；这项研究有可能影响这些领域的一些问题。此外，PI的研究在计算机视觉领域也有应用，而计算机视觉本身在自动驾驶和老年人护理方面也产生了新的影响。该项目还包括进一步将计算机科学的课程作业与统计学的课程作业相结合的努力，目的是继续弥合这两个领域之间的差距。最后，这项研究将引入本科生和高中生进行研究，并将开发出可供机器学习领域以外的实践者应用的非参数问题的新软件。该职业项目探索可扩展的优化方法，旨在使贝叶斯非参数模型适用于大规模环境。本课题的研究主要集中在三个方面：1)可伸缩非参数模型的小方差渐近性。该项目的这一部分旨在为非参数问题开发可扩展的新算法，这些算法可以应用于主题建模、图像分割和图像特征学习等问题。目标包括将小方差渐近技术扩展到新的贝叶斯非参数模型，改进渐近技术的理论基础，并开发用于几个问题的大型软件。2)非参数问题的新变分推理技术。该项目的这一部分侧重于将变分推理方法扩展到新的环境。特别是，PI专注于将变分推理应用于Gamma过程模型，并将开发针对新兴的指数族完全随机测量的变分推理方法。3)大规模贝叶斯非参数的新应用。该项目的这一部分使用前两部分的结果来探索贝叶斯非参数模型的应用，这些模型以前是无法实现的。应用包括大规模图像建模、社交网络分析和大规模文档分析。

Large-scale data analysis has become an indispensable tool throughout academia and industry. When the amount of data is very large, one often faces a tradeoff between the richness, flexibility, and potential predictive power of the models, and the computational requirements. While recent advances in statistics and machine learning provide us with a rich set of models and tools, many of these cannot be applied in contemporary applications due to the sheer volume of data available for analysis. In particular, Bayesian nonparametric models are a rich class of models which have largely been restricted to small-scale setting. This class of methods, in contrast to standard parametric statistical models, does not fix the complexity of the model, instead allowing the data to determine how complex the resulting models are. While these models appear well-suited for large-scale data analysis, current methods for inference in Bayesian nonparametric models have not been shown to work at scale. In terms of broader impacts, applications of machine learning continue to emerge in fields such as medicine, engineering, and the humanities; this research has the potential to impact a number of problems in these fields. Further, the PI's research has applications in the field of computer vision, which itself has emerging impacts in autonomous driving and eldercare. The project also includes an effort to further integrate coursework in computer science with coursework in statistics, aiming to continue to bridge the gap between the fields. Finally, the research will introduce undergraduate and high school students to research, and will also yield new software for nonparametric problems that can be applied by practitioners outside the machine learning field.This CAREER project explores scalable optimization methods aimed at making Bayesian nonparametric models applicable in large-scale settings. The research in this project is focused on three general themes:1) Small-variance asymptotics for scalable nonparametric modeling. This part of the project aims to develop new and scalable algorithms for nonparametric problems that can be applied to problems such as topic modeling, image segmentation, and image feature learning. The goals include extending the technique of small-variance asymptotics to new Bayesian nonparametric models, improving the theoretical underpinnings of the asymptotic techniques, and developing large-scale software for several problems.2) New variational inference techniques for nonparametric problems. This part of the project focuses on extending variational inference methods to new settings. In particular, the PI focuses on applying variational inference to gamma process models, and will develop variational inference methods for the emerging class of exponential-family completely random measures.3) New applications for large-scale Bayesian nonparametrics. This part of the project uses the results in the previous two parts to explore applications of Bayesian nonparametric models that were previously unattainable. Applications include large-scale image modeling, social network analysis, and large-scale document analysis.