CAREER: Rich and Scalable Optimization for Modern Bayesian Nonparametric Learning

大规模数据分析已成为学术界和工业界不可或缺的工具。 当数据量非常大时，人们经常面临模型的丰富性、灵活性和潜在预测能力与计算要求之间的权衡。 虽然统计学和机器学习的最新进展为我们提供了丰富的模型和工具，但由于可用于分析的数据量太大，其中许多无法应用于当代应用程序。 特别是，贝叶斯非参数模型是一类丰富的模型，在很大程度上被限制在小规模的设置。 与标准的参数统计模型相比，这类方法并不固定模型的复杂性，而是允许数据来确定结果模型的复杂程度。 虽然这些模型似乎非常适合大规模数据分析，但贝叶斯非参数模型中的当前推断方法尚未显示出大规模工作。在更广泛的影响方面，机器学习的应用继续出现在医学，工程和人文等领域;这项研究有可能影响这些领域的许多问题。 此外，PI的研究在计算机视觉领域也有应用，而计算机视觉本身对自动驾驶和老年人护理产生了新的影响。 该项目还包括进一步整合计算机科学课程与统计课程的努力，旨在继续弥合这两个领域之间的差距。 最后，该研究将介绍本科生和高中生的研究，也将产生新的软件，非参数问题，可以应用于机器学习领域以外的从业者。这个CAREER项目探索可扩展的优化方法，旨在使贝叶斯非参数模型适用于大规模设置。 本项目的研究主要集中在三个方面：1）可扩展非参数建模的小方差渐近性。 该项目的这一部分旨在为非参数问题开发新的可扩展算法，这些算法可以应用于主题建模，图像分割和图像特征学习等问题。 目标包括将小方差渐近技术扩展到新的贝叶斯非参数模型，改进渐近技术的理论基础，并开发用于几个问题的大型软件。2）非参数问题的新变分推理技术。 项目的这一部分着重于将变分推理方法扩展到新的设置。 特别是，PI专注于将变分推理应用于伽马过程模型，并将为新兴的指数族完全随机测度类开发变分推理方法。3）大规模贝叶斯非参数的新应用。 本专题的这一部分使用前两部分的结果来探索以前无法实现的贝叶斯非参数模型的应用。 应用包括大规模图像建模，社交网络分析和大规模文档分析。

Large-scale data analysis has become an indispensable tool throughout academia and industry. When the amount of data is very large, one often faces a tradeoff between the richness, flexibility, and potential predictive power of the models, and the computational requirements. While recent advances in statistics and machine learning provide us with a rich set of models and tools, many of these cannot be applied in contemporary applications due to the sheer volume of data available for analysis. In particular, Bayesian nonparametric models are a rich class of models which have largely been restricted to small-scale setting. This class of methods, in contrast to standard parametric statistical models, does not fix the complexity of the model, instead allowing the data to determine how complex the resulting models are. While these models appear well-suited for large-scale data analysis, current methods for inference in Bayesian nonparametric models have not been shown to work at scale. In terms of broader impacts, applications of machine learning continue to emerge in fields such as medicine, engineering, and the humanities; this research has the potential to impact a number of problems in these fields. Further, the PI's research has applications in the field of computer vision, which itself has emerging impacts in autonomous driving and eldercare. The project also includes an effort to further integrate coursework in computer science with coursework in statistics, aiming to continue to bridge the gap between the fields. Finally, the research will introduce undergraduate and high school students to research, and will also yield new software for nonparametric problems that can be applied by practitioners outside the machine learning field.This CAREER project explores scalable optimization methods aimed at making Bayesian nonparametric models applicable in large-scale settings. The research in this project is focused on three general themes:1) Small-variance asymptotics for scalable nonparametric modeling. This part of the project aims to develop new and scalable algorithms for nonparametric problems that can be applied to problems such as topic modeling, image segmentation, and image feature learning. The goals include extending the technique of small-variance asymptotics to new Bayesian nonparametric models, improving the theoretical underpinnings of the asymptotic techniques, and developing large-scale software for several problems.2) New variational inference techniques for nonparametric problems. This part of the project focuses on extending variational inference methods to new settings. In particular, the PI focuses on applying variational inference to gamma process models, and will develop variational inference methods for the emerging class of exponential-family completely random measures.3) New applications for large-scale Bayesian nonparametrics. This part of the project uses the results in the previous two parts to explore applications of Bayesian nonparametric models that were previously unattainable. Applications include large-scale image modeling, social network analysis, and large-scale document analysis.