CAREER: Bayesian Nonparametric Learning for Large-Scale Structure Discovery

职业生涯：用于大规模结构发现的贝叶斯非参数学习这个职业项目将推动在图像和视频、自然语言、音频序列以及社会和生物网络等各种数据中自动发现结构的最先进水平。统计机器学习的当代应用被参数模型所主导。该方法使用训练数据来构建预定大小的模型(具有参数的有限维向量)。为了有效，这些模型的底层结构必须由具有特定应用程序知识的专家手动指定。这种假定的结构对即使从非常大的数据集也可以学习到的东西施加了限制。相反，贝叶斯非参数模型定义了具有无限维空间的函数、划分或其他组合结构的任意大小模型的分布。它们导致了灵活的、数据驱动的非监督学习算法，以及内部结构不断增长并适应新观测的模型。贝叶斯非参数模型虽然前景看好，但它是一种不完全发展的技术，给实践带来了巨大的挑战。这个职业项目将通过追求三个相互关联的主题来提高贝叶斯非参数方法的实践可行性和影响力：1)非参数模型设计和评估。研究了具有层次、空间、时间或关系结构的数据的新的模型族。将强调对这些模型中固有的统计假设和偏差的定量验证，评估这些假设和偏差是否与重要应用领域的经验统计一致。2)可靠结构发现。将开发超越标准(和广泛使用的)蒙特卡罗和变分方法的局部移动的统计推断算法。有说服力的例子表明，局部最优是当代方法的一个重要问题，因此提出了一类新的算法，该算法随着学习过程的进行动态调整模型的复杂性。3)可伸缩和可扩展的非参数学习。识别了广泛流行的非参数模型的共同模式，提出了相应的可伸缩和可并行化的在线学习算法家族。“记忆”在线变分推理算法避免了传统方法的一些实际不稳定性和敏感性，同时允许对非参数模型结构和复杂性进行可证明的正确优化。一个可扩展的“BNPy：Python中的贝叶斯非参数学习”软件包正在开发中，以便于将新的学习算法应用于广泛的当前和未来的BNP模型。该职业项目的教育和推广计划利用该软件创建了探索自然和社会科学应用的跨学科本科研究团队，并将在布朗大学的计算和实验数学研究所(ICERM)举办为期一周的贝叶斯非参数学暑期班两次。

CAREER: Bayesian Nonparametric Learning for Large-Scale Structure DiscoveryThis CAREER project will advance the state-of-the-art for automated discovery of structure within data as diverse as images and video, natural language, audio sequences, and social and biological networks. Contemporary applications of statistical machine learning are dominated by parametric models. This approach constructs models of pre-determined size (with a finite-dimensional vector of parameters which) are tuned using training data. To be effective, the underlying structure of such models must be manually specified by experts with application-specific knowledge. This presumed structure imposes limits on what can possibly be learned even from very big datasets.Bayesian nonparametric models instead define distributions on models of arbitrary size with infinite-dimensional spaces of functions, partitions, or other combinatorial structures. They lead to flexible, data-driven unsupervised learning algorithms, and models whose internal structure continually grows and adapts to new observations. Bayesian nonparametric models, while promising, are an incompletely-developed technology posing significant challenges to practice. This CAREER project will increase the practical feasibility and impact of Bayesian nonparametric approaches by pursuing three interrelated themes:1) Nonparametric Model Design and Evaluation. New families of models for data with hierarchical, spatial, temporal, or relational structure are investigated. Quantitative validation of the statistical assumptions and biases inherent in these models will be emphasized, evaluating whether these align with the empirical statistics of significant application areas.2) Reliable Structure Discovery. Statistical inference algorithms which move beyond the local moves of standard (and widely used) Monte Carlo and variational methods will be developed. Compelling examples indicate that local optima are a significant issue for contemporary methods, so a family of novel algorithms is proposed, which dynamically adjust model complexity as learning proceeds.3) Scalable and Extensible Nonparametric Learning. Common patterns across a wide range of popular nonparametric models are identified, which suggest a corresponding family of scalable and parallelizable online learning algorithms. The "memoized" online variational inference algorithm avoids some practical instabilities and sensitivities of conventional methods, while allowing provably correct optimization of the nonparametric model structure and complexity.An extensible "BNPy: Bayesian Nonparametric Learning in Python" software package is under development to allow easy application of the novel learning algorithms to a wide range of current and future BNP models. The education and outreach plan of this CAREER project leverages this software to create interdisciplinary undergraduate research teams exploring applications in the natural and social sciences, and a week-long summer school on Bayesian nonparametrics to be held twice at Brown University's Institute for Computational and Experimental Research in Mathematics (ICERM).