CAREER: Bayesian Nonparametric Learning for Large-Scale Structure Discovery

职业：大规模结构发现的贝叶斯非参数学习这个职业项目将推进最先进的数据结构的自动发现，如图像和视频、自然语言、音频序列、社会和生物网络等。统计机器学习的当代应用主要是参数模型。该方法构建预先确定大小的模型（具有有限维参数向量），并使用训练数据进行调优。为了有效，这些模型的底层结构必须由具有应用程序特定知识的专家手动指定。这种假定的结构限制了从非常大的数据集中可能学到的东西。贝叶斯非参数模型在任意大小的模型上定义分布，这些模型具有无限维的函数、分区或其他组合结构空间。它们导致了灵活的、数据驱动的无监督学习算法，以及内部结构不断增长并适应新观察结果的模型。贝叶斯非参数模型虽然很有前途，但它是一种不完全发展的技术，对实践提出了重大挑战。本CAREER项目将通过追求三个相互关联的主题来增加贝叶斯非参数方法的实际可行性和影响：1)非参数模型设计和评估。研究了具有层次结构、空间结构、时间结构或关系结构的数据模型的新家族。将强调这些模型中固有的统计假设和偏差的定量验证，评估这些是否与重要应用领域的经验统计相一致。2)可靠的结构发现。统计推理算法超越了标准的（和广泛使用的）蒙特卡罗和变分方法的局部移动将被开发。令人信服的例子表明，局部最优是当代方法的重要问题，因此提出了一系列新的算法，随着学习的进行动态调整模型的复杂度。3)可扩展非参数学习。在广泛流行的非参数模型中识别了常见模式，这表明了相应的可扩展和可并行的在线学习算法家族。“记忆”在线变分推理算法避免了传统方法的一些实际不稳定性和敏感性，同时允许对非参数模型结构和复杂性进行可证明的正确优化。一个可扩展的“BNPy: Python中的贝叶斯非参数学习”软件包正在开发中，以便将新的学习算法轻松应用于广泛的当前和未来的BNP模型。CAREER项目的教育和推广计划利用该软件创建跨学科本科生研究团队，探索自然科学和社会科学的应用，并在布朗大学数学计算和实验研究所（ICERM）举办为期一周的贝叶斯非参数暑期学校。

CAREER: Bayesian Nonparametric Learning for Large-Scale Structure DiscoveryThis CAREER project will advance the state-of-the-art for automated discovery of structure within data as diverse as images and video, natural language, audio sequences, and social and biological networks. Contemporary applications of statistical machine learning are dominated by parametric models. This approach constructs models of pre-determined size (with a finite-dimensional vector of parameters which) are tuned using training data. To be effective, the underlying structure of such models must be manually specified by experts with application-specific knowledge. This presumed structure imposes limits on what can possibly be learned even from very big datasets.Bayesian nonparametric models instead define distributions on models of arbitrary size with infinite-dimensional spaces of functions, partitions, or other combinatorial structures. They lead to flexible, data-driven unsupervised learning algorithms, and models whose internal structure continually grows and adapts to new observations. Bayesian nonparametric models, while promising, are an incompletely-developed technology posing significant challenges to practice. This CAREER project will increase the practical feasibility and impact of Bayesian nonparametric approaches by pursuing three interrelated themes:1) Nonparametric Model Design and Evaluation. New families of models for data with hierarchical, spatial, temporal, or relational structure are investigated. Quantitative validation of the statistical assumptions and biases inherent in these models will be emphasized, evaluating whether these align with the empirical statistics of significant application areas.2) Reliable Structure Discovery. Statistical inference algorithms which move beyond the local moves of standard (and widely used) Monte Carlo and variational methods will be developed. Compelling examples indicate that local optima are a significant issue for contemporary methods, so a family of novel algorithms is proposed, which dynamically adjust model complexity as learning proceeds.3) Scalable and Extensible Nonparametric Learning. Common patterns across a wide range of popular nonparametric models are identified, which suggest a corresponding family of scalable and parallelizable online learning algorithms. The "memoized" online variational inference algorithm avoids some practical instabilities and sensitivities of conventional methods, while allowing provably correct optimization of the nonparametric model structure and complexity.An extensible "BNPy: Bayesian Nonparametric Learning in Python" software package is under development to allow easy application of the novel learning algorithms to a wide range of current and future BNP models. The education and outreach plan of this CAREER project leverages this software to create interdisciplinary undergraduate research teams exploring applications in the natural and social sciences, and a week-long summer school on Bayesian nonparametrics to be held twice at Brown University's Institute for Computational and Experimental Research in Mathematics (ICERM).