CAREER: Bayesian Nonparametric Learning for Large-Scale Structure Discovery

职业：用于大规模结构发现的贝叶斯非参数学习

• 批准号: 1758028
• 负责人: Erik Sudderth
• 金额: $ 33.13万
• 依托单位: University of California-Irvine
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-09-01 至 2021-02-28
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1758028&HistoricalAwards=false
• 关键词: CAREER; Bayesian; Nonparametric; Learning; Large

CAREER: Bayesian Nonparametric Learning for Large-Scale Structure DiscoveryThis CAREER project will advance the state-of-the-art for automated discovery of structure within data as diverse as images and video, natural language, audio sequences, and social and biological networks. Contemporary applications of statistical machine learning are dominated by parametric models. This approach constructs models of pre-determined size (with a finite-dimensional vector of parameters which) are tuned using training data. To be effective, the underlying structure of such models must be manually specified by experts with application-specific knowledge. This presumed structure imposes limits on what can possibly be learned even from very big datasets.Bayesian nonparametric models instead define distributions on models of arbitrary size with infinite-dimensional spaces of functions, partitions, or other combinatorial structures. They lead to flexible, data-driven unsupervised learning algorithms, and models whose internal structure continually grows and adapts to new observations. Bayesian nonparametric models, while promising, are an incompletely-developed technology posing significant challenges to practice. This CAREER project will increase the practical feasibility and impact of Bayesian nonparametric approaches by pursuing three interrelated themes:1) Nonparametric Model Design and Evaluation. New families of models for data with hierarchical, spatial, temporal, or relational structure are investigated. Quantitative validation of the statistical assumptions and biases inherent in these models will be emphasized, evaluating whether these align with the empirical statistics of significant application areas.2) Reliable Structure Discovery. Statistical inference algorithms which move beyond the local moves of standard (and widely used) Monte Carlo and variational methods will be developed. Compelling examples indicate that local optima are a significant issue for contemporary methods, so a family of novel algorithms is proposed, which dynamically adjust model complexity as learning proceeds.3) Scalable and Extensible Nonparametric Learning. Common patterns across a wide range of popular nonparametric models are identified, which suggest a corresponding family of scalable and parallelizable online learning algorithms. The "memoized" online variational inference algorithm avoids some practical instabilities and sensitivities of conventional methods, while allowing provably correct optimization of the nonparametric model structure and complexity.An extensible "BNPy: Bayesian Nonparametric Learning in Python" software package is under development to allow easy application of the novel learning algorithms to a wide range of current and future BNP models. The education and outreach plan of this CAREER project leverages this software to create interdisciplinary undergraduate research teams exploring applications in the natural and social sciences, and a week-long summer school on Bayesian nonparametrics to be held twice at Brown University's Institute for Computational and Experimental Research in Mathematics (ICERM).

职业：Bayesian Nonparametric Learning for Large-Scale Structure DiscoveryThis CAREER项目将推进最先进的自动化发现数据中的结构，如图像和视频，自然语言，音频序列以及社交和生物网络。 统计机器学习的当代应用由参数模型主导。 这种方法构建了预定大小的模型（具有有限维参数向量），并使用训练数据进行调整。 为了有效，此类模型的底层结构必须由具有特定于应用程序知识的专家手动指定。 这种假定的结构限制了即使从非常大的数据集也可能学习到的内容。贝叶斯非参数模型定义了任意大小的模型上的分布，这些模型具有无限维的函数空间，分区或其他组合结构。 它们导致了灵活的、数据驱动的无监督学习算法，以及内部结构不断增长并适应新观察的模型。 贝叶斯非参数模型，而有前途的，是一个不完全发展的技术带来重大挑战的实践。 这个CAREER项目将通过追求三个相互关联的主题来增加贝叶斯非参数方法的实际可行性和影响力：1）非参数模型设计和评估。 研究了具有层次、空间、时间或关系结构的数据模型的新家族。 将强调这些模型中固有的统计假设和偏差的定量验证，评估这些假设和偏差是否与重要应用领域的经验统计相一致。 将开发超越标准（和广泛使用的）蒙特卡罗和变分方法的局部移动的统计推断算法。 令人信服的例子表明，局部最优是当代方法的一个重要问题，因此提出了一系列新的算法，动态调整模型的复杂性作为学习过程。3）可扩展的可扩展非参数学习。 在广泛的流行的非参数模型的共同模式被确定，这表明相应的家庭的可扩展性和并行化的在线学习算法。 “memoized”的在线变分推理算法避免了一些实际的不稳定性和传统方法的敏感性，同时允许可证明正确的优化的非参数模型的结构和复杂性。一个可扩展的“BNPy：贝叶斯非参数学习在Python”软件包正在开发中，以允许容易地应用到广泛的当前和未来的BNP模型的新的学习算法。 这个CAREER项目的教育和推广计划利用该软件创建跨学科的本科研究团队，探索自然科学和社会科学的应用，以及为期一周的贝叶斯非参数学暑期学校将在布朗大学的计算和实验研究所举行两次数学（ICERM）。

项目成果

Variational Training for Large-Scale Noisy-OR Bayesian Networks

• 发表时间: 2019
• 作者: Geng Ji;Dehua Cheng;Huazhong Ning;Changhe Yuan;Hanning Zhou;Liang Xiong;Erik B. Sudderth