BIGDATA: F: DKA: Collaborative Research: Theory and Algorithms for Parallel Probabilistic Inference with Big Data, via Big Model, in Realistic Distributed Computing Environments

该项目开发了一个新的框架，使机器学习（ML）系统能够通过大型计算机集群上的并行贝叶斯推理自动理解和挖掘大量复杂的数据。该研究对大学习的实践和方向产生了深远的影响。开发的技术对机器学习研究和应用都有催化作用：机器学习科学家能够以最小的编程工作量快速实验新颖，尖端的机器学习模型，不受单个机器的限制。来自其他领域（如生物学和社会科学）的研究人员能够运行超越简单模型能力的当代高级ML方法，对数据产生新的科学见解，否则这些数据的规模将令人望而生畏。小型初创企业的数据科学家能够用复杂的模型进行机器学习分析，使他们的能力与拥有专门工程和基础设施团队的大公司不相上下。学生和初学者能够用几行代码见证分布式ML的作用，将ML教育推向新的高度。从技术上讲，本研究的重点是扩展和并行贝叶斯机器学习，它为各种数据集的建模提供了一个强大、优雅和理论上合理的框架。研究小组为分层贝叶斯模型开发了一套互补的分布式推理算法，涵盖了最常用的贝叶斯机器学习方法。该项目关注于将速度和可伸缩性与理论保证结合起来，使我们能够评估结果方法的准确性，并允许从业者在速度和准确性之间进行权衡。该项目不是专注于几个不相关的模型，而是开发了适用于广泛层次贝叶斯模型的技术，从而形成了一个构建模块工具包，可以根据需要组合成任意概率模型-无论是参数模型还是非参数模型，判别模型还是生成模型。这与许多现有的并行推理工作形成对比，这些工作倾向于关注特定模型中的并行化，并且不容易扩展。该项目以强大的模型为大数据学习提供了坚实的算法基础。该研究为用户和开发者社区提供了一个通用并行算法库，用于使用计算机集群和云处理各种问题，弥合了ML中数据和基础研究的实际需求之间的差距，从而为广泛应用的先进和大规模ML方法的民主化做出了贡献。

This project develops a new framework that enables machine learning (ML) systems to automatically comprehend and mine massive and complex data via parallel Bayesian inference on large computer clusters. The research has a profound impact on the practice and direction of Big Learning. The developed technologies have a catalytic effect on both ML research and applications: ML scientists are able to rapidly experiment on novel, cutting-edge ML models with minimal programming effort, unhindered by the limitations of single machines. Researchers from other fields, like biology and social sciences, are able to run contemporary advanced ML methods that transcend the capabilities of simple models, yielding new scientific insights on data whose size would otherwise be daunting. Data scientists at small start-ups are able to conduct ML analytics with complex models, putting their capabilities on par with huge companies possessing dedicated engineering and infrastructure teams. Students and beginners are able to witness distributed ML in action with just a few lines of code, driving ML education to new heights. Technically, this research focuses on scaling up and parallelizing Bayesian machine learning, which provides a powerful, elegant and theoretically justified framework for modeling a wide variety of datasets. The research team develops a suite of complementary distributed inference algorithms for hierarchical Bayesian models, which cover most commonly used Bayesian ML methods. The project focuses on combining speed and scalability with theoretical guarantees that allow us to assess the accuracy of the resulting methods, and allow practitioners to make trade-offs between speed and accuracy. Rather than focus on a few disconnected models, the project develops techniques applicable to a broad spectrum of hierarchical Bayesian models, resulting in a toolkit of building blocks that can be combined as needed for arbitrary probabilistic models - be they parametric or nonparametric, discriminative or generative. This is in contrast to much existing work on parallel inference, which tends to focus on parallelization in a specific model and cannot be easily extended. The project provides a solid algorithmic foundation for learning on Big Data with powerful models. The research contributes to democratizing advanced and large-scale ML methods for broad applications, by offering the user and developer community a library of general-purpose parallelizable algorithms for working on diverse problems using computer clusters and the cloud, bridging the gap between practical needs from data and basic research in ML.