BIGDATA: F: DKA: Collaborative Research: Theory and Algorithms for Parallel Probabilistic Inference with Big Data, via Big Model, in Realistic Distributed Computing Environments

该项目开发了一个新的框架，使机器学习（ML）系统能够通过大型计算机集群上的并行贝叶斯推理自动理解和挖掘大量复杂的数据。该研究对大学习的实践和方向产生了深远的影响。所开发的技术对机器学习研究和应用都有催化作用：机器学习科学家能够以最少的编程工作快速试验新颖、尖端的机器学习模型，而不受单机限制的阻碍。来自生物学和社会科学等其他领域的研究人员能够运行当代先进的机器学习方法，这些方法超越了简单模型的能力，从而对数据产生新的科学见解，否则这些数据的规模将令人望而生畏。小型初创企业的数据科学家能够使用复杂的模型进行机器学习分析，其能力与拥有专门工程和基础设施团队的大公司相当。学生和初学者只需几行代码即可见证分布式机器学习的实际应用，将机器学习教育推向新的高度。从技术上讲，这项研究的重点是扩大和并行化贝叶斯机器学习，它为建模各种数据集提供了一个强大、优雅且理论上合理的框架。 研究团队为分层贝叶斯模型开发了一套互补的分布式推理算法，涵盖了最常用的贝叶斯机器学习方法。该项目侧重于将速度和可扩展性与理论保证相结合，使我们能够评估所得方法的准确性，并允许从业者在速度和准确性之间进行权衡。该项目不是专注于几个互不相关的模型，而是开发适用于广泛的分层贝叶斯模型的技术，从而形成一个构建块工具包，可以根据任意概率模型的需要进行组合——无论是参数模型还是非参数模型、判别模型还是生成模型。这与许多现有的并行推理工作形成鲜明对比，后者往往侧重于特定模型中的并行化，并且不能轻易扩展。该项目通过强大的模型为学习大数据提供了坚实的算法基础。该研究通过为用户和开发者社区提供通用并行算法库来使用计算机集群和云解决各种问题，从而弥合数据实际需求与 ML 基础研究之间的差距，从而有助于使先进和大规模的 ML 方法民主化以适应广泛的应用。

This project develops a new framework that enables machine learning (ML) systems to automatically comprehend and mine massive and complex data via parallel Bayesian inference on large computer clusters. The research has a profound impact on the practice and direction of Big Learning. The developed technologies have a catalytic effect on both ML research and applications: ML scientists are able to rapidly experiment on novel, cutting-edge ML models with minimal programming effort, unhindered by the limitations of single machines. Researchers from other fields, like biology and social sciences, are able to run contemporary advanced ML methods that transcend the capabilities of simple models, yielding new scientific insights on data whose size would otherwise be daunting. Data scientists at small start-ups are able to conduct ML analytics with complex models, putting their capabilities on par with huge companies possessing dedicated engineering and infrastructure teams. Students and beginners are able to witness distributed ML in action with just a few lines of code, driving ML education to new heights. Technically, this research focuses on scaling up and parallelizing Bayesian machine learning, which provides a powerful, elegant and theoretically justified framework for modeling a wide variety of datasets. The research team develops a suite of complementary distributed inference algorithms for hierarchical Bayesian models, which cover most commonly used Bayesian ML methods. The project focuses on combining speed and scalability with theoretical guarantees that allow us to assess the accuracy of the resulting methods, and allow practitioners to make trade-offs between speed and accuracy. Rather than focus on a few disconnected models, the project develops techniques applicable to a broad spectrum of hierarchical Bayesian models, resulting in a toolkit of building blocks that can be combined as needed for arbitrary probabilistic models - be they parametric or nonparametric, discriminative or generative. This is in contrast to much existing work on parallel inference, which tends to focus on parallelization in a specific model and cannot be easily extended. The project provides a solid algorithmic foundation for learning on Big Data with powerful models. The research contributes to democratizing advanced and large-scale ML methods for broad applications, by offering the user and developer community a library of general-purpose parallelizable algorithms for working on diverse problems using computer clusters and the cloud, bridging the gap between practical needs from data and basic research in ML.