RI: Small: Parallel Methods for Large-Scale Probabilistic Inference

我们正在经历一场数据革命。我们已经习惯了计算的不断剧变--更快的处理器、更大的存储空间和更快的网络--但本世纪提出了一个新的挑战，即几乎不受限制地访问原始数据。无论是来自传感器网络、社会计算还是高通量细胞生物学，我们都面临着关于我们这个世界的海量数据。科学家、工程师、政策制定者和实业家需要使用这些海量数据来做出更好的决策。本研究项目旨在为实现这些目标的工具提供基础。简单的模型只能提供粗略的理解。世界是复杂和动态的，提供了丰富的信息。此外，不确定性的表示对于发现复杂数据中的模式至关重要。不仅许多自然过程本质上是随机的，而且我们的知识总是有限的。概率演算使我们能够表示这种不确定性，并设计算法以在不可预测的世界中有效地行动。概率分析的黄金标准是马尔可夫链蒙特卡罗(MCMC)，这是一种识别与观测数据一致的关于世界未观测结构的假设的方法。它是执行数据分析的一种强大而有原则的方法，但传统的MCMC方法不能很好地映射到现代计算环境中。MCMC是一个顺序过程，通常不能利用多核台式机和笔记本电脑、云计算和图形处理单元提供的并行性。这项研究将为MCMC开发新的方法，这些方法被证明是正确的，但利用了大规模并行计算的优势。这项工作有各种更广泛的影响。除了核心技术贡献外，该项目还参与了深度科学合作。新的光伏材料将带来更好的太阳能电池和更可持续的能源生产。揭示基因调控机制的新技术将导致对疾病的更好理解。老鼠活动的量化模型将使我们深入了解行为的神经基础，并提供对大脑疾病的更深层次的理解。从技术角度来看，这项工作追求两种互补的方法来使用MCMC进行大规模贝叶斯数据分析：1)一个新的通用框架，用于并行马尔可夫链之间的信息共享，以实现更快的混合；2)一个新的计算概念，用于单个马尔可夫链的推测并行化。这些理论和实践探索，结合相关开源软件的发布，将产生更健壮和可扩展的概率建模。这将为后验模拟的马尔可夫转移算子的并行化提供可证明正确的基础和高效的新算法。这些操作符将用于代表大规模统计推断的方法学需求的三个合作：1)预测新型有机光伏材料的效率，2)发现新的遗传调控机制，3)小鼠行为的定量神经科学模型。虽然这项建议关注的是这些问题的一般性技术挑战，但这些合作提供了令人信服的例子，说明机器学习如何能够广泛转型。最后，该项目包括一个重要的外展部分，与当地中学生接触，并在夏季研究中涉及代表性不足的少数族裔。

We are undergoing a revolution in data. We have grown accustomed to constant upheaval in computing -- quicker processors, bigger storage and faster networks -- but this century presents the new challenge of almost unlimited access to raw data. Whether from sensor networks, social computing, or high-throughput cell biology, we face a deluge of data about our world. Scientists, engineers, policymakers, and industrialists need to use these enormous floods of data to make better decisions. This research project is about providing foundations for tools to achieve these goals. Simple models give only coarse understanding. The world is sophisticated and dynamic, providing rich information. Furthermore, representation of uncertainty is critical to discovering patterns in complex data. Not only are many natural processes intrinsically random, but our knowledge is always limited. The calculus of probability allows us to represent this uncertainty and design algorithms to act effectively in an unpredictable world. The gold standard for probabilistic analysis is Markov chain Monte Carlo (MCMC), a way to identify hypotheses about the unobserved structure of the world that are consistent with observed data. It is a powerful and principled way to perform data analysis, but traditional MCMC methods do not map well onto modern computing environments. MCMC is a sequential procedure that cannot generally take advantage of the parallelism offered by multi-core desktops and laptops, cloud computing, and graphical processing units. This research will develop new methods for MCMC that are provably correct, but that take advantage of large-scale parallel computing. There are a variety of broader impacts of this work. In addition to the core technical contributions, the project engages in deep scientific collaborations. New photovoltaic materials will lead to better solar cells and more sustainable energy production. New techniques for uncovering genetic regulatory mechanisms will lead to better understanding of disease. Quantitative models of mouse activity will give insight into the neural basis of behavior and provide a deeper understanding of brain disorders. From a technical point of view, this work pursues two complementary approaches to large-scale Bayesian data analysis with MCMC: 1) a novel general-purpose framework for sharing of information between parallel Markov chains for faster mixing, and 2) a new computational concept for speculative parallelization of individual Markov chains. These theoretical and practical explorations, combined with the release of associated open source software, will yield more robust and scalable probabilistic modeling. It will develop provably-correct foundations and efficient new algorithms for parallelization of Markov transition operators for posterior simulation. These operators will be used in three collaborations that are representative of the methodological demands for large-scale statistical inference: 1) predicting the efficiencies of novel organic photovoltaic materials, 2) discovering new genetic regulatory mechanisms, and 3) quantitative neuroscientific models for mouse behavior. While this proposal focuses on the generalizable technical challenges of these problems, these collaborations provide compelling examples of how machine learning can be broadly transformative.Finally, the project includes a significant outreach component, engaging with local middle schoolers, and involving underrepresented minorities in summer research.