RI: Small: Parallel Methods for Large-Scale Probabilistic Inference

我们正在经历一场数据革命。我们已经习惯了计算领域的不断变化——更快的处理器、更大的存储空间和更快的网络——但本世纪出现了几乎无限制地获取原始数据的新挑战。无论是来自传感器网络、社会计算还是高通量细胞生物学，我们都面临着关于我们世界的海量数据。科学家、工程师、政策制定者和实业家需要利用这些海量的数据来做出更好的决策。本研究项目旨在为实现这些目标的工具提供基础。简单的模型只能给出粗略的理解。世界是复杂和动态的，提供了丰富的信息。此外，不确定性的表示对于发现复杂数据中的模式至关重要。不仅许多自然过程本质上是随机的，而且我们的知识总是有限的。概率演算使我们能够表示这种不确定性，并设计算法，以便在不可预测的世界中有效地采取行动。概率分析的黄金标准是马尔可夫链蒙特卡罗（MCMC），这是一种识别与观测数据一致的关于未观测到的世界结构的假设的方法。它是一种执行数据分析的强大且有原则的方法，但是传统的MCMC方法不能很好地映射到现代计算环境中。MCMC是一个顺序过程，通常不能利用多核台式机和笔记本电脑、云计算和图形处理单元提供的并行性。这项研究将为MCMC开发新的方法，这些方法可以证明是正确的，但会利用大规模并行计算的优势。这项工作有许多更广泛的影响。除了核心技术贡献外，该项目还参与深入的科学合作。新的光伏材料将带来更好的太阳能电池和更可持续的能源生产。揭示遗传调控机制的新技术将有助于更好地了解疾病。小鼠活动的定量模型将深入了解行为的神经基础，并为大脑疾病提供更深入的了解。从技术角度来看，本工作采用MCMC进行大规模贝叶斯数据分析的两种互补方法：1)在并行马尔可夫链之间共享信息的新型通用框架，以实现更快的混合；2)单个马尔可夫链的推测并行化的新计算概念。这些理论和实践的探索，结合相关开源软件的发布，将产生更健壮和可扩展的概率模型。它将为后验模拟的马尔可夫转移算子的并行化提供可证明正确的基础和有效的新算法。这些操作符将用于三个代表大规模统计推断方法需求的合作：1)预测新型有机光伏材料的效率，2)发现新的遗传调控机制，以及3)小鼠行为的定量神经科学模型。虽然本提案侧重于这些问题的一般技术挑战，但这些合作提供了令人信服的例子，说明机器学习如何具有广泛的变革性。最后，该项目包括一个重要的外展组成部分，与当地中学生接触，并在夏季研究中涉及代表性不足的少数民族。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

We are undergoing a revolution in data. We have grown accustomed to constant upheaval in computing -- quicker processors, bigger storage and faster networks -- but this century presents the new challenge of almost unlimited access to raw data. Whether from sensor networks, social computing, or high-throughput cell biology, we face a deluge of data about our world. Scientists, engineers, policymakers, and industrialists need to use these enormous floods of data to make better decisions. This research project is about providing foundations for tools to achieve these goals. Simple models give only coarse understanding. The world is sophisticated and dynamic, providing rich information. Furthermore, representation of uncertainty is critical to discovering patterns in complex data. Not only are many natural processes intrinsically random, but our knowledge is always limited. The calculus of probability allows us to represent this uncertainty and design algorithms to act effectively in an unpredictable world. The gold standard for probabilistic analysis is Markov chain Monte Carlo (MCMC), a way to identify hypotheses about the unobserved structure of the world that are consistent with observed data. It is a powerful and principled way to perform data analysis, but traditional MCMC methods do not map well onto modern computing environments. MCMC is a sequential procedure that cannot generally take advantage of the parallelism offered by multi-core desktops and laptops, cloud computing, and graphical processing units. This research will develop new methods for MCMC that are provably correct, but that take advantage of large-scale parallel computing. There are a variety of broader impacts of this work. In addition to the core technical contributions, the project engages in deep scientific collaborations. New photovoltaic materials will lead to better solar cells and more sustainable energy production. New techniques for uncovering genetic regulatory mechanisms will lead to better understanding of disease. Quantitative models of mouse activity will give insight into the neural basis of behavior and provide a deeper understanding of brain disorders. From a technical point of view, this work pursues two complementary approaches to large-scale Bayesian data analysis with MCMC: 1) a novel general-purpose framework for sharing of information between parallel Markov chains for faster mixing, and 2) a new computational concept for speculative parallelization of individual Markov chains. These theoretical and practical explorations, combined with the release of associated open source software, will yield more robust and scalable probabilistic modeling. It will develop provably-correct foundations and efficient new algorithms for parallelization of Markov transition operators for posterior simulation. These operators will be used in three collaborations that are representative of the methodological demands for large-scale statistical inference: 1) predicting the efficiencies of novel organic photovoltaic materials, 2) discovering new genetic regulatory mechanisms, and 3) quantitative neuroscientific models for mouse behavior. While this proposal focuses on the generalizable technical challenges of these problems, these collaborations provide compelling examples of how machine learning can be broadly transformative.Finally, the project includes a significant outreach component, engaging with local middle schoolers, and involving underrepresented minorities in summer research.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.