The Kernel Interaction Trick: Fast Bayesian Discovery of Pairwise Interactions in High Dimensions
The Kernel Interaction Trick: Fast Bayesian Discovery of Pairwise Interactions in High Dimensions
复制标题
DOI：
--
复制
发表时间：
2019-05
期刊：
影响因子：
--
通讯作者：
Raj Agrawal;Jonathan Huggins;Brian L. Trippe;Tamara Broderick
中科院分区：
文献类型：
--
作者：
Raj Agrawal;Jonathan Huggins;Brian L. Trippe;Tamara Broderick
发现感兴趣的反应的相互作用效应是生物学、医学、经济学和许多其他科学学科面临的基本问题。在理论上，贝叶斯方法发现成对相互作用享有许多好处，如一致的不确定性量化，结合背景知识的能力，和理想的收缩性能。然而，在实践中，贝叶斯方法往往是计算上棘手的，即使是中等规模的问题。我们的关键见解是，许多实际感兴趣的分层模型都允许特定的高斯过程（GP）表示; GP允许我们用O（p）核超参数向量而不是O（p^2）相互作用和主效应来捕获后验。使用隐式表示，我们可以在模型超参数上运行马尔可夫链蒙特卡罗（MCMC），每次迭代在p中记忆线性。我们专注于稀疏诱导模型，并在具有各种协变量行为的数据集上显示，我们的方法：（1）在MCMC的朴素应用程序上减少了几个数量级的运行时间，（2）相对于最先进的基于LASSO的方法，提供了较低的I型和II型错误，（3）相对于现有的贝叶斯和基于LASSO的方法，在高维中提供了改进的计算缩放。
Discovering interaction effects on a response of interest is a fundamental problem faced in biology, medicine, economics, and many other scientific disciplines. In theory, Bayesian methods for discovering pairwise interactions enjoy many benefits such as coherent uncertainty quantification, the ability to incorporate background knowledge, and desirable shrinkage properties. In practice, however, Bayesian methods are often computationally intractable for even moderate-dimensional problems. Our key insight is that many hierarchical models of practical interest admit a particular Gaussian process (GP) representation; the GP allows us to capture the posterior with a vector of O(p) kernel hyper-parameters rather than O(p^2) interactions and main effects. With the implicit representation, we can run Markov chain Monte Carlo (MCMC) over model hyper-parameters in time and memory linear in p per iteration. We focus on sparsity-inducing models and show on datasets with a variety of covariate behaviors that our method: (1) reduces runtime by orders of magnitude over naive applications of MCMC, (2) provides lower Type I and Type II error relative to state-of-the-art LASSO-based approaches, and (3) offers improved computational scaling in high dimensions relative to existing Bayesian and LASSO-based approaches.