Adversarial Interpretation of Bayesian Inference
Adversarial Interpretation of Bayesian Inference
复制标题
DOI：
--
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Hisham Husain;Jeremias Knoblauch
中科院分区：
文献类型：
--
作者：
Hisham Husain;Jeremias Knoblauch
我们以Knoblauch等人（2019）倡导的以优化为中心的贝叶斯推理观点为基础。考虑贝叶斯和广义贝叶斯后验作为正则化最小化问题的解决方案，我们可以回答一个有趣的问题：如果最小化是原始问题，那么它的对偶是什么？通过推导问题的Fenchel对偶，我们证明了这个对偶对应于一个对抗性博弈：在对偶空间中，先验成为对手的成本函数，对手试图干扰标准[广义]贝叶斯推理所针对的可能性[损失]函数。这意味着类贝叶斯程序具有对抗鲁棒性——为它们的经验表现提供了另一个坚实的理论基础。我们的贡献是基础性的，适用于广泛的机器学习方法。这包括标准贝叶斯推理，广义贝叶斯和吉布斯后验（Bissiri等人，2016），以及各种其他方法，包括广义变分推理（Knoblauch等人，2019）和Wasserstein自动编码器（Tolstikhin等人，2017）。
We build on the optimization-centric view on Bayesian inference advocated by Knoblauch et al. (2019). Thinking about Bayesian and generalized Bayesian posteriors as the solutions to a regularized minimization problem allows us to answer an intriguing question: If minimization is the primal problem, then what is its dual? By deriving the Fenchel dual of the problem, we demonstrate that this dual corresponds to an adversarial game: In the dual space, the prior becomes the cost function for an adversary that seeks to perturb the likelihood [loss] function targeted by standard [generalized] Bayesian inference. This implies that Bayes-like procedures are adversarially robust— providing another firm theoretical foundation for their empirical performance. Our contributions are foundational, and apply to a wide-ranging set of Machine Learning methods. This includes standard Bayesian inference, generalized Bayesian and Gibbs posteriors (Bissiri et al., 2016), as well as a diverse set of other methods including Generalized Variational Inference (Knoblauch et al., 2019) and the Wasserstein Autoencoder (Tolstikhin et al., 2017).