Likelihood-Free Inference with Generative Neural Networks via Scoring Rule Minimization

Likelihood-Free Inference with Generative Neural Networks via Scoring Rule Minimization
复制标题

DOI：
10.48550/arxiv.2205.15784
复制
发表时间：
2022-05
期刊：
ArXiv
影响因子：
--
通讯作者：
Lorenzo Pacchiardi;Ritabrata Dutta
Lorenzo Pacchiardi;Ritabrata Dutta
中科院分区：
其他
文献类型：
--
作者：
Lorenzo Pacchiardi;Ritabrata Dutta

文献摘要

被引文献

相似文献

贝叶斯似然自由推理方法产生后验近似的模拟器模型与棘手的可能性。近年来，许多工作训练神经网络直接逼近难处理的似然或后验。大多数建议使用归一化流，即神经网络参数化可逆映射，用于从底层基本度量转换样本;然后可以访问转换样本的概率密度，并且可以通过模拟参数-观测对的最大似然来训练归一化流。最近的一项工作[拉梅什等人，2022]用生成网络近似后验，这降低了可逆性要求，因此是一类更灵活的分布，可以扩展到高维和结构化数据。然而，生成网络只允许从参数化分布中采样;因此，Ramesh et al. [2022]遵循对抗训练的常见解决方案，其中生成网络与“批评者“网络进行最小-最大游戏。这个过程是不稳定的，可能导致一个学习的分布低估了不确定性-在极端情况下崩溃到一个点。在这里，我们建议使用通过评分规则最小化训练的生成网络来近似后验，这是一种被忽视的无对抗方法，可以实现平滑训练和更好的不确定性量化。在模拟研究中，评分规则方法在对抗框架方面具有更短的训练时间，从而产生更好的性能。
Bayesian Likelihood-Free Inference methods yield posterior approximations for simulator models with intractable likelihood. Recently, many works trained neural networks to approximate either the intractable likelihood or the posterior directly. Most proposals use normalizing flows, namely neural networks parametrizing invertible maps used to transform samples from an underlying base measure; the probability density of the transformed samples is then accessible and the normalizing flow can be trained via maximum likelihood on simulated parameter-observation pairs. A recent work [Ramesh et al., 2022] approximated instead the posterior with generative networks, which drop the invertibility requirement and are thus a more flexible class of distributions scaling to high-dimensional and structured data. However, generative networks only allow sampling from the parametrized distribution; for this reason, Ramesh et al. [2022] follows the common solution of adversarial training, where the generative network plays a min-max game against a"critic"network. This procedure is unstable and can lead to a learned distribution underestimating the uncertainty - in extreme cases collapsing to a single point. Here, we propose to approximate the posterior with generative networks trained by Scoring Rule minimization, an overlooked adversarial-free method enabling smooth training and better uncertainty quantification. In simulation studies, the Scoring Rule approach yields better performances with shorter training time with respect to the adversarial framework.