Variational free energy and the Laplace approximation
Variational free energy and the Laplace approximation
复制标题
DOI：
10.1016/j.neuroimage.2006.08.035
复制
发表时间：
2007-01-01
期刊：
影响因子：
5.7
通讯作者：
Penny, Will
中科院分区：
文献类型：
--
作者：
Friston, Karl J.;Mattout, Jeremie;Penny, Will
本文推导了拉普拉斯近似下的变分自由能，重点考虑了因模型参数数量增加而导致的额外模型复杂性。当使用自由能作为贝叶斯模型平均和选择中对数证据的近似时，这一点是相关的。通过在变分学习和期望最大化（EM）的更大背景下设定约束最大似然（ReML），我们展示了如何调整ReML目标函数，以便为特定模型提供对数证据的近似值。这意味着ReML可用于模型选择，特别是用于选择或比较具有不同协方差成分的模型。这在分层模型的背景下是有用的，因为它能够合理地选择先验，在简单的超先验条件下，可用于自动模型选择和相关性确定（ARD）。从基本变分原理推导ReML目标函数，揭示了变分贝叶斯、EM和ReML之间的简单关系。此外，我们表明当精度在超参数中是线性的时候，EM在形式上等同于完全变分处理。最后，我们还简要考虑了动态模型以及这些模型如何为自由能上升方案（如EM和ReML）的正则化提供信息。（c）2006爱思唯尔公司。保留所有权利。
This note derives the variational free energy under the Laplace approximation, with a focus on accounting for additional model complexity induced by increasing the number of model parameters. This is relevant when using the free energy as an approximation to the log-evidence in Bayesian model averaging and selection. By setting restricted maximum likelihood (ReML) in the larger context of variational learning and expectation maximisation (EM), we show how the ReML objective function can be adjusted to provide an approximation to the log-evidence for a particular model. This means ReML can be used for model selection, specifically to select or compare models with different covariance components. This is useful in the context of hierarchical models because it enables a principled selection of priors that, under simple hyperpriors, can be used for automatic model selection and relevance determination (ARD). Deriving the ReML objective function, from basic variational principles, discloses the simple relationships among Variational Bayes, EM and ReML. Furthermore, we show that EM is formally identical to a full variational treatment when the precisions are linear in the hyperparameters. Finally, we also consider, briefly, dynamic models and how these inform the regularisation of free energy ascent schemes, like EM and ReML. (c) 2006 Elsevier Inc. All rights reserved.