Grassmann Stein Variational Gradient Descent
Grassmann Stein Variational Gradient Descent
复制标题
DOI：
--
复制
发表时间：
2022-02
期刊：
影响因子：
--
通讯作者：
Xingtu Liu;Harrison Zhu;Jean-Francois Ton;George Wynne;A. Duncan
中科院分区：
文献类型：
--
作者：
Xingtu Liu;Harrison Zhu;Jean-Francois Ton;George Wynne;A. Duncan
Stein变分梯度下降（SVGD）是一种确定性粒子推理算法，它提供了马尔科夫链蒙特卡罗算法的有效替代方案。然而，当目标分布的维数较高时，发现SVGD存在方差低估的问题。最近的发展主张将分数函数和数据投影到实线上以回避这个问题，尽管这可能会严重高估认知（模型）的不确定性。在这项工作中，我们提出了Grassmann Stein变分梯度下降（GSVGD）作为一种替代方法，它允许在任意维子空间上进行投影。与其他依赖降维的SVGD变体相比，GSVGD同时更新分数函数和数据的投影，并通过寻找有利子空间的耦合grassmann值扩散过程确定最优投影。我们的理论和实验结果都表明，GSVGD在具有内在低维结构的高维问题中具有有效的状态空间探索。
Stein variational gradient descent (SVGD) is a deterministic particle inference algorithm that provides an efficient alternative to Markov chain Monte Carlo. However, SVGD has been found to suffer from variance underestimation when the dimensionality of the target distribution is high. Recent developments have advocated projecting both the score function and the data onto real lines to sidestep this issue, although this can severely overestimate the epistemic (model) uncertainty. In this work, we propose Grassmann Stein variational gradient descent (GSVGD) as an alternative approach, which permits projections onto arbitrary dimensional subspaces. Compared with other variants of SVGD that rely on dimensionality reduction, GSVGD updates the projectors simultaneously for the score function and the data, and the optimal projectors are determined through a coupled Grassmann-valued diffusion process which explores favourable subspaces. Both our theoretical and experimental results suggest that GSVGD enjoys efficient state-space exploration in high-dimensional problems that have an intrinsic low-dimensional structure.