粒子变分推断中的梯度流问题

粒子变分推断方法是近年来诞生的一类高效贝叶斯后验推断方法。这类方法使用多个粒子模拟概率空间上的梯度流，迭代式地更新粒子位置，使得粒子的经验分布不断趋近目标后验分布。然而，当前使用的梯度流还存在如下问题：1)从表达能力来看，目前采用的梯度流均无法导出合理有效的权重更新方式，现有粒子变分推断方法只更新粒子位置而固定粒子权重不变，极大地限制了经验分布的近似能力；2) 从采样空间来看，当前使用的梯度流均定义在全空间上，无法有效处理限制域上的采样问题；3)从近似有效性观察，根据当前梯度流导出的方法均为确定性方法，使用中可能会出现所有粒子都坍缩到同一个局部众数上的现象。因此，本研究将聚焦粒子变分推断方法中的梯度流问题，通过探索不同距离空间上梯度流的特性针对性地扩展梯度流，基于此构建新的兼容不同平滑操作的高效粒子变分推断方法框架，系统性的解决现有粒子变分推断方法中的权重固定、受限采样、粒子坍缩等问题。

The recently developed Particle-based Variational Inference (ParVI) methods is a kind of efficient Bayesian inference method designed for posterior inference of complex Bayesian models. This type of methods utilizes the empirical distribution of M particles to simulate a gradient flow in the probability space, and update the particle positions iteratively so that the empirical distribution of particles is driven towards the target posterior distribution. However, the gradient flow used in the current method still has the following problems. 1)For the expressiveness of the variational distribution, existing gradient flow is not compatible with weight adjustment, and existing ParVIs have a common fixed weight restriction. Actually, existing methods all keep the particles' weights fixed during the whole procedure, and only update the positions of particles, which severely confines the empirical distribution's approximation ability, especially when the particle number is limited. 2) From the perspective of sampling domain, the currently used Wasserstein Gradient Flow is defined in the whole space, which is not suitable for a given target distribution with constrained domain. 3) Existing ParVIs all update particles in a deterministic way, restricted by the formation of existing gradient flow. Although these methods obtain asymptotic accuracy property in the mean-field limit, they have an intractable pitfall that particles tend to collapse under certain condition due to their deterministic-update fashion. Therefore, this project will focus on the Gradient Flow problem in ParVI research field, aiming to further improve the effectiveness of ParVI and its usability in complex scenarios. Based on existing research results, we construct effective frameworks for ParVIs which is compatible with different smoothing operations, and systematically solve certain open problems in ParVI field, such as fixed-weight, restricted sampling, and particle collapsing by extending current Gradient Flow with different characteristics.