AdaAnn: Adaptive Annealing Scheduler for Probability Density Approximation
AdaAnn: Adaptive Annealing Scheduler for Probability Density Approximation
复制标题
DOI：
10.1615/int.j.uncertaintyquantification.2022043110
复制
发表时间：
2022-02
期刊：
影响因子：
--
通讯作者：
Emma R. Cobian;J. Hauenstein;Fang Liu;D. Schiavazzi
中科院分区：
文献类型：
--
作者：
Emma R. Cobian;J. Hauenstein;Fang Liu;D. Schiavazzi
近似概率分布可能是一项具有挑战性的任务，特别是当它们在高几何复杂性区域上得到支持或呈现多种模式时。退火可用于促进这一任务，其通常与逆温度中的恒定先验选择增量相结合。然而，使用恒定增量限制了计算效率，因为无法适应退火密度的平滑变化可以用较大增量同样好地处理的情况。我们介绍AdaAnn，一个自适应退火调度程序，自动调整的基础上的预期变化的Kullback-Leibler分歧的退火温度足够接近的两个分布之间的温度增量。AdaAnn易于实现，可以集成到现有的采样方法中，例如变分推理和马尔可夫链蒙特卡罗的归一化流。我们证明了计算效率的AdaAnn调度变分推理与规范化流的一些例子，包括密度近似和参数估计的动力系统。
Approximating probability distributions can be a challenging task, particularly when they are supported over regions of high geometrical complexity or exhibit multiple modes. Annealing can be used to facilitate this task which is often combined with constant a priori selected increments in inverse temperature. However, using constant increments limit the computational efficiency due to the inability to adapt to situations where smooth changes in the annealed density could be handled equally well with larger increments. We introduce AdaAnn, an adaptive annealing scheduler that automatically adjusts the temperature increments based on the expected change in the Kullback-Leibler divergence between two distributions with a sufficiently close annealing temperature. AdaAnn is easy to implement and can be integrated into existing sampling approaches such as normalizing flows for variational inference and Markov chain Monte Carlo. We demonstrate the computational efficiency of the AdaAnn scheduler for variational inference with normalizing flows on a number of examples, including density approximation and parameter estimation for dynamical systems.