Sampling Sparse Representations with Randomized Measurement Langevin Dynamics
Sampling Sparse Representations with Randomized Measurement Langevin Dynamics
复制标题
DOI：
10.1145/3427585
复制
发表时间：
2021-02
期刊：
影响因子：
--
通讯作者：
Kafeng Wang;Haoyi Xiong;Jiang Bian;Zhanxing Zhu;Qian Gao;Zhishan Guo;Chengzhong Xu;Jun Huan;D. Dou
中科院分区：
文献类型：
--
作者：
Kafeng Wang;Haoyi Xiong;Jiang Bian;Zhanxing Zhu;Qian Gao;Zhishan Guo;Chengzhong Xu;Jun Huan;D. Dou
随机梯度朗之万动力学（SGLD）已被广泛用于贝叶斯抽样从某些概率分布，结合导数的对数后验。通过对对数后验分布的导数估计，SGLD方法通过执行作为恒温器动态来从分布生成样本，该恒温器动态以一定可控的扰动遍历对数后验分布的梯度流。即使当密度未知时，现有的解决方案仍然可以首先从给定的数据集学习核密度模型，然后使用SGLD在核密度导数上产生新的样本。在这项工作中，而不是从核空间探索新的样本，一种新的SGLD采样器，即随机测量朗之万动力学（RMLD）提出了从给定数据集的谱域采样高维稀疏表示。具体来说，给定一个随机测量矩阵的稀疏编码，RMLD首先推导出一个新的可能性评估的概率分布的损失函数的LASSO，然后从高维分布的样本使用随机Langevin动力学与对数似然和Metropolis-Hastings采样的导数。此外，可以使用采样的高维向量和测量矩阵来重新生成低维测量空间中的新样本。算法分析表明，RMLD确实将给定的数据集投影到具有拉普拉斯先验的高维高斯分布中，然后通过对该分布进行SGLD，从数据集中提取新的稀疏表示。已经进行了大量的实验，以评估所提出的算法使用真实世界的数据集。在三个实际应用中的性能比较表明，RMLD的性能优于基线方法的上级。
Stochastic Gradient Langevin Dynamics (SGLD) have been widely used for Bayesian sampling from certain probability distributions, incorporating derivatives of the log-posterior. With the derivative evaluation of the log-posterior distribution, SGLD methods generate samples from the distribution through performing as a thermostats dynamics that traverses over gradient flows of the log-posterior with certainly controllable perturbation. Even when the density is not known, existing solutions still can first learn the kernel density models from the given datasets, then produce new samples using the SGLD over the kernel density derivatives. In this work, instead of exploring new samples from kernel spaces, a novel SGLD sampler, namely, Randomized Measurement Langevin Dynamics (RMLD) is proposed to sample the high-dimensional sparse representations from the spectral domain of a given dataset. Specifically, given a random measurement matrix for sparse coding, RMLD first derives a novel likelihood evaluator of the probability distribution from the loss function of LASSO, then samples from the high-dimensional distribution using stochastic Langevin dynamics with derivatives of the logarithm likelihood and Metropolis–Hastings sampling. In addition, new samples in low-dimensional measuring spaces can be regenerated using the sampled high-dimensional vectors and the measurement matrix. The algorithm analysis shows that RMLD indeed projects a given dataset into a high-dimensional Gaussian distribution with Laplacian prior, then draw new sparse representation from the dataset through performing SGLD over the distribution. Extensive experiments have been conducted to evaluate the proposed algorithm using real-world datasets. The performance comparisons on three real-world applications demonstrate the superior performance of RMLD beyond baseline methods.