Replicated Vector Approximate Message Passing For Resampling Problem
Replicated Vector Approximate Message Passing For Resampling Problem
复制标题
DOI：
--
复制
发表时间：
2019-05
期刊：
影响因子：
--
通讯作者：
Takashi Takahashi;Y. Kabashima
中科院分区：
文献类型：
--
作者：
Takashi Takahashi;Y. Kabashima
估计量的统计性质是统计推断和集成学习中的重要参数。然而，现有的方法在计算上要求很高，因为需要对每个重新采样的数据重复进行经由数值优化/积分的估计/学习。在这项研究中，我们引入了一个计算效率高的方法来解决这样的问题：复制向量近似消息传递。这是基于统计物理学的复制方法和信息论的向量近似消息传递的精确近似推理算法的结合。该方法提供了易于处理的密度，而无需重复估计/学习，并且密度近似地提供了在实际时间内估计器矩的任意程度。在实验中，我们将所提出的方法应用于稳定性选择方法，这是常用的变量选择问题。数值结果表明，它的快速收敛性和高逼近精度的问题，涉及合成和真实世界的数据集。
Resampling techniques are widely used in statistical inference and ensemble learning, in which estimators' statistical properties are essential. However, existing methods are computationally demanding, because repetitions of estimation/learning via numerical optimization/integral for each resampled data are required. In this study, we introduce a computationally efficient method to resolve such problem: replicated vector approximate message passing. This is based on a combination of the replica method of statistical physics and an accurate approximate inference algorithm, namely the vector approximate message passing of information theory. The method provides tractable densities without repeating estimation/learning, and the densities approximately offer an arbitrary degree of the estimators' moment in practical time. In the experiment, we apply the proposed method to the stability selection method, which is commonly used in variable selection problems. The numerical results show its fast convergence and high approximation accuracy for problems involving both synthetic and real-world datasets.