Generative Multi-purpose Sampler for Weighted M-estimation
Generative Multi-purpose Sampler for Weighted M-estimation
复制标题
DOI：
10.1080/10618600.2023.2292668
复制
发表时间：
2020-06
影响因子：
2.4
通讯作者：
Minsuk Shin;Shijie Wang;Jun S. Liu
中科院分区：
文献类型：
--
作者：
Minsuk Shin;Shijie Wang;Jun S. Liu
为了克服各种数据扰动过程（如自举和交叉验证）的计算瓶颈，我们提出了生成式多用途采样器（GMS），它构造了一个生成器函数，从一组给定的权重和调谐参数中产生加权m估计的解。GMS通过单个优化实现，而无需重复评估加权损失的最小值，因此能够显着减少计算时间。我们证明了GMS框架能够实现在传统框架中不可实现的各种统计过程，例如迭代自举、惩罚似然的自举交叉验证、具有非参数最大似然的自举经验贝叶斯等。为了构造一个计算效率高的生成器函数，我们还提出了一种新的神经网络形式，称为\emph{权乘多层感知器}，以实现快速收敛。数值结果表明，与传统的神经网络结构相比，新的神经网络结构具有几个数量级的速度优势。提供了一个名为GMS的R包，它在Pytorch下运行以实现所建议的方法，并允许用户提供自定义的损失函数来定制他们自己感兴趣的模型。
To overcome the computational bottleneck of various data perturbation procedures such as the bootstrap and cross validations, we propose the Generative Multiple-purpose Sampler (GMS), which constructs a generator function to produce solutions of weighted M-estimators from a set of given weights and tuning parameters. The GMS is implemented by a single optimization without having to repeatedly evaluate the minimizers of weighted losses, and is thus capable of significantly reducing the computational time. We demonstrate that the GMS framework enables the implementation of various statistical procedures that would be unfeasible in a conventional framework, such as the iterated bootstrap, bootstrapped cross-validation for penalized likelihood, bootstrapped empirical Bayes with nonparametric maximum likelihood, etc. To construct a computationally efficient generator function, we also propose a novel form of neural network called the \emph{weight multiplicative multilayer perceptron} to achieve fast convergence. Our numerical results demonstrate that the new neural network structure enjoys a few orders of magnitude speed advantage in comparison to the conventional one. An R package called GMS is provided, which runs under Pytorch to implement the proposed methods and allows the user to provide a customized loss function to tailor to their own models of interest.