Optimal Poisson Subsampling for Softmax Regression
Optimal Poisson Subsampling for Softmax Regression
复制标题
DOI：
10.1007/s11424-023-1179-z
复制
发表时间：
2023-08
影响因子：
2.1
通讯作者：
Yaqiong Yao;Jiahui Zou;Haiying Wang
中科院分区：
文献类型：
--
作者：
Yaqiong Yao;Jiahui Zou;Haiying Wang
Softmax回归，也称为多项式逻辑回归，广泛应用于各个领域，用于对多水平的协变量和分类响应之间的关系进行建模。不断增加的数据量给softmax回归中的参数估计带来了新的挑战，而最优子采样方法是解决这些问题的有效方法。然而，带有替换的最佳子采样需要同时访问所有采样概率来绘制子样本，并且所得子样本可能包含重复的观测值。在本文中，作者考虑了泊松子采样，因为它具有更高的估计精度以及在数据超出内存限制的情况下的适用性。作者推导了一般泊松子采样估计器的渐近性质，并通过在 A 和 L 最优性标准下最小化渐近方差-协方差矩阵来获得最优子采样概率。最佳二次采样概率包含来自完整数据集的未知量，因此作者提出了一种近似最佳的泊松二次采样算法，该算法包含两个采样步骤，其中第一步作为试点阶段。作者通过数值模拟和真实数据示例展示了我们的最佳泊松子采样算法的性能。
Softmax regression, which is also called multinomial logistic regression, is widely used in various fields for modeling the relationship between covariates and categorical responses with multiple levels. The increasing volumes of data bring new challenges for parameter estimation in softmax regression, and the optimal subsampling method is an effective way to solve them. However, optimal subsampling with replacement requires to access all the sampling probabilities simultaneously to draw a subsample, and the resultant subsample could contain duplicate observations. In this paper, the authors consider Poisson subsampling for its higher estimation accuracy and applicability in the scenario that the data exceed the memory limit. The authors derive the asymptotic properties of the general Poisson subsampling estimator and obtain optimal subsampling probabilities by minimizing the asymptotic variance-covariance matrix under both A- and L- optimality criteria. The optimal subsampling probabilities contain unknown quantities from the full dataset, so the authors suggest an approximately optimal Poisson subsampling algorithm which contains two sampling steps, with the first step as a pilot phase. The authors demonstrate the performance of our optimal Poisson subsampling algorithm through numerical simulations and real data examples.