Cross-Validation With Confidence
Cross-Validation With Confidence
复制标题
DOI：
10.1080/01621459.2019.1672556
复制
发表时间：
2019-10-30
影响因子：
3.7
通讯作者：
Lei, Jing
中科院分区：
文献类型：
--
作者：
Lei, Jing
交叉验证是统计学和机器学习中最流行的模型和调优参数选择方法之一。传统的交叉验证方法虽然具有广泛的适用性，但由于忽略了测试样本的不确定度，往往会出现过拟合的情况。我们开发了一种基于交叉验证的新型统计原则推理工具，该工具考虑了测试样本中的不确定性。该方法输出一组高度竞争的候选模型，其中包含有保证概率的最优模型。因此，我们的方法可以在经典线性回归设置中实现一致的变量选择，而现有的交叉验证方法需要非常规的分割比率。当用于调整参数选择时，该方法可以在预测精度和模型可解释性之间提供替代权衡，而不是现有的交叉验证变体。通过仿真和实际数据实例验证了该方法的有效性。本文的补充材料可以在网上找到。
Cross-validation is one of the most popular model and tuning parameter selection methods in statistics and machine learning. Despite its wide applicability, traditional cross-validation methods tend to overfit, due to the ignorance of the uncertainty in the testing sample. We develop a novel statistically principled inference tool based on cross-validation that takes into account the uncertainty in the testing sample. This method outputs a set of highly competitive candidate models containing the optimal one with guaranteed probability. As a consequence, our method can achieve consistent variable selection in a classical linear regression setting, for which existing cross-validation methods require unconventional split ratios. When used for tuning parameter selection, the method can provide an alternative trade-off between prediction accuracy and model interpretability than existing variants of cross-validation. We demonstrate the performance of the proposed method in several simulated and real data examples. Supplemental materials for this article can be found online.