Tuning variable selection procedures by adding noise
Tuning variable selection procedures by adding noise
复制标题
DOI：
10.1198/004017005000000319
复制
发表时间：
2006-05-01
期刊：
影响因子：
2.5
通讯作者：
Boos, Dennis D.
中科院分区：
文献类型：
--
作者：
Luo, Xiaohui;Stefanski, Leonard A.;Boos, Dennis D.
许多用于线性回归的变量选择方法在很大程度上依赖于控制方法性能的调整参数。例如，“进入”和“停留”在向前和向后选择中的重要性级别。然而，大多数方法不会使调优参数适应特定的数据集。我们提出了一种自适应变量选择调整参数的通用策略，该策略有效地估计了调整参数，从而避免了选择方法的过拟合和欠拟合。该策略基于这样的原理，即在向响应变量添加受控数量的附加独立噪声后，可以在误差方差估计中直接观察到过高和不足。然后运行变量选择方法。这与在测量误差文献中找到的SIMEX模拟技术有关。我们专注于前向选择，因为它的简单性和处理大量解释变量的能力。蒙特卡罗研究表明，新方法优于已有的方法。
Many variable selection methods for linear regression depend critically on tuning parameters that control the performance of the method. for example, "entry" and "stay" significance levels in forward and backward selection. However, most methods do not adapt the tuning parameters to particular datasets. We propose a general strategy for adapting variable selection tuning parameters that effectively estimates the tuning parameters so that the selection method avoids overfitting and underfitting. The strategy is based on the principle that overtitting and underfitting can be directly observed in estimates of the error variance after adding controlled amounts of additional independent noise to the response variable. then running a variable selection method. It is related to the simulation technique SIMEX found in the measurement error literature. We focus on forward selection because of its simplicity and ability to handle large numbers of explanatory variables. Monte Carlo studies show that the new method compares favorably with established methods.