A simple method to improve principal components regression
A simple method to improve principal components regression
复制标题
作者：
Lang, Wenjun;Zou, Hui
主成分回归（PCR）是一种众所周知的方法，以实现降维，并经常改善预测比普通最小二乘。传统的PCR保留方差大的主成分，丢弃方差小的主成分。当响应变量与方差较小的主成分相关时，此操作很容易导致预测效果不佳。在这项工作中，我们提出了一个简单的补救措施，称为响应引导的主成分回归（RgPCR），选择主成分回归的基础上方差的主成分和拟合优度的响应。RgPCR很容易实现，无需使用任何优化，并且自然适用于低维和高维数据。我们推导出一个C-p型统计量，用于选择RgPCR中的调谐参数。在我们的数值实验中，RgPCR表现出良好的性能。
Principal components regression (PCR) is a well-known method to achieve dimension reduction and often improved prediction over the ordinary least squares. The conventional PCR retains the principal components with large variance and discards those with smaller variance. This operation can easily lead to poor prediction when the response variable is related to principal components with small variance. In this work, we propose a simple remedy named response-guided principal components regression (RgPCR) that selects principal components for regression based on both the variance of principal components and the goodness of fit to the response. RgPCR is easy to implement without using any optimization and works naturally for both low dimensional and high dimensional data. We derive a C-p type statistic for selecting the tuning parameter in RgPCR. In our numerical experiments, RgPCR is shown to enjoy promising performance.