OPTIMAL SELECTION OF REDUCED RANK ESTIMATORS OF HIGH-DIMENSIONAL MATRICES
OPTIMAL SELECTION OF REDUCED RANK ESTIMATORS OF HIGH-DIMENSIONAL MATRICES
复制标题
DOI：
10.1214/11-aos876
复制
发表时间：
2011-04-01
影响因子：
4.5
通讯作者：
Wegkamp, Marten H.
中科院分区：
文献类型：
--
作者：
Bunea, Florentina;She, Yiyuan;Wegkamp, Marten H.
本文提出了一个新的准则，秩选择准则（RSC），用于选择多元响应回归模型中系数矩阵的最优降秩估计。相应的RSC估计量最小化拟合的Frobenius范数加上与降秩模型中的参数数目成比例的正则化项。RSC估计量的秩提供了系数矩阵秩的一致估计量;一般来说，我们的估计量的秩是有效秩的一致估计，我们将其定义为目标矩阵的适当大的奇异值的数量。一致性结果不仅在经典渐进状态下有效，即当响应数量n和预测变量数量p保持有界，并且观测数量m增长时，而且当n和p之一或两者增长时，一致性结果也有效。可能比m快得多。我们为估计量的均方误差建立了极大极小最优界限。我们的RSC估计的有限样本性能界表明，它实现了近似误差和惩罚项之间的最佳平衡。此外，我们的程序具有非常低的计算复杂度，线性的候选模型的数量，使其特别吸引人的大规模问题。我们将我们的估计与核范数惩罚最小二乘（NNP）估计进行了对比，对于多元回归模型，NNP估计具有固有的比RSC更高的计算复杂性。我们发现，NNP的估计性质类似的RSC，虽然在更强的条件下。然而，它不像RSC那样吝啬，我们提供了一个简单的校正NNP估计，导致一致的秩估计。我们通过广泛的模拟研究验证和说明我们的理论研究结果。
We introduce a new criterion, the Rank Selection Criterion (RSC), for selecting the optimal reduced rank estimator of the coefficient matrix in multivariate response regression models. The corresponding RSC estimator minimizes the Frobenius norm of the fit plus a regularization term proportional to the number of parameters in the reduced rank model.The rank of the RSC estimator provides a consistent estimator of the rank of the coefficient matrix; in general, the rank of our estimator is a consistent estimate of the effective rank, which we define to be the number of singular values of the target matrix that are appropriately large. The consistency results are valid not only in the classic asymptotic regime, when n, the number of responses, and p, the number of predictors, stay bounded, and m, the number of observations, grows, but also when either, or both, n and p grow, possibly much faster than m.We establish minimax optimal bounds on the mean squared errors of our estimators. Our finite sample performance bounds for the RSC estimator show that it achieves the optimal balance between the approximation error and the penalty term.Furthermore, our procedure has very low computational complexity, linear in the number of candidate models, making it particularly appealing for large scale problems. We contrast our estimator with the nuclear norm penalized least squares (NNP) estimator, which has an inherently higher computational complexity than RSC, for multivariate regression models. We show that NNP has estimation properties similar to those of RSC, albeit under stronger conditions. However, it is not as parsimonious as RSC.We offer a simple correction of the NNP estimator which leads to consistent rank estimation. We verify and illustrate our theoretical findings via an extensive simulation study.