Fundamental limits for rank-one matrix estimation with groupwise heteroskedasticity
Fundamental limits for rank-one matrix estimation with groupwise heteroskedasticity
复制标题
DOI：
--
复制
发表时间：
2021-06
期刊：
影响因子：
--
通讯作者：
Joshua K. Behne;G. Reeves
中科院分区：
文献类型：
--
作者：
Joshua K. Behne;G. Reeves
涉及高维和异质数据的低阶矩阵恢复问题在整个统计学和机器学习的应用中都会出现。这篇论文的贡献在于为这类广泛的问题确立了恢复的基本限度。特别是，我们研究了在不同噪声水平下观察到不同的矩阵块时，从高斯观测中估计秩一矩阵的问题。在块个数固定而变量个数趋于无穷大的情况下，我们证明了估计矩阵和基本因子的最小均方误差的渐近精确公式。这些结果是基于从低阶矩阵张量积模型(具有均匀噪声)到具有异方差噪声的一阶模型的一种新的简化。作为我们主要结果的应用，我们表明，最近提出的基于主成分分析(PCA)的数据加权组合方法在某些情况下是最优的，但在另一些情况下是次优的。我们还提供了数值结果，将我们的渐近公式与基于加权主元分析、梯度下降和近似消息传递的方法的性能进行了比较。
Low-rank matrix recovery problems involving high-dimensional and heterogeneous data appear in applications throughout statistics and machine learning. The contribution of this paper is to establish the fundamental limits of recovery for a broad class of these problems. In particular, we study the problem of estimating a rank-one matrix from Gaussian observations where different blocks of the matrix are observed under different noise levels. In the setting where the number of blocks is fixed while the number of variables tends to infinity, we prove asymptotically exact formulas for the minimum mean-squared error in estimating both the matrix and underlying factors. These results are based on a novel reduction from the low-rank matrix tensor product model (with homogeneous noise) to a rank-one model with heteroskedastic noise. As an application of our main result, we show that recently proposed methods based on applying principal component analysis (PCA) to weighted combinations of the data are optimal in some settings but sub-optimal in others. We also provide numerical results comparing our asymptotic formulas with the performance of methods based on weighted PCA, gradient descent, and approximate message passing.