Sufficient ensemble size for random matrix theory-based handling of singular covariance matrices
Sufficient ensemble size for random matrix theory-based handling of singular covariance matrices
复制标题
DOI：
10.1142/s0219530520400072
复制
发表时间：
2020-04
影响因子：
2.2
通讯作者：
A. Kabán
中科院分区：
文献类型：
--
作者：
A. Kabán
奇异协方差矩阵在机器学习和优化问题中经常遇到，最常见的是由于数据的高维性和样本量不足。在众多的正则化方法中，我们在这里重点介绍一种相对较新的随机矩阵理论方法，其思想是通过对其随机投影的期望来创建奇异协方差矩阵及其逆矩阵的良好近似。我们感兴趣的错误的Monte Carlo实现这种方法，它允许在实践中的低维后续并行处理。我们发现[公式：随机投影，其中[公式：见正文]是原始矩阵的大小，足以使Monte Carlo误差变得可忽略不计，在期望谱范数差的意义下，对于协方差和逆协方差近似，在后一种情况下，在温和的假设下。
Singular covariance matrices are frequently encountered in both machine learning and optimization problems, most commonly due to high dimensionality of data and insufficient sample sizes. Among many methods of regularization, here we focus on a relatively recent random matrix-theoretic approach, the idea of which is to create well-conditioned approximations of a singular covariance matrix and its inverse by taking the expectation of its random projections. We are interested in the error of a Monte Carlo implementation of this approach, which allows subsequent parallel processing in low dimensions in practice. We find that [Formula: see text] random projections, where [Formula: see text] is the size of the original matrix, are sufficient for the Monte Carlo error to become negligible, in the sense of expected spectral norm difference, for both covariance and inverse covariance approximation, in the latter case under mild assumptions.