The Sparse Matrix Transform for Covariance Estimation and Analysis of High Dimensional Signals
The Sparse Matrix Transform for Covariance Estimation and Analysis of High Dimensional Signals
复制标题
DOI：
10.1109/tip.2010.2071390
复制
发表时间：
2011-03-01
影响因子：
10.6
通讯作者：
Bouman, Charles A.
中科院分区：
文献类型：
--
作者：
Cao, Guangzhi;Bachega, Leonardo R.;Bouman, Charles A.
高维信号的协方差估计是统计信号分析和机器学习中的经典难题。在本文中，我们提出了一个最大似然（ML）的协方差估计方法，它采用了一种新的非线性稀疏约束。更具体地，协方差被约束为具有可以表示为稀疏矩阵变换（SMT）的特征分解。SMT由成对坐标旋转的乘积形成，称为吉文斯旋转。使用这个框架，协方差可以有效地估计使用贪婪优化的对数似然函数，和吉文斯旋转的数量可以有效地计算使用交叉验证过程。由此产生的估计量通常是正定的和良好的条件，即使当样本量是有限的。模拟数据，标准高光谱数据和人脸图像集的组合实验表明，基于SMT的协方差估计始终比传统的收缩估计和最近提出的图形套索估计的各种不同的类和样本大小更准确。新的协方差估计的一个重要属性是，它自然地产生一个快速实现的估计特征变换使用SMT表示。事实上，SMT可以被视为经典快速傅立叶变换（FFT）的推广，因为它使用“蝴蝶”来表示正交变换。然而，与FFT不同，SMT可用于一般非平稳信号的快速特征信号分析。
Covariance estimation for high dimensional signals is a classically difficult problem in statistical signal analysis and machine learning. In this paper, we propose a maximum likelihood (ML) approach to covariance estimation, which employs a novel non-linear sparsity constraint. More specifically, the covariance is constrained to have an eigen decomposition which can be represented as a sparse matrix transform (SMT). The SMT is formed by a product of pairwise coordinate rotations known as Givens rotations. Using this framework, the covariance can be efficiently estimated using greedy optimization of the log-likelihood function, and the number of Givens rotations can be efficiently computed using a cross-validation procedure. The resulting estimator is generally positive definite and well-conditioned, even when the sample size is limited. Experiments on a combination of simulated data, standard hyperspectral data, and face image sets show that the SMT-based covariance estimates are consistently more accurate than both traditional shrinkage estimates and recently proposed graphical lasso estimates for a variety of different classes and sample sizes. An important property of the new covariance estimate is that it naturally yields a fast implementation of the estimated eigen-transformation using the SMT representation. In fact, the SMT can be viewed as a generalization of the classical fast Fourier transform (FFT) in that it uses "butterflies" to represent an orthonormal transform. However, unlike the FFT, the SMT can be used for fast eigen-signal analysis of general non-stationary signals.