Why your model parameter confidences might be too optimistic. Unbiased estimation of the inverse covariance matrix
Why your model parameter confidences might be too optimistic. Unbiased estimation of the inverse covariance matrix
复制标题
DOI：
10.1051/0004-6361:20066170
复制
发表时间：
2007-03-01
影响因子：
6.5
通讯作者：
Schneider, P.
中科院分区：
文献类型：
--
作者：
Hartlap, J.;Simon, P.;Schneider, P.
目标。最大似然法是获得模型拟合观测数据和相应置信区域的标准方法。我们调查可能的来源偏差的对数似然函数及其随后的分析，专注于逆协方差矩阵的估计。此外，我们研究在什么情况下估计的协方差矩阵是可逆的。我们进行蒙特-卡罗模拟研究的逆协方差矩阵的估计的行为，依赖于独立的数据集的数量和数据向量的变量的数量。我们发现协方差的最大似然估计的逆是有偏的，偏差的量取决于bin（数据向量变量）的数量p与数据集的数量n的比率。这种偏差不可避免地导致对置信区域大小的低估--在极端情况下是灾难性的。我们报告的方法，以消除这种偏见的理想情况下的高斯噪声和统计独立的数据向量。此外，我们证明了边缘化参数引入了偏置的边缘化对数似然函数。置信区域大小的度量也存在同样的问题。此外，我们给出了一个解析证明的事实，即估计的协方差矩阵是奇异的，如果p > n。
Aims. The maximum-likelihood method is the standard approach to obtain model fits to observational data and the corresponding confidence regions. We investigate possible sources of bias in the log-likelihood function and its subsequent analysis, focusing on estimators of the inverse covariance matrix. Furthermore, we study under which circumstances the estimated covariance matrix is invertible.Methods. We perform Monte-Carlo simulations to investigate the behaviour of estimators for the inverse covariance matrix, depending on the number of independent data sets and the number of variables of the data vectors.Results. We find that the inverse of the maximum-likelihood estimator of the covariance is biased, the amount of bias depending on the ratio of the number of bins (data vector variables), p, to the number of data sets, n. This bias inevitably leads to an - in extreme cases catastrophic - underestimation of the size of confidence regions. We report on a method to remove this bias for the idealised case of Gaussian noise and statistically independent data vectors. Moreover, we demonstrate that marginalisation over parameters introduces a bias into the marginalised log-likelihood function. Measures of the sizes of confidence regions suffer from the same problem. Furthermore, we give an analytic proof for the fact that the estimated covariance matrix is singular if p > n.