Putting nonnegative matrix factorization to the test: a tutorial derivation of pertinent cramer—rao bounds and performance benchmarking
Putting nonnegative matrix factorization to the test: a tutorial derivation of pertinent cramer—rao bounds and performance benchmarking
复制标题
DOI：
10.1109/msp.2013.2296172
复制
发表时间：
2014-04
影响因子：
14.9
通讯作者：
Kejun Huang;N. Sidiropoulos
中科院分区：
文献类型：
--
作者：
Kejun Huang;N. Sidiropoulos
非负矩阵分解（NMF）是一个广泛应用的有用工具，从信号分离到计算机视觉和机器学习。NMF是一个难（NP-hard）计算问题，多年来已经开发了各种近似解。鉴于对NMF及其应用的广泛兴趣，可能令人惊讶的是，有关非负潜在因素估计准确性的克莱默-拉奥下限（CRLB）尚未在文献中得到解决。事后看来，一个原因可能是所需的计算比平常更微妙：问题涉及必须处理的约束和模糊性，而且Fisher信息矩阵总是单一的。我们使用最新的CRLB工具，为对称NMF和非对称NMF提供了一个简明的CRLB推导教程，这应该对相关因子分析问题的类似推导有广泛的兴趣。我们说明了这些边界相对于模型参数的行为，并将一些最好的NMF算法用于彼此和CRLB的测试。结果有助于阐明NMF算法的当前状态，并且它们令人放心，因为在相对稀疏和低秩的场景中，与最优性的差距很小。
Nonnegative matrix factorization (NMF) is a useful tool in a broad range of applications, from signal separation to computer vision and machine learning. NMF is a hard (NP-hard) computational problem for which various approximate solutions have been developed over the years. Given the widespread interest in NMF and its applications, it is perhaps surprising that the pertinent Cramer-Rao lower bound (CRLB) on the accuracy of the nonnegative latent factor estimates has not been worked out in the literature. In hindsight, one reason may be that the required computations are more subtle than usual: the problem involves constraints and ambiguities that must be dealt with, and the Fisher information matrix is always singular. We provide a concise tutorial derivation of the CRLB for both symmetric NMF and asymmetric NMF, using the latest CRLB tools, which should be of broad interest for analogous derivations in related factor analysis problems. We illustrate the behavior of these bounds with respect to model parameters and put some of the best NMF algorithms to the test against one another and the CRLB. The results help illuminate what can be expected from the current state of art in NMF algorithms, and they are reassuring in that the gap to optimality is small in relatively sparse and low rank scenarios.