Maximum Likelihood Estimation of a Low-Rank Probability Mass Tensor From Partial Observations
Maximum Likelihood Estimation of a Low-Rank Probability Mass Tensor From Partial Observations
复制标题
根据部分观测值对低秩概率质量张量进行最大似然估计
DOI：
10.1109/lsp.2019.2938663
复制
发表时间：
2019
影响因子：
3.9
通讯作者：
M. Haardt
中科院分区：
文献类型：
--
作者：
A. Yeredor;M. Haardt
我们考虑了从部分观测中估计离散随机向量(RV)的概率质量函数(PMF)的问题，即当每个观测实现中的某些元素可能丢失时。由于PMF采用多向张量的形式，在某些模型假设下，问题变得与张量因式分解密切相关。事实上，最近的研究表明，通过对固定基数大于2的子集(例如，三元组)的所有估计联合PMF应用低阶(近似)联合因式分解，可以完全恢复低阶PMF张量(在一些温和的条件下)。联合分解是基于最小二乘(LS)对估计的低阶子张量进行拟合的。在这封信中，我们采取了一种不同的估计方法，即在Kullback-Leibler发散(KLD)意义下将部分分解直接拟合到观察到的部分数据。因此，我们避免了对特定阶次张量的特定选择和直接估计的需要，因为我们内在地对所有可用的部分数据应用了适当的加权。我们证明了我们的方法基本上达到了完全PMF张量的最大似然估计(在低阶模型下)，因此具有众所周知的相合性和渐近有效性。此外，基于贝叶斯模型对低阶模型的解释，我们提出了一种基于估计最大化(EM)的方法，该方法每次迭代的计算量都很小。仿真结果表明，与子张量的最小二乘拟合法相比，本文提出的基于KLD的混合方法(结合交替方向极小化和EM)具有更好的性能。
We consider the problem of estimating the Probability Mass Function (PMF) of a discrete random vector (RV) from partial observations, namely when some elements in each observed realization may be missing. Since the PMF takes the form of a multi-way tensor, under certain model assumptions the problem becomes closely associated with tensor factorization. Indeed, in recent studies it was shown that a low-rank PMF tensor can be fully recovered (under some mild conditions) by applying a low-rank (approximate) joint factorization to all estimated joint PMFs of subsets of fixed cardinality larger than two (e.g., triplets). The joint factorization is based on a Least Squares (LS) fit to the estimated lower-order sub-tensors. In this letter we take a different estimation approach by fitting the partial factorization directly to the observed partial data in the sense of Kullback-Leibler divergence (KLD). Consequently, we avoid the need for particular selection and direct estimation of sub-tensors of a particular order, as we inherently apply proper weighting to all the available partial data. We show that our approach essentially attains the Maximum Likelihood estimate of the full PMF tensor (under the low-rank model) and therefore enjoys its well-known properties of consistency and asymptotic efficiency. In addition, based on the Bayesian model interpretation of the low-rank model, we propose an Estimation-Maximization (EM) based approach, which is computationally cheap per iteration. Simulation results demonstrate the advantages of our proposed KLD-based hybrid approach (combining alternating-directions minimization with EM) over LS fitting of sub-tensors.
学习平滑产品分布的混合：可识别性和算法
DOI：
--
发表时间：
2019
期刊：
Proceedings AISTATS
影响因子：
--
作者：
Nikos Kargas, Nicholas D.
通讯作者：
Nikos Kargas, Nicholas D.
有限字母表随机向量的张量、学习和“Kolmogorov Extension”
影响因子：
5.4
作者：
Kargas, Nikos;Sidiropoulos, Nicholas D.;Fu, Xiao
通讯作者：
Fu, Xiao