PROMETHEUS: PRObability Mass Estimation in Tensors with Hidden Elements Using Structure (Methods, Theory, and Applications)

PROMETHEUS：使用结构对具有隐藏元素的张量进行概率质量估计（方法、理论和应用）

• 批准号: 462458843
• 负责人: Professor Dr.-Ing. Martin Haardt
• 金额: --
• 依托单位: School of Electrical Engineeringl
• 依托单位国家: 德国
• 项目类别: Research Grants
• 资助国家: 德国
• 项目状态: 未结题
• 来源: https://gepris.dfg.de/gepris/projekt/462458843?language=en
• 关键词: PROMETHEUS; PRObability; Mass; Estimation; Tensors

In the PROMETHEUS project, we will devise, explore, and analyze new tools for learning the statistical behavior of discrete random vectors (RVs) from partial realizations thereof. More specifically, the joint probability mass function (PMF) of the discrete-valued elements of the RV takes the form of a multi-way tensor. Our goal is to estimate the PMF tensor from multiple realizations of the RV, mainly when in most realizations some of its elements are missing. The ability to estimate (or to learn) the PMF can be paramount in a variety of statistical inference tasks, ranging from collaborative filtering for recommender systems, or automated recruiting in hiring processes, to higher education admission systems or computer-aided diagnostics.The total number of elements in the PMF tensor grows exponentially with the dimension of the RV and can become huge relative to the number of available observations. Consequently, reliable estimation of a general PMF is practically hopeless. However, structural constraints on the estimated PMF tensor, such as a low-rank assumption (whenever justified), can reduce the number of free parameters dramatically. However, the estimation of these parameters calls for the design of properly constrained loss-functions, giving rise to non-linear and nonconvex constrained optimization problems.In recent related work, it was shown that full recovery of a complete low-rank PMF tensor using joint factorization of its sub-tensors of a fixed order is possible under mild conditions. When the sub-tensors can be consistently estimated from the available partial observations, their (approximate) joint factorization can, therefore, yield a consistent estimate of the full tensor. However, in our own recent work, we have shown that the choice of a criterion function for the approximate joint factorization affects the accuracy and the computational complexity of the estimates. We have proposed a different estimation and factorization scheme which yields the Maximum Likelihood (ML) estimate of the tensor (subject to the low-rank constraint) and therefore enjoys the asymptotic optimality of ML estimation.Based on this observation and on our accumulated experience in tensor factorization in general and in conjunction with PMF estimation in particular, we will develop and analyze novel methods for this challenging estimation (or learning) problem. We seek estimation schemes realizing trade-offs between computational complexity, estimation accuracy, robustness, and reliability. To this end, we will also explore different theoretically and practically justified succinct statistical models which help to further reduce the number of estimated parameters. Moreover, we will propose and test approaches for a data-driven determination of the model order parameters, analyze the resulting estimation accuracy, derive practical performance bounds, and evaluate these new algorithms for some selected application examples with real data.

在 PROMETHEUS 项目中，我们将设计、探索和分析新工具，用于从离散随机向量（RV）的部分实现中学习其统计行为。更具体地说，RV 的离散值元素的联合概率质量函数 (PMF) 采用多路张量的形式。我们的目标是从 RV 的多个实现中估计 PMF 张量，主要是在大多数实现中缺少某些元素时。估计（或学习）PMF 的能力在各种统计推断任务中至关重要，从推荐系统的协同过滤、招聘流程中的自动招聘，到高等教育录取系统或计算机辅助诊断。PMF 张量中的元素总数随着 RV 的维度呈指数级增长，并且相对于可用观测的数量可能会变得巨大。因此，对一般 PMF 进行可靠估计实际上是没有希望的。然而，估计 PMF 张量的结构约束，例如低秩假设（只要合理），可以显着减少自由参数的数量。然而，这些参数的估计需要设计适当约束的损失函数，从而引起非线性和非凸约束优化问题。最近的相关工作表明，在温和的条件下，使用固定阶子张量的联合分解来完全恢复完整的低秩 PMF 张量是可能的。当子张量可以从可用的部分观测值一致地估计时，它们的（近似）联合分解可以产生完整张量的一致估计。然而，在我们最近的工作中，我们已经表明，近似联合分解的标准函数的选择会影响估计的准确性和计算复杂性。我们提出了一种不同的估计和分解方案，它产生张量的最大似然（ML）估计（受低秩约束），因此享有 ML 估计的渐近最优性。基于这一观察以及我们在张量分解方面积累的经验，特别是与 PMF 估计相结合，我们将针对这一具有挑战性的估计（或学习）问题开发和分析新方法。我们寻求实现计算复杂性、估计精度、鲁棒性和可靠性之间权衡的估计方案。为此，我们还将探索不同的理论和实践合理的简洁统计模型，这有助于进一步减少估计参数的数量。此外，我们将提出并测试数据驱动确定模型阶数参数的方法，分析所得的估计精度，得出实际性能范围，并使用实际数据针对某些选定的应用示例评估这些新算法。