Matrix estimation under rank constraints for complete and incomplete noisy data

这个建议的中心目标是：（a）提供的方法，估计未知的秩从完全和不完全观察到的噪声矩阵，使用秩正则化风险最小化和（B）建立新的预言型风险界的矩阵估计和秩估计，在最小的假设。从观测到的噪声矩阵中恢复潜在目标矩阵的问题的困难在于独立参数的数量相对于观测的数量是大的。 特别注意的是多变量响应回归模型。低秩假设下的矩阵估计和稀疏假设下的一般回归模型估计之间有一个有趣的相似之处，但矩阵模型提出了不同的数学和计算挑战。以矩阵格式排列的高维数据在许多科学学科中越来越常见，如遗传学，医学成像，工程学，心理学和神经科学。由于噪声的存在，包含这些区域中的观测数据的矩阵往往具有高秩，但是数据基础的信号矩阵可能具有显著较低的秩。在任何推理过程中忽视这一点可能会导致目标的恢复不佳，对结果的解释产生严重影响。必须以尽可能高的精度恢复的目标包括：背景下的人脸，与疾病相关的基因集合，与认知过程相关的大脑结构，仅举几个例子。与这些数据的分析相关的一些挑战可以通过秩约束下的矩阵估计问题的方法和理论研究来解决。第二个问题，这是相当困难的，是执行相同的任务时，只有部分观察到的噪声矩阵是可用的。对这两个问题的系统调查是本建议的重点。这些技术的有用性将立即传播给科学界，将其应用于从艾滋病毒对大脑结构和功能的影响的研究中获得的数据。将在网上以易于实施的形式提供实施所制定方法的自由软件。

The central goals of this proposal are:(a) to provide methods for the estimation of matrices of unknown rank from both completely and incompletely observed noisy matrices, using rank regularized risk minimization and (b) to establish novel oracle type risk bounds for the matrix estimates and the rank estimates, under minimal assumptions. The difficulty of the problem of recovering the underlying target matrix from an observed noisy matrix is that the number of independent parameters is large relative to the number of observations. Special attention is given to multivariate response regression models. There is an interesting resemblance between matrix estimation under low rank assumptions and estimation in general regression models under sparsity assumptions, but matrix models pose different mathematical and computational challenges.High dimensional data arranged in matrix format are increasingly common in many scientific disciplines such as genetics, medical imaging, engineering, psychology and neuroscience. The matrices containing observed data in these areas tend to have high rank due to the presence of noise, but the signal matrix underlying the data may have significantly lower rank. Ignoring this in any inferential procedure may lead to poor recovery of the target, with severe repercussions on the interpretation of the results. Instances of targets that must be recovered with the highest possible precision include: faces against background, ensembles of genes that are associated with a disease, brain structures associated with cognitive processes, to name just a few example. Some of the challenges associated with the analysis of such data can be met via the methodological and theoretical study of the problem of matrix estimation under rank constraints. A second problem, which is substantially more difficult, is to perform the same task when only partially observed noisy matrices are available. Systematic investigation of these two problems is the focus of this proposal. The usefulness of these techniques will be immediately disseminated to the scientific community by applying them to data obtained from a study of the effects of HIV on brain structure and functions. Free software that implements the developed methodology will be made available on the web in a readily implementable form.