Bridging the Gap Between Theory and Applications: Robust and Scalable Statistical Estimation

现代数据丰富的环境中的主要挑战需要新的统计算法在现实场景和模型假设下取得成功，例如分布式环境中的估计、处理重尾数据、离群值和丢失观测的能力。首席调查员(PI)在本项目过程中将进行的研究侧重于当代统计科学面临的两个重要问题：可扩展性和稳健性。该项目的目标是促进我们对涉及(A)高维协方差矩阵估计和(B)分布式统计估计协议的统计技术的理解。所获得的结果将对从事理论和应用工作的科学家产生兴趣。本项目的一部分旨在回答与重尾分布的高维协方差矩阵估计相关的公开问题。对于被离群值破坏的数据，这种分布是一种可行的模型，这在应用程序中几乎是不可避免的场景。协方差矩阵是高维数据分析中最基本的对象之一：许多重要的统计工具，如主成分分析和回归分析，都涉及到协方差估计作为一个关键步骤。例如，主成分分析与非线性降维和多种学习技术、遗传学、计算生物学等有着惊人的联系。然而，大多数现有估计量的理论分析所依据的假设，如样本协方差矩阵的各种修改，往往是限制性的，不适用于现实世界的情景。利用随机矩阵理论的工具，PI将开发一类新的稳健估计器，这些估计器在比目前可用的替代方案弱得多的条件下，具有数值易处理、表现出良好的实际性能和强大的理论保证。具体地说，该项目的目标是设计估计器，允许在对潜在分布的弱假设下，例如仅存在低阶矩的情况下，允许紧集中在未知的“真实”协方差矩阵周围。该项目的另一部分致力于可伸缩估计的新算法，这些算法可以利用“分而治之”的方法。分治范式假设数据由多台机器组成的集群以分布式方式存储和分析：集群中的每台机器都使用自己的子样本，而不同机器之间的通信是有限的，最终结果是通过将这些分布式计算的结果拼凑在一起得到的。PI将开发一类新的分而治之的战略，并有强有力的理论证据支持。该项目将调查分布式评估策略和结果算法的健壮性之间的联系--这是大型分布式系统的一个重要特征。

Major challenges in modern data-rich environment require new statistical algorithms that succeed under realistic scenarios and model assumptions, such as estimation in the distributed setting, ability to handle heavy-tailed data, outliers, and missing observations. Research that will be performed by the Principal Investigator (PI) in the course of this project focuses on two important problems faced by contemporary statistical science: scalability and robustness. The goal of the project is to advance our understanding of statistical techniques that involve (a) high-dimension covariance matrix estimation, and (b) distributed statistical estimation protocols. Obtained results will be of interest to scientists working on theory as well as applications.One part of this project aims at answering open questions related to high-dimensional covariance matrix estimation for the heavy-tailed distributions. Such distributions serve as a viable model for data corrupted with outliers, an almost inevitable scenario in applications. Covariance matrix is one of the most fundamental objects in high-dimensional data analysis: many important statistical tools, such as Principal Component Analysis (PCA) and regression analysis, involve covariance estimation as a crucial step. For instance, PCA has striking connections to nonlinear dimension reduction and manifold learning techniques, genetics, computational biology, among many others. However, the assumptions underlying the theoretical analysis of most existing estimators, such as various modifications of the sample covariance matrix, are often restrictive and do not hold for real-world scenarios. Using tools from the random matrix theory, the PI will develop a new class of robust estimators that are numerically tractable, show good practical performance and enjoy strong theoretical guarantees under much weaker conditions than currently available alternatives. Specifically, the goal of the project is to design estimators that admit tight concentration around the unknown "true" covariance matrix under weak assumptions on the underlying distribution, such as existence of moments of only low order. Another part of this project is devoted to novel algorithms for scalable estimation that can take advantage of the "divide and conquer" approach. Divide and conquer paradigm assumes that data is stored and analyzed in a distributed way by a cluster consisting of several machines: each of the machines in a cluster works on its own sub-sample while communication among different machines is limited, and final results are obtained by piecing the outcomes of these distributed computations together. The PI will develop a class of new divide and conquer strategies supported by strong theoretical evidence. The project will investigate connections between the distributed estimation strategies and robustness of resulting algorithms -- an important characteristic of large distributed systems.