High-Dimensional Challenges in Statistical Machine Learning: Theory, Models and Algorithms

技术概述：本研究计划由四个密切相关的研究重点组成，所有研究重点都围绕着在处理信息技术（IT）中出现的高维数据集时综合处理统计和计算问题的共同目标。前两个研究重点是在基于惩罚和其他正则化算法的设计中出现的基本问题。要解决的关键开放问题包括正则化方法与稀疏性、一致性和其他理论问题之间的联系，以及用于模型选择的结构化正则化方法。由于科学原因（包括可解释性）和计算原因（例如执行分类或回归的效率），稀疏模型都是可取的。第三个研究重点是分散环境中的统计推断问题，这对于各种各样的IT应用（如无线传感器网络、计算机服务器“农场”、交通监控系统）越来越重要。设计合适的数据压缩方案是关键的挑战。一方面，这些方案应该尊重系统施加的去中心化要求（例如，由于通信数据的功率或带宽有限）；另一方面，它们也应该是（接近）最优的关于优点的统计标准（例如，分类任务的贝叶斯误差；回归或平滑问题的MSE）。第四个项目解决了以马尔可夫随机场的使用为中心的统计问题，马尔可夫随机场广泛用于对大量相互作用的随机变量进行建模，以及在这些模型中近似矩和可能性的相关变分方法。概论：统计机器学习领域是由信息科学中的一系列问题驱动的，其中包括遥感、数据挖掘和压缩以及统计信号处理。它的应用范围从国土安全（例如，检测大数据集中的异常模式）到环境监测和评估（例如，估计北极冰的变化）。这类应用程序面临的一个挑战是，数据集往往非常复杂、庞大（通常以tb级的数据为单位），并且可能具有丰富的特征（数十万到数百万）。这些特征在统计模型和算法的设计和应用中提出了基本的挑战，这些模型和算法用于测试假设和执行估计。尽管经典统计方法的设计与计算考虑是分开的，但有效处理极高维度的数据集需要在统计模型的设计和测试过程中以更集成的方式解决计算问题；而过度拟合和正则化的问题，虽然总是与统计相关，却变得至关重要。

TECHNICAL SUMMARY: This research proposal consists of four closely related research thrusts, all centered around the common goal of an integrated treatment of statistical and computational issues in dealing with high-dimensional data sets arising in information technology (IT). The first two research thrusts focus on fundamental issues that arise in the design of penalty-based and other algorithmic methods for regularization. Key open problems to be addressed include the link between regularization methods and sparsity, consistency and other theoretical issues, as well as structured regularization methods for model selection. Sparse models are desirable both for scientific reasons including interpretability, and for computational reasons, such as the efficiency of performing classification or regression. The third research thrust focuses on problems of statistical inference in decentralized settings, which are of increasing importance for a broad variety of IT applications such as wireless sensor networks, computer server ``farms'', traffic monitoring systems. Designing suitable data compression schemes is the key challenge. On one hand, these schemes should respect the decentralization requirements imposed by the system (e.g., due to limited power or bandwidth of communicating data); on the other hand, they should also be (near)-optimal with respect to a statistical criterion of merit (e.g., Bayes error for a classification task; MSE for a regression or smoothing problem). The fourth project addresses statistical issues centered around the use of Markov random fields, widely-used for modeling large collections of interacting random variables, and associated variational methods for approximating moments and likelihoods in such models.BROAD SUMMARY: The field of statistical machine learning is motivated by a broad range of problems in the information sciences, among them remote sensing, data mining and compression, and statistical signal processing. Its applications range from homeland security (e.g., detecting anomalous patterns in large data sets) to environmental monitoring and assessment (e.g., estimating changes in Arctic ice). A challenging aspect to such applications is that data sets tend to be complex, massive (frequently measured in terabytes of data), and rich in terms of possible features (hundreds of thousands to millions). These characteristics presents fundamental challenges in the design and application of statistical models and algorithms for testinghypotheses and performing estimation. Whereas classical statistical methods are designed separately from computational considerations, dealing effectively with extremely high-dimensional data sets requires that computational issues be addressed in a more integrated manner during the design and testing of statistical models; and that issues of over-fitting and regularization, while always statistically relevant, become of paramount importance.