High-Dimensional Challenges in Statistical Machine Learning: Theory, Models and Algorithms

统计机器学习的高维挑战：理论、模型和算法

• 批准号: 0605165
• 负责人: Bin Yu
• 金额: $ 45万
• 依托单位: University of California-Berkeley
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2006
• 资助国家: 美国
• 起止时间: 2006-08-15 至 2010-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0605165&HistoricalAwards=false
• 关键词: Dimensional; Challenges; Statistical; Machine; Learning

TECHNICAL SUMMARY: This research proposal consists of four closely related research thrusts, all centered around the common goal of an integrated treatment of statistical and computational issues in dealing with high-dimensional data sets arising in information technology (IT). The first two research thrusts focus on fundamental issues that arise in the design of penalty-based and other algorithmic methods for regularization. Key open problems to be addressed include the link between regularization methods and sparsity, consistency and other theoretical issues, as well as structured regularization methods for model selection. Sparse models are desirable both for scientific reasons including interpretability, and for computational reasons, such as the efficiency of performing classification or regression. The third research thrust focuses on problems of statistical inference in decentralized settings, which are of increasing importance for a broad variety of IT applications such as wireless sensor networks, computer server ``farms'', traffic monitoring systems. Designing suitable data compression schemes is the key challenge. On one hand, these schemes should respect the decentralization requirements imposed by the system (e.g., due to limited power or bandwidth of communicating data); on the other hand, they should also be (near)-optimal with respect to a statistical criterion of merit (e.g., Bayes error for a classification task; MSE for a regression or smoothing problem). The fourth project addresses statistical issues centered around the use of Markov random fields, widely-used for modeling large collections of interacting random variables, and associated variational methods for approximating moments and likelihoods in such models.BROAD SUMMARY: The field of statistical machine learning is motivated by a broad range of problems in the information sciences, among them remote sensing, data mining and compression, and statistical signal processing. Its applications range from homeland security (e.g., detecting anomalous patterns in large data sets) to environmental monitoring and assessment (e.g., estimating changes in Arctic ice). A challenging aspect to such applications is that data sets tend to be complex, massive (frequently measured in terabytes of data), and rich in terms of possible features (hundreds of thousands to millions). These characteristics presents fundamental challenges in the design and application of statistical models and algorithms for testinghypotheses and performing estimation. Whereas classical statistical methods are designed separately from computational considerations, dealing effectively with extremely high-dimensional data sets requires that computational issues be addressed in a more integrated manner during the design and testing of statistical models; and that issues of over-fitting and regularization, while always statistically relevant, become of paramount importance.

技术概要：该研究提案包括四个密切相关的研究重点，都围绕着一个共同的目标，即在处理信息技术（IT）中出现的高维数据集时，综合处理统计和计算问题。 前两个研究重点集中在基于惩罚和其他算法的正则化方法的设计中出现的基本问题。 需要解决的关键开放问题包括正则化方法与稀疏性、一致性和其他理论问题之间的联系，以及用于模型选择的结构化正则化方法。 稀疏模型是理想的，既有科学原因，包括可解释性，也有计算原因，如执行分类或回归的效率。第三个研究重点集中在分散的设置，这是越来越重要的各种IT应用，如无线传感器网络，计算机服务器“农场”，交通监控系统的统计推断的问题。 设计合适的数据压缩方案是关键的挑战。一方面，这些计划应尊重系统所施加的权力下放要求（例如，由于通信数据的有限功率或带宽）;另一方面，它们还应当相对于统计的优值标准是（接近）最优的（例如，分类任务的贝叶斯误差;回归或平滑问题的MSE）。 第四个项目解决了围绕马尔可夫随机场的使用的统计问题，广泛用于建模大量的相互作用的随机变量，以及相关的变分方法近似的时刻和这种模型中的可能性。统计机器学习领域受到信息科学中广泛问题的推动，其中包括遥感，数据挖掘和压缩，和统计信号处理。 其应用范围从国土安全（例如，检测大数据集中的异常模式）到环境监测和评估（例如，北极冰的变化）。 这些应用程序的一个挑战性方面是数据集往往是复杂的，庞大的（通常以TB的数据来衡量），并且在可能的特征方面丰富（数十万到数百万）。这些特点提出了基本的挑战，在设计和应用的统计模型和算法来测试hypotheses和执行估计。 虽然经典的统计方法是从计算考虑分开设计的，但有效处理极高维数据集需要在统计模型的设计和测试过程中以更综合的方式解决计算问题;而过度拟合和正则化问题虽然总是与统计相关，但变得至关重要。