Collaborative Research: Inference and Decentralized Computing for Quantile Regression and Other Non-Smooth Methods

近年来，统计分析已经从小规模或中等规模的数据环境过渡到涉及并行和分布式计算平台上的海量数据的世界。然而，这样的过渡提出了显着的统计和计算的挑战，许多重要的方法与非光滑损失函数。作为一个代表性的例子，分位数回归方法是统计学和计量经济学中许多先进方法的基石，经常用于对金融数据和医疗数据进行建模。计算的不确定性使得分位数回归在统计学习工具包的各个分支中不太有利。该项目旨在为大规模学习开发一个统一的框架，以解决上述问题。开发的方法将被应用于分析复杂的生物医学数据审查或隐私协议和大规模的公共卫生数据。研究生和本科生都将通过参与该项目的研究接受培训，从开发新的方法和理论到不同平台下的开源软件。 主要研究人员将使用统计，优化和概率的工具组合来开发统一的卷积平滑框架，并为一类具有不可微损失的统计方法建立严格的理论和算法基础，以分位数回归和支持向量机为代表。前者对于理解依赖路径和通过标准条件均值回归分析不可挽回的异质效应是不可或缺的。然而，大多数现有的分位数回归计算方法都是基于通用算法，当变量数量很大时，这些算法在大规模机器学习应用中不可扩展。卷积平滑允许快速校准的基于梯度的算法，而不损害估计的质量，因此在统计精度和计算精度之间提供了平衡的权衡。它还扩展了分位数回归的适用性，从低维到高维，完全观察到部分观察的样本，以及线性到非线性结构，在现代大数据分析中。本计画的第一部分将集中于三个统计问题：（a）高维稀疏分位数回归，（B）大规模删失分位数回归，以及（c）具降阶M-估计的稳健回归。第二部分的研究重点是在两种现代数据类型下为具有非光滑损失函数的方法开发高效的分散算法：（i）并行和分布式数据，以及（ii）在线流数据。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Recent years have witnessed the transition of statistical analysis from a small- or moderate-scale data environment to a world involving massive data on parallel and distributed computing platforms. However, such a transition poses significant statistical and computational challenges for many important methods with non-smooth loss functions. As a representative example, quantile regression methods are building blocks for many advanced methods in statistics and econometrics and are frequently used to model financial data and medical data. The computational inflexibility makes quantile regression less favorable among various branches of the statistical learning tool kit. The project aims to develop a unified framework for large-scale learning with non-smooth loss functions to address the aforementioned problems. The developed methods will be applied to analyze complex biomedical data subject to censoring or privacy protocol and large-scale public health data. Both graduate and undergraduate students will receive training through research involvement in the project, ranging from developing new methods and theory to open-source software under different platforms. The principal investigators will use a combination of tools from statistics, optimization, and probability to develop a unified convolution smoothing framework and establish rigorous theoretical and algorithmic foundations for a class of statistical methods with non-differentiable loss, typified by quantile regression and support vector machine. The former is indispensable for understanding pathways of dependence and heterogeneous effects irretrievable through standard conditional mean regression analysis. However, most existing computational methods for quantile regression are based on generic algorithms, which are not scalable in large-scale machine learning applications when the number of variables is large. Convolution smoothing admits fast calibrated gradient-based algorithms without compromising the estimates' quality, therefore offering a balanced trade-off between statistical accuracy and computational precision. It also extends the applicability of quantile regression, from low to high dimensions, fully to partially observed samples, and linear to nonlinear structures, in modern big data analytics. The first part of the project will focus on three statistical problems: (a) high-dimensional sparse quantile regression, (b) large-scale censored quantile regression, and (c) robust regression with redescending M-estimation. The second part of the research focuses on developing efficient decentralized algorithms for methods with non-smooth loss functions under two modern data types: (i) parallel and distributed data, and (ii) online streaming data.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.