Collaborative Research: Inference and Decentralized Computing for Quantile Regression and Other Non-Smooth Methods

近年来，统计分析从中小规模的数据环境过渡到并行和分布式计算平台上的海量数据世界。然而，这种转变给许多具有非光滑损失函数的重要方法带来了巨大的统计和计算挑战。作为一个有代表性的例子，分位数回归方法是统计学和计量经济学中许多先进方法的基石，经常被用于对金融数据和医学数据进行建模。计算上的不灵活使得分位数回归在统计学习工具包的不同分支中不那么受欢迎。该项目旨在制定一个具有非平稳损失函数的大规模学习的统一框架，以解决上述问题。开发的方法将用于分析受审查或隐私协议约束的复杂生物医学数据和大规模公共卫生数据。研究生和本科生都将通过参与该项目的研究接受培训，范围从开发新方法和理论到在不同平台上开发开源软件。主要研究人员将综合运用统计学、最优化和概率论的工具，建立统一的卷积平滑框架，为以分位数回归和支持向量机为代表的一类具有不可微损失的统计方法建立严密的理论和算法基础。前者对于理解依赖路径和通过标准条件均值回归分析无法挽回的异质效应是必不可少的。然而，现有的大多数分位数回归计算方法都是基于通用算法的，当变量数量很大时，这种算法在大规模机器学习应用中不具有可扩展性。卷积平滑允许快速校准的基于梯度的算法，而不会影响估计的质量，因此提供了统计精度和计算精度之间的平衡。它还扩展了分位数回归的适用范围，从低维到高维，从完全观察到部分观察样本，从线性结构到非线性结构，在现代大数据分析中。该项目的第一部分将侧重于三个统计问题：(A)高维稀疏分位数回归，(B)大规模删失分位数回归，和(C)具有再降M估计的稳健回归。研究的第二部分集中于为两种现代数据类型下的非光滑损失函数方法开发高效的分散算法：(I)并行和分布式数据，以及(Ii)在线流数据。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Recent years have witnessed the transition of statistical analysis from a small- or moderate-scale data environment to a world involving massive data on parallel and distributed computing platforms. However, such a transition poses significant statistical and computational challenges for many important methods with non-smooth loss functions. As a representative example, quantile regression methods are building blocks for many advanced methods in statistics and econometrics and are frequently used to model financial data and medical data. The computational inflexibility makes quantile regression less favorable among various branches of the statistical learning tool kit. The project aims to develop a unified framework for large-scale learning with non-smooth loss functions to address the aforementioned problems. The developed methods will be applied to analyze complex biomedical data subject to censoring or privacy protocol and large-scale public health data. Both graduate and undergraduate students will receive training through research involvement in the project, ranging from developing new methods and theory to open-source software under different platforms. The principal investigators will use a combination of tools from statistics, optimization, and probability to develop a unified convolution smoothing framework and establish rigorous theoretical and algorithmic foundations for a class of statistical methods with non-differentiable loss, typified by quantile regression and support vector machine. The former is indispensable for understanding pathways of dependence and heterogeneous effects irretrievable through standard conditional mean regression analysis. However, most existing computational methods for quantile regression are based on generic algorithms, which are not scalable in large-scale machine learning applications when the number of variables is large. Convolution smoothing admits fast calibrated gradient-based algorithms without compromising the estimates' quality, therefore offering a balanced trade-off between statistical accuracy and computational precision. It also extends the applicability of quantile regression, from low to high dimensions, fully to partially observed samples, and linear to nonlinear structures, in modern big data analytics. The first part of the project will focus on three statistical problems: (a) high-dimensional sparse quantile regression, (b) large-scale censored quantile regression, and (c) robust regression with redescending M-estimation. The second part of the research focuses on developing efficient decentralized algorithms for methods with non-smooth loss functions under two modern data types: (i) parallel and distributed data, and (ii) online streaming data.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.