Collaborative Research: Inference and Decentralized Computing for Quantile Regression and Other Non-Smooth Methods

近年来，统计分析已经从小型或中等规模的数据环境过渡到并行和分布式计算平台上的海量数据世界。然而，这种转换对许多具有非光滑损失函数的重要方法提出了重大的统计和计算挑战。作为一个代表性的例子，分位数回归方法是统计学和计量经济学中许多高级方法的基础，经常用于金融数据和医疗数据的建模。计算的不灵活性使得分位数回归在统计学习工具包的各个分支中不那么有利。该项目旨在开发具有非光滑损失函数的大规模学习的统一框架，以解决上述问题。开发的方法将应用于分析受审查或隐私协议约束的复杂生物医学数据和大规模公共卫生数据。研究生和本科生都将通过参与该项目的研究来接受培训，从开发新的方法和理论到在不同平台下开发开源软件。主要研究人员将使用统计学、优化和概率论的组合工具来开发统一的卷积平滑框架，并为一类具有不可微损失的统计方法（以分位数回归和支持向量机为代表）建立严格的理论和算法基础。前者对于理解通过标准条件平均回归分析无法恢复的依赖性和异质性效应的途径是必不可少的。然而，大多数现有的分位数回归计算方法都是基于通用算法的，在变量数量较大的大规模机器学习应用中，这种算法是不可扩展的。卷积平滑允许快速校准基于梯度的算法，而不会影响估计的质量，因此在统计准确性和计算精度之间提供平衡的权衡。它还扩展了分位数回归在现代大数据分析中的适用性，从低维到高维，从完全观察到部分观察样本，从线性到非线性结构。该项目的第一部分将集中在三个统计问题上：(a)高维稀疏分位数回归，(b)大规模删节分位数回归，以及(c)带重降m估计的稳健回归。研究的第二部分侧重于在两种现代数据类型(i)并行和分布式数据以及（ii）在线流数据下为具有非光滑损失函数的方法开发高效的分散算法。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Recent years have witnessed the transition of statistical analysis from a small- or moderate-scale data environment to a world involving massive data on parallel and distributed computing platforms. However, such a transition poses significant statistical and computational challenges for many important methods with non-smooth loss functions. As a representative example, quantile regression methods are building blocks for many advanced methods in statistics and econometrics and are frequently used to model financial data and medical data. The computational inflexibility makes quantile regression less favorable among various branches of the statistical learning tool kit. The project aims to develop a unified framework for large-scale learning with non-smooth loss functions to address the aforementioned problems. The developed methods will be applied to analyze complex biomedical data subject to censoring or privacy protocol and large-scale public health data. Both graduate and undergraduate students will receive training through research involvement in the project, ranging from developing new methods and theory to open-source software under different platforms. The principal investigators will use a combination of tools from statistics, optimization, and probability to develop a unified convolution smoothing framework and establish rigorous theoretical and algorithmic foundations for a class of statistical methods with non-differentiable loss, typified by quantile regression and support vector machine. The former is indispensable for understanding pathways of dependence and heterogeneous effects irretrievable through standard conditional mean regression analysis. However, most existing computational methods for quantile regression are based on generic algorithms, which are not scalable in large-scale machine learning applications when the number of variables is large. Convolution smoothing admits fast calibrated gradient-based algorithms without compromising the estimates' quality, therefore offering a balanced trade-off between statistical accuracy and computational precision. It also extends the applicability of quantile regression, from low to high dimensions, fully to partially observed samples, and linear to nonlinear structures, in modern big data analytics. The first part of the project will focus on three statistical problems: (a) high-dimensional sparse quantile regression, (b) large-scale censored quantile regression, and (c) robust regression with redescending M-estimation. The second part of the research focuses on developing efficient decentralized algorithms for methods with non-smooth loss functions under two modern data types: (i) parallel and distributed data, and (ii) online streaming data.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.