Advancing Theory and Computation in Statistical Learning Problems

本研究由四个相关的统计学习项目组成。前两个项目都是理论上的。首先，研究人员将研究自适应建模技术的自由度(即有效参数数)。已经证明，基于L1范数的变量选择过程，例如套索，表现出对它们的有效参数数量的控制，因为这里的适应性被系数估计的收缩所抵消。这个项目转而考虑不使用收缩的自适应程序，例如最佳子集选择，其中参数的有效数量(相对地)大大膨胀。在第二个项目中，研究者将检验趋势滤波，这是最近提出的一种通过惩罚离散导数的L1范数来拟合的非参数回归估计量。趋势过滤估计可以有效地计算(例如，使用第三个项目的工作)，但其理论性质并不被很好地理解。其目的是研究广义函数类上趋势滤波估计的收敛速度，并与现有的非参数回归估计(如光滑样条、局部自适应回归样条等)进行详细的比较。最后两个项目是计算性的。第三个项目是关于广义套索路径算法的高效计算。广义套索是一种使用L1范数鼓励特定结构属性的估计器，而不是纯粹的稀疏性本身；上面提到的趋势滤波估计器就是这样一个例子。第四个也是最后一个项目是将逐步回归背后的思想扩展到一般的凸正则化问题。正向逐步回归是一种简单、可扩展的算法，其估计可视为套索正则化路径的近似值。对一般问题的阶段性扩展为群体套索、矩阵补全等产生了有效的近似算法；近似保证是未知的，将被研究。在许多科学学科中，统计建模、估计和推断正在成为问题的组成部分。因此，统计学习领域-概括了这三项统计任务-最近见证了研究的爆炸性增长。可以说，当前该领域的研究集中在创建新方法或将方法扩展到新领域，而对理解现有方法的研究则更少。取而代之的是，研究人员将进行四个项目，旨在(I)加深我们对一些众所周知(但不那么被理解)的统计学习技术的理解，以及(Ii)开发算法，以便我们能够在更大范围内有效地使用这些技术，从而评估它们的性能。此类算法的代码将通过开源软件免费提供。这项工作的潜在应用包括医疗诊断的预测，神经科学中的大脑信号建模，以及推荐系统的开发。

This research is composed of four related statistical learning projects. The first two projects are theoretical. In the first, the investigator will study of degrees of freedom (i.e., the effective number of parameters) of adaptive modeling techniques. It has been shown that variable selection procedures based on the L1 norm, such as the lasso, exhibit control over their effective number of parameters, since adaptivity here is counterbalanced by shrinkage in coefficient estimation. This project instead considers adaptive procedures that do not employ shrinkage, such as best subset selection, in which the effective number of parameters is (comparatively) greatly inflated. In the second project, the investigator will examine trend filtering, a recently proposed nonparametric regression estimator fit by penalizing the L1 norm of discrete derivatives. Trend filtering estimates can be computed efficiently (e.g., using the work of the third project), but their theoretical properties are not well-understood. The goal is to study the rate of convergence of trend filtering estimates over broad function classes, and make detailed comparisons to existing nonparametric regression estimators (such as smoothing splines, locally adaptive regression splines, etc.). The last two projects are computational. The third project is focused on efficient computations for the generalized lasso path algorithm. The generalized lasso is an estimator that encourages specific structural properties, as opposed to pure sparsity itself, using the L1 norm; one such example is the trend filtering estimator mentioned above. The fourth and final project is an extension of the idea behind stagewise regression to general convex regularization problems. Forward stagewise regression is a simple, scalable algorithm whose estimates can be seen as an approximation to the lasso regularization path. The stagewise extension to general problems produces efficient approximation algorithms for the group lasso, matrix completion, and more; approximation guarantees are unknown and will be studied. Statistical modeling, estimation, and inference are becoming integral aspects of problems in many scientific disciplines. As a result, the field of statistical learning---which broadly encapsulates these three statistical tasks---has witnessed a recent explosion of research. Arguably, current research in this field focuses on creating new methods or extending methods to new domains, and much less so on understanding existing methods. Instead, the investigator will pursue four projects aimed at (i) deepening our understanding of a few well-known (but not as well-understood) statistical learning techniques, and (ii) developing algorithms so that we can employ these techniques efficiently at a larger scale, and hence evaluate their performance. Code for such algorithms will be made freely available through open-source software. Potential applications of this work include the forecasting of medical diagnoses, the modeling of brain signals in neuroscience, and the development of recommender systems.