Taking Advantage of Sparsity in Multi-Task Learning
Taking Advantage of Sparsity in Multi-Task Learning
复制标题
DOI：
--
复制
发表时间：
2009-03
期刊：
影响因子：
--
通讯作者：
Karim Lounici;M. Pontil;A. Tsybakov;S. Geer
中科院分区：
文献类型：
--
作者：
Karim Lounici;M. Pontil;A. Tsybakov;S. Geer
为了预测和选择变量，我们研究了多元线性回归方程的估计问题。在最近关于多任务学习的工作之后，Argyriou等人。[2008]，我们假设回归向量共享相同的稀疏模式。这意味着在不同的方程式中，相关预测变量的集合是相同的。这一假设使我们考虑将组合套索作为一种候选估计方法。我们证明了该估计具有良好的稀疏性、预言不等式和变量选择性质。所得结果在设计矩阵的约束特征值条件和相干性条件下成立，这自然地推广了Bickel等人最近的工作。[2007]，Lounici[2008]。特别是，在任务数量可以增长的多任务学习场景中，我们能够完全消除边界内预测器变量数量的影响。最后，我们展示了如何将我们的结果推广到更一般的噪声分布，其中我们只要求方差是有限的。
We study the problem of estimating multiple linear regression equations for the purpose of both prediction and variable selection. Following recent work on multi-task learning Argyriou et al. [2008], we assume that the regression vectors share the same sparsity pattern. This means that the set of relevant predictor variables is the same across the different equations. This assumption leads us to consider the Group Lasso as a candidate estimation method. We show that this estimator enjoys nice sparsity oracle inequalities and variable selection properties. The results hold under a certain restricted eigenvalue condition and a coherence condition on the design matrix, which naturally extend recent work in Bickel et al. [2007], Lounici [2008]. In particular, in the multi-task learning scenario, in which the number of tasks can grow, we are able to remove completely the effect of the number of predictor variables in the bounds. Finally, we show how our results can be extended to more general noise distributions, of which we only require the variance to be finite.