Enumerating Multiple Equivalent Lasso Solutions
Enumerating Multiple Equivalent Lasso Solutions
复制标题
枚举多个等效套索解决方案
DOI：
--
复制
发表时间：
2017
期刊：
影响因子：
--
通讯作者：
I. Tsamardinos
中科院分区：
文献类型：
--
作者：
Yannis Pantazis;V. Lagani;I. Tsamardinos
预测建模是许多科学领域中常见的数据分析任务。然而，对于同一个问题，多个预测模型可以同样表现良好，这是相当未知的。当在具有低样本数、高维特征空间和/或高噪声水平的数据集中搜索唯一解决方案时，这种多样性通常导致再现性差，这是生物学和医学中的常见情况。Lasso回归是最强大和最流行的正则化方法之一，但它也会产生单一的稀疏解。在本文中，我们表明，接近最优的Lasso解决方案，其样本外的统计误差实际上是无法区分的最佳之一，存在。我们形式化的各种概念之间的等价Lasso解决方案，我们设计了一个算法来枚举那些在统计意义上是等价的：我们定义了一个公差的均方根误差（RMSE），创建一个RMSE等价的Lasso解决方案空间。回归和分类任务的结果表明，由于RMSE松弛的样本外误差是由于抽样大小的统计误差的范围内。
Predictive modelling is a data-analysis task common in many scientific fields. However, it is rather unknown that multiple predictive models can be equally well-performing for the same problem. This multiplicity often leads to poor reproducibility when searching for a unique solution in datasets with low number of samples, high dimensional feature space and/or high levels of noise, a common scenario in biology and medicine. The Lasso regression is one of the most powerful and popular regularization methods, yet it also produces a single, sparse solution. In this paper, we show that nearly-optimal Lasso solutions, whose out-of-sample statistical error is practically indistinguishable from the optimal one, exist. We formalize various notions of equivalence between Lasso solutions, and we devise an algorithm to enumerate the ones that are equivalent in a statistical sense: we define a tolerance on the root mean square error (RMSE) which creates a RMSE-equivalent Lasso solution space. Results in both regression and classification tasks reveal that the out-of-sample error due to the RMSE relaxation is within the range of the statistical error due to the sampling size.