Parametric Programming Approach for More Powerful and General Lasso Selective Inference
Parametric Programming Approach for More Powerful and General Lasso Selective Inference
复制标题
DOI：
--
复制
发表时间：
2020-04
期刊：
影响因子：
--
通讯作者：
Vo Nguyen Le Duy;I. Takeuchi
中科院分区：
文献类型：
--
作者：
Vo Nguyen Le Duy;I. Takeuchi
选择性推理（Selective Inference， SI）在过去的几年中得到了积极的研究，用于对由Lasso等特征选择方法自适应选择的线性模型的特征进行推理。逻辑推理的基本思想是使推理以选择事件为条件。不幸的是，在Lee等人\cite{lee2016exact}的开创性工作中提出的Lasso的原始SI方法的主要局限性是，推断不仅取决于所选择的特征，而且取决于它们的符号——这导致由于过度调节而导致功率损失。虽然可以通过考虑所有可能的符号组合的选择事件的联合来规避这一限制，但这只有在选择的特征数量足够小时才可行。为了解决这一计算瓶颈，我们提出了一种基于参数规划的方法，即使我们有数千个活动特征，也可以在不影响符号的情况下进行SI。其主要思想是沿检验统计量方向计算Lasso解的连续路径，并通过求解路径识别特征选择事件对应的数据空间子集。提出的基于参数规划的方法不仅避免了上述计算瓶颈，而且在各个方面提高了Lasso SI的性能和实用性。我们进行了几个实验来证明我们提出的方法的有效性和效率。
Selective Inference (SI) has been actively studied in the past few years for conducting inference on the features of linear models that are adaptively selected by feature selection methods such as Lasso. The basic idea of SI is to make inference conditional on the selection event. Unfortunately, the main limitation of the original SI approach for Lasso, proposed in the seminal work by Lee et al. \cite{lee2016exact}, is that the inference is conducted not only conditional on the selected features but also on their signs---this leads to loss of power because of over-conditioning. Although this limitation can be circumvented by considering the union of such selection events for all possible combinations of signs, this is only feasible when the number of selected features is sufficiently small. To address this computational bottleneck, we propose a parametric programming-based method that can conduct SI without conditioning on signs even when we have thousands of active features. The main idea is to compute the continuum path of Lasso solutions in the direction of a test statistic, and identify the subset of the data space corresponding to the feature selection event by following the solution path. The proposed parametric programming-based method not only avoids the aforementioned computational bottleneck but also improves the performance and practicality of SI for Lasso in various respects. We conduct several experiments to demonstrate the effectiveness and efficiency of our proposed method.