Utilizing stability criteria in choosing feature selection methods yields reproducible results in microbiome data.
Utilizing stability criteria in choosing feature selection methods yields reproducible results in microbiome data.
复制标题
DOI：
10.1111/biom.13481
复制
发表时间：
2022-09
期刊：
影响因子：
1.9
通讯作者：
Natarajan, Loki
中科院分区：
文献类型：
--
作者：
Jiang, Lingjing;Haiminen, Niina;Carrieri, Anna-Paola;Huang, Shi;Vazquez-Baeza, Yoshiki;Parida, Laxmi;Kim, Ho-Cheol;Swafford, Austin D.;Knight, Rob;Natarajan, Loki
特征选择在微生物组数据分析中是必不可少的，但它可能特别具有挑战性，因为微生物组数据集是高维的，欠定的，稀疏的和组成的。最近已经做出了很大的努力，开发新的方法来处理上述数据特征的特征选择，但几乎所有的方法进行评估的基础上模型预测的性能。然而，很少有人注意到一个根本问题：这些评价标准是否适当？大多数特征选择方法通常控制模型拟合，但是识别有意义的特征子集的能力不能简单地基于预测精度来评估。如果数据的微小变化会导致所选特征子集的大变化，则许多所选特征可能是数据伪影，而不是真实的生物信号。这种识别相关和可重现特征的关键需求促使了重现性评价标准，如稳定性，它量化了方法对数据扰动的鲁棒性。在我们的论文中，我们比较了流行的模型预测指标（MSE或AUC）的性能与建议的再现性标准稳定性，以评估模拟和实验微生物组应用中四种广泛使用的特征选择方法的连续或二进制结果。我们的结论是，稳定性是一个首选的特征选择标准模型预测指标，因为它更好地量化的特征选择方法的再现性。
Feature selection is indispensable in microbiome data analysis, but it can be particularly challenging as microbiome data sets are high dimensional, underdetermined, sparse and compositional. Great efforts have recently been made on developing new methods for feature selection that handle the above data characteristics, but almost all methods were evaluated based on performance of model predictions. However, little attention has been paid to address a fundamental question: how appropriate are those evaluation criteria? Most feature selection methods often control the model fit, but the ability to identify meaningful subsets of features cannot be evaluated simply based on the prediction accuracy. If tiny changes to the data would lead to large changes in the chosen feature subset, then many selected features are likely to be a data artifact rather than real biological signal. This crucial need of identifying relevant and reproducible features motivated the reproducibility evaluation criterion such as Stability, which quantifies how robust a method is to perturbations in the data. In our paper, we compare the performance of popular model prediction metrics (MSE or AUC) with proposed reproducibility criterion Stability in evaluating four widely used feature selection methods in both simulations and experimental microbiome applications with continuous or binary outcomes. We conclude that Stability is a preferred feature selection criterion over model prediction metrics because it better quantifies the reproducibility of the feature selection method.
登录
查看更多内容
影响因子：
16.6
作者：
Duvallet C;Gibbons SM;Gurry T;Irizarry RA;Alm EJ
通讯作者：
Alm EJ
影响因子：
4.4
作者：
Lauber, Christian L.;Hamady, Micah;Fierer, Noah
通讯作者：
Fierer, Noah
影响因子：
30.3
作者：
Gevers D;Kugathasan S;Denson LA;Vázquez-Baeza Y;Van Treuren W;Ren B;Schwager E;Knights D;Song SJ;Yassour M;Morgan XC;Kostic AD;Luo C;González A;McDonald D;Haberman Y;Walters T;Baker S;Rosh J;Stephens M;Heyman M;Markowitz J;Baldassano R;Griffiths A;Sylvester F;Mack D;Kim S;Crandall W;Hyams J;Huttenhower C;Knight R;Xavier RJ
通讯作者：
Xavier RJ
影响因子：
8.5
作者：
Liu, Yong;Tang, Shaoxun;Gonzalez-Diaz, Humberto
通讯作者：
Gonzalez-Diaz, Humberto
DOI：
10.1146/annurev-statistics-010814-020351
发表时间：
2015-01-01
期刊：
ANNUAL REVIEW OF STATISTICS AND ITS APPLICATION, VOL 2
影响因子：
--
作者：
Li, Hongzhe
通讯作者：
Li, Hongzhe