Model-Augmented Conditional Mutual Information Estimation for Feature Selection
Model-Augmented Conditional Mutual Information Estimation for Feature Selection
复制标题
DOI：
--
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
Alan Yang;AmirEmad Ghassami;M. Raginsky;N. Kiyavash;E. Rosenbaum
中科院分区：
文献类型：
--
作者：
Alan Yang;AmirEmad Ghassami;M. Raginsky;N. Kiyavash;E. Rosenbaum
马尔可夫毯子特征选择虽然在理论上是最优的，但通常很难实现。这是由于现有条件独立（CI）测试方法的缺点，这些方法往往与维数的诅咒或计算复杂性作斗争。我们提出了一种新的两步法，方便了高维马尔可夫毯子特征的选择。首先，使用神经网络将特征映射到低维表示。在第二步中，通过对学习到的特征映射应用k -NN条件互信息估计器来进行CI测试。映射的设计是为了确保映射的样本既保留信息，又共享目标变量的相似信息，当且仅当它们在欧几里得距离上接近。在第二步中，我们证明了这些特性提高了k -NN估计器的性能。在综合数据和实际数据上对该方法的性能进行了评价。
Markov blanket feature selection, while theoretically optimal, is generally challenging to implement. This is due to the shortcomings of existing approaches to conditional independence (CI) testing, which tend to struggle either with the curse of dimensionality or computational complexity. We propose a novel two-step approach which facilitates Markov blanket feature selection in high dimensions. First, neural networks are used to map features to low-dimensional representations. In the second step, CI testing is performed by applying the k -NN conditional mutual information estimator to the learned feature maps. The mappings are designed to ensure that mapped samples both preserve information and share similar information about the target variable if and only if they are close in Euclidean distance. We show that these properties boost the performance of the k -NN estimator in the second step. The performance of the proposed method is evaluated on both synthetic and real data.