USING MUTUAL INFORMATION FOR SELECTING FEATURES IN SUPERVISED NEURAL-NET LEARNING
USING MUTUAL INFORMATION FOR SELECTING FEATURES IN SUPERVISED NEURAL-NET LEARNING
复制标题
DOI：
10.1109/72.298224
复制
发表时间：
1994-07-01
影响因子：
--
通讯作者：
BATTITI, R
中科院分区：
文献类型：
--
作者：
BATTITI, R
本文研究了应用互信息准则来评估一组候选特征，并选择一个信息子集作为神经网络分类器的输入数据。由于互信息度量随机变量之间的任意依赖关系，因此它适用于评估复杂分类任务中特征的“信息内容”，其中基于线性关系（如相关性）的方法容易出错。互信息与所选坐标无关的事实允许鲁棒估计。尽管如此，使用互信息的任务，其特征在于高输入维数需要适当的近似，因为禁止的要求计算和样本。提出了一种算法，该算法是基于一个“贪婪”的功能选择，并考虑到输出类和相对于已经选定的功能的互信息。最后对一系列实验结果进行了讨论。
This paper investigates the application of the mutual information criterion to evaluate a set of candidate features and to select an informative subset to be used as input data for a neural network classifier. Because the mutual information measures arbitrary dependencies between random variables, it is suitable for assessing the ''information content'' of features in complex classification tasks, where methods bases on linear relations (like the correlation) are prone to mistakes. The fact that the mutual information is independent of the coordinates chosen permits a robust estimation. Nonetheless, the use of the mutual information for tasks characterized by high input dimensionality requires suitable approximations because of the prohibitive demands on computation and samples. An algorithm is proposed that is based on a ''greedy'' selection of the features and that takes both the mutual information with respect to the output class and with respect to the already-selected features into account. Finally the results of a series of experiments are discussed.