Interactive Reinforcement Learning for Feature Selection With Decision Tree in the Loop
Interactive Reinforcement Learning for Feature Selection With Decision Tree in the Loop
复制标题
循环决策树特征选择的交互式强化学习
DOI：
10.1109/tkde.2021.3102120
复制
发表时间：
2020-10
影响因子：
8.9
通讯作者：
Wei Fan;Kunpeng Liu;Hao Liu;Yong Ge;Hui Xiong;Yanjie Fu
中科院分区：
文献类型：
--
作者：
Wei Fan;Kunpeng Liu;Hao Liu;Yong Ge;Hui Xiong;Yanjie Fu
研究了自动特征选择中的有效性和效率的平衡问题。特征选择就是从大的特征空间中找到一个最优的特征子集。在研究了多种特征选择方法后，我们发现了一个计算上的困境：1)传统的特征选择方法(如mRMR)大多是有效的，但很难确定最佳子集；2)新出现的强化特征选择算法自动导航特征空间来搜索最佳子集，但效率通常较低。自动化和效率总是分开的吗？我们能在自动化下弥合效率和效率之间的差距吗？在这种困境的驱使下，我们的目标是开发一种新的特征空间导航方法。在我们的初步工作中，我们利用交互式强化学习通过外部训练器-代理交互来加快特征选择。通过将其下游任务(例如决策树)的结构化知识建模为学习反馈，我们的初步工作可以得到显著改进。在这个期刊版本中，我们提出了一个新的交互式和闭环体系结构来同时建模交互式强化学习(IRL)和决策树反馈(DTF)。具体来说，IRL是创建一个交互式的特征选择循环，DTF是将结构化的特征知识反馈给该循环。DTF从两个方面对IRL进行了改进。首先，利用决策树生成的树状特征层次结构来改进状态表示。特别地，我们将所选特征子集表示为特征-特征相关性的无向图和决策特征的有向树。我们提出了一种新的嵌入方法，该方法能够使图卷积网络(GCN)同时从图和树中学习状态表示。其次，利用树形结构的特征层次结构设计了一种新的奖励方案。特别地，我们基于决策树特征重要性对代理的奖励分配进行个性化。此外，观察智能体的行为也可以是一种反馈，我们设计了另一种新的奖励机制，根据历史行为记录中每个智能体选择的频率比例来权衡和分配奖励。最后，我们用真实世界的数据集进行了大量的实验，证明了我们方法的改进性能。
We study the problem of balancing effectiveness and efficiency in automated feature selection. Feature selection is to find an optimal feature subset from large feature space. After exploring many feature selection methods, we observe a computational dilemma: 1) traditional feature selection (e.g., mRMR) is mostly efficient, but difficult to identify the best subset; 2) the emerging reinforced feature selection automatically navigates feature space to search the best subset, but is usually inefficient. Are automation and efficiency always apart from each other? Can we bridge the gap between effectiveness and efficiency under automation? Motivated by this dilemma, we aim to develop a novel feature space navigation method. In our preliminary work, we leveraged interactive reinforcement learning to accelerate feature selection by external trainer-agent interaction. Our preliminary work can be significantly improved by modeling the structured knowledge of its downstream task (e.g., decision tree) as learning feedback. In this journal version, we propose a novel interactive and closed-loop architecture to simultaneously model interactive reinforcement learning (IRL) and decision tree feedback (DTF). Specifically, IRL is to create an interactive feature selection loop and DTF is to feed structured feature knowledge back to the loop. The DTF improves IRL from two aspects. First, the tree-structured feature hierarchy generated by decision tree is leveraged to improve state representation. In particular, we represent the selected feature subset as an undirected graph of feature-feature correlations and a directed tree of decision features. We propose a new embedding method capable of empowering Graph Convolutional Network (GCN) to jointly learn state representation from both the graph and the tree. Second, the tree-structured feature hierarchy is exploited to develop a new reward scheme. In particular, we personalize reward assignment of agents based on decision tree feature importance. In addition, observing agents’ actions can also be a feedback, we devise another new reward scheme, to weigh and assign reward based on the selected frequency ratio of each agent in historical action records. Finally, we present extensive experiments with real-world datasets to demonstrate the improved performances of our method.