III: Small: Deep Interactive Reinforcement Learning for Self-optimizing Feature Selection

特征选择是一个经典但基本的机器学习任务，旨在选择相关特征（变量，预测因子）的子集来构建预测模型。特征选择在生物标记物发现、系统监控、故障诊断、图像识别、文本挖掘、金融欺诈检测等领域有着广泛的应用。然而，一个长期存在的批评，在功能选择的实践状态是，现有的方法需要经验规范的超参数，缺乏搜索最佳的功能子集的能力，并忽略了集级别的功能功能的相互作用。为了填补研究空白，该项目将开发一个深入且交互式的增强特征选择学习框架（RFSL）。该项目的创新之处在于提出了一个自优化特征选择的概念，以实现两个目标：1）自优化和2）全局最优。该项目的影响是提高特征选择的自动化和最优性，丰富需要特征选择的预测建模的可用性和适用性，并推进生物医学应用的代表性生物标志物发现。首先是框架的挑战：我们如何开发一个机器学习框架来自动化自优化特征选择，同时提供有效性保证？第二个是互动的挑战：互动机制可以帮助代理商利用外部和先验知识，以提高学习？第三个是反馈挑战：下游任务是否可以反馈其中间结果以改进特征选择？ 我们将通过以下方面来回答这些问题：（1）学习框架：将开发一个新的学习框架（RFSL），以平衡自动化和自优化特征选择的有效性。(2)互动学习机制：互动的观点将被提出，以增加RFSL的外部和先验知识的学习能力。三种新机制（即，行动层面、奖励层面、环境层面），拓展RFSL的互动渠道。 (3)循环中的算法反馈：将开发循环中算法的新方法，以利用下游预测任务中的特征树结构反馈，提供特征选择策略的自适应学习，以克服分布偏移和模型漂移。(4)嵌入到真实的系统中：所提出的框架将被集成到最近开发的下一代测序平台中（例如，该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Feature selection is a classic, yet fundamental, machine learning task that aims to select a subset of relevant features (variables, predictors) to construct a predictive model. Feature selection has a wide range of applications, such as biomarker discovery, system monitoring, fault diagnosis, image recognition, text mining, and financial fraud detection. However, a longstanding criticism of the state of the practice in feature selection is that existing methods require empirical specifications of hyperparameters, lack the ability to search for the best feature subset, and ignore set-level feature-feature interaction. To fill the research gap, this project will develop a deep and interactive reinforced feature selection learning framework (RFSL). The project’s novelties are to propose a self-optimizing feature selection concept to achieve two goals: 1) self-optimization and 2) global optimality. The project's impacts are to improve the automation and optimality of feature selection, enrich the availability and applicability of predictive modeling that need feature selection, and advance representative biomarker discovery for biomedical applications.To achieve this goal, we will address three technical challenges. The first is the framework challenges: how can we develop a machine learning framework to automate the self-optimizing feature selection while providing an effectiveness guarantee? The second is interaction challenges: which interaction mechanisms can help agents to leverage external and prior knowledge to improve learning? The Third is feedback challenges: can downstream tasks feed their intermediate results back to improve feature selection? We will answer the questions by the following thrusts: (1) Learning Framework: a new learning framework (RFSL) will be developed to balance automation and effectiveness in self-optimizing feature selection. (2) Interactive Learning Mechanisms: an interactive perspective will be proposed to augment the external and prior knowledge learning ability of RFSL. Three novel mechanisms (i.e., action level, reward level, and environment level) will be developed to expand the interaction channels of RFSL. (3) Algorithm in The Loop Feedback: new approaches for algorithms in the loop will be developed to take advantage of feature tree structure feedback in a downstream predictive task to provide adaptive learning of feature selection policies to overcome distribution shifts and model drifts. (4) Embedding into Real Systems: the proposed framework will be integrated into the recently developed next-generation sequencing platform (e.g., mRNA-sequencing, whole-genome sequencing) with high-dimension low sample-size genomic data to identify robust molecular signatures to better understand the biological mechanisms behind different diseases.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.