III: Small: Deep Interactive Reinforcement Learning for Self-optimizing Feature Selection

特征选择是一项经典但基本的机器学习任务，旨在选择相关特征（变量、预测因子）的子集来构建预测模型。特征选择在生物标志物发现、系统监控、故障诊断、图像识别、文本挖掘、金融欺诈检测等领域有着广泛的应用。然而，长期以来对特征选择实践状态的批评是，现有方法需要超参数的经验规范，缺乏搜索最佳特征子集的能力，并且忽略集级特征-特征交互。为了填补研究空白，本项目将开发一个深度交互式强化特征选择学习框架（RFSL）。该项目的新颖之处在于提出了一个自优化的特征选择概念，以实现两个目标：1)自优化和2)全局最优。该项目的影响是提高特征选择的自动化和最优性，丰富需要特征选择的预测建模的可用性和适用性，并推进生物医学应用中代表性生物标志物的发现。为了实现这一目标，我们将解决三个技术挑战。首先是框架的挑战：我们如何开发一个机器学习框架，在提供有效性保证的同时，实现自优化特征选择的自动化？第二是交互挑战：哪种交互机制可以帮助代理利用外部和先验知识来提高学习？第三是反馈挑战：下游任务能否反馈中间结果以改进特征选择？我们将从以下几个方面来回答这些问题：(1)学习框架：我们将开发一个新的学习框架（RFSL）来平衡自优化特征选择的自动化和有效性。(2)互动学习机制：从互动的角度增强大学生的外部和先验知识学习能力。本文将从行动层面、奖励层面和环境层面三个层面建立新的机制，拓展RFSL的互动渠道。(3)环路反馈算法：将开发环路算法的新方法，利用下游预测任务中的特征树结构反馈来提供特征选择策略的自适应学习，以克服分布变化和模型漂移。(4)嵌入真实系统：所提出的框架将集成到最近开发的具有高维低样本基因组数据的下一代测序平台（例如mrna测序，全基因组测序）中，以识别强大的分子特征，从而更好地了解不同疾病背后的生物学机制。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Feature selection is a classic, yet fundamental, machine learning task that aims to select a subset of relevant features (variables, predictors) to construct a predictive model. Feature selection has a wide range of applications, such as biomarker discovery, system monitoring, fault diagnosis, image recognition, text mining, and financial fraud detection. However, a longstanding criticism of the state of the practice in feature selection is that existing methods require empirical specifications of hyperparameters, lack the ability to search for the best feature subset, and ignore set-level feature-feature interaction. To fill the research gap, this project will develop a deep and interactive reinforced feature selection learning framework (RFSL). The project’s novelties are to propose a self-optimizing feature selection concept to achieve two goals: 1) self-optimization and 2) global optimality. The project's impacts are to improve the automation and optimality of feature selection, enrich the availability and applicability of predictive modeling that need feature selection, and advance representative biomarker discovery for biomedical applications.To achieve this goal, we will address three technical challenges. The first is the framework challenges: how can we develop a machine learning framework to automate the self-optimizing feature selection while providing an effectiveness guarantee? The second is interaction challenges: which interaction mechanisms can help agents to leverage external and prior knowledge to improve learning? The Third is feedback challenges: can downstream tasks feed their intermediate results back to improve feature selection? We will answer the questions by the following thrusts: (1) Learning Framework: a new learning framework (RFSL) will be developed to balance automation and effectiveness in self-optimizing feature selection. (2) Interactive Learning Mechanisms: an interactive perspective will be proposed to augment the external and prior knowledge learning ability of RFSL. Three novel mechanisms (i.e., action level, reward level, and environment level) will be developed to expand the interaction channels of RFSL. (3) Algorithm in The Loop Feedback: new approaches for algorithms in the loop will be developed to take advantage of feature tree structure feedback in a downstream predictive task to provide adaptive learning of feature selection policies to overcome distribution shifts and model drifts. (4) Embedding into Real Systems: the proposed framework will be integrated into the recently developed next-generation sequencing platform (e.g., mRNA-sequencing, whole-genome sequencing) with high-dimension low sample-size genomic data to identify robust molecular signatures to better understand the biological mechanisms behind different diseases.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.