Collaborative Research: MLWiNS: Dino-RL: A Domain Knowledge Enriched Reinforcement Learning Framework for Wireless Network Optimization

合作研究：MLWiNS：Dino-RL：用于无线网络优化的领域知识丰富的强化学习框架

• 批准号: 2003131
• 负责人: Jing Yang
• 金额: $ 18.16万
• 依托单位: Pennsylvania State Univ University Park
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-06-01 至 2024-05-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2003131&HistoricalAwards=false
• 关键词: Collaborative; Research; MLWiNS; Dino; RL

Reinforcement learning (RL) methods have met with renewed interest in recent years for adaptively configuring wireless networks. Despite the promising early results and the conceptual match, many existing approaches do not develop and tailor the RL methods to fit the unique characteristics of wireless networking. The goal of this project is to develop a novel domain knowledge enriched RL framework, or Dino-RL, to address this problem. The Dino-RL framework aims to seamlessly integrate the physical-law based modeling and an abstract episodic memory into the RL process, and has the potential to revamp the operation and management of future wireless networks. Developing this novel technology would also help maintain the nation's continued leadership in wireless technologies and its pipeline of highly qualified engineers. The project pursues synergistic activities for the successful design and implementation of Dino-RL, followed by a comprehensive, real-world data driven evaluation. Episodic RL is first studied with the objective to incorporate domain knowledge into building an efficient episodic memory. In addition, a hierarchical hidden variable model is built to enable meta-reinforcement learning for knowledge transfer and efficient exploration. Lastly, the conflict between enhancing the physical-law based modeling and reinforcement learning is balanced via novel sample-efficient model selection algorithms.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

近年来，强化学习(RL)方法在自适应配置无线网络方面重新引起了人们的兴趣。尽管有很有希望的早期结果和概念匹配，但许多现有的方法并没有开发和定制RL方法来适应无线网络的独特特征。该项目的目标是开发一个新的领域知识丰富的RL框架，或Dino-RL，以解决这一问题。Dino-RL框架旨在将基于物理定律的建模和抽象情节记忆无缝地集成到RL过程中，并具有改造未来无线网络运营和管理的潜力。开发这项新技术还将有助于保持美国在无线技术和高素质工程师队伍中的持续领先地位。该项目开展协同活动，以成功设计和实施Dino-RL，随后进行全面的、以真实世界数据为导向的评估。首次对情景记忆进行研究的目的是将领域知识融入到建立有效的情景记忆中。此外，还建立了分层隐变量模型，以实现元强化学习的知识转移和有效探索。最后，通过新的样本效率模型选择算法平衡了增强基于物理定律的建模和强化学习之间的冲突。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Safe Exploration Incurs Nearly No Additional Sample Complexity for Reward-free RL

• DOI: 10.48550/arxiv.2206.14057
• 发表时间: 2022-06
• 期刊: ArXiv
• 作者: Ruiquan Huang;J. Yang;Yingbin Liang

Provably Efficient Offline Reinforcement Learning with Perturbed Data Sources

• DOI: 10.48550/arxiv.2306.08364
• 发表时间: 2023-06
• 期刊: ArXiv
• 作者: Chengshuai Shi;Wei Xiong;Cong Shen;Jing Yang

Federated Linear Contextual Bandits

• 发表时间: 2021-10
• 作者: Ruiquan Huang;Weiqiang Wu;Jing Yang;Cong Shen

On Federated Learning with Energy Harvesting Clients

• DOI: 10.1109/icassp43922.2022.9746608
• 发表时间: 2022-02
• 期刊: ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
• 作者: Cong Shen;Jing Yang;Jie Xu

Near-optimal Conservative Exploration in Reinforcement Learning under Episode-wise Constraints

• DOI: 10.48550/arxiv.2306.06265
• 发表时间: 2023-06
• 期刊: Mathematics
• 影响因子: 2.4
• 作者: Donghao Li;Ruiquan Huang;Cong Shen;Jing Yang