Collaborative Research: MLWiNS: Dino-RL: A Domain Knowledge Enriched Reinforcement Learning Framework for Wireless Network Optimization

近年来，强化学习（RL）方法在自适应配置无线网络方面重新受到关注。尽管有很好的早期结果和概念匹配，许多现有的方法不开发和定制RL方法，以适应无线网络的独特特性。这个项目的目标是开发一个新的领域知识丰富的RL框架，或Dino-RL，以解决这个问题。Dino-RL框架旨在将基于物理定律的建模和抽象情景记忆无缝集成到RL过程中，并有可能改进未来无线网络的运营和管理。开发这项新技术还将有助于保持美国在无线技术和高素质工程师队伍中的持续领先地位。该项目追求协同活动，以成功设计和实施Dino-RL，然后进行全面的，真实世界的数据驱动评估。情景强化学习首先研究的目标是将领域知识纳入建立一个有效的情景记忆。此外，层次隐变量模型的建立，使元强化学习的知识转移和有效的探索。最后，通过新颖的样本效率模型选择算法，平衡了增强基于物理定律的建模和强化学习之间的冲突。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Reinforcement learning (RL) methods have met with renewed interest in recent years for adaptively configuring wireless networks. Despite the promising early results and the conceptual match, many existing approaches do not develop and tailor the RL methods to fit the unique characteristics of wireless networking. The goal of this project is to develop a novel domain knowledge enriched RL framework, or Dino-RL, to address this problem. The Dino-RL framework aims to seamlessly integrate the physical-law based modeling and an abstract episodic memory into the RL process, and has the potential to revamp the operation and management of future wireless networks. Developing this novel technology would also help maintain the nation's continued leadership in wireless technologies and its pipeline of highly qualified engineers. The project pursues synergistic activities for the successful design and implementation of Dino-RL, followed by a comprehensive, real-world data driven evaluation. Episodic RL is first studied with the objective to incorporate domain knowledge into building an efficient episodic memory. In addition, a hierarchical hidden variable model is built to enable meta-reinforcement learning for knowledge transfer and efficient exploration. Lastly, the conflict between enhancing the physical-law based modeling and reinforcement learning is balanced via novel sample-efficient model selection algorithms.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.