Collaborative Research: CNS CORE: Small: RUI: Hierarchical Deep Reinforcement Learning for Routing in Mobile Wireless Networks

合作研究：CNS CORE：小型：RUI：移动无线网络中路由的分层深度强化学习

• 批准号: 2154190
• 负责人: Victoria Manfredi
• 金额: $ 32.63万
• 依托单位: Wesleyan University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-04-15 至 2025-03-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2154190&HistoricalAwards=false
• 关键词: Collaborative; Research; CNS; CORE; Small

The use of multi-hop routing in mobile wireless networks is becoming more prevalent, just as these networks are becoming more dense, dynamic, and heterogeneous. Designing a universal multi-hop routing strategy for mobile wireless networks is challenging, however, due to the need to seamlessly adapt routing behavior to spatially diverse and temporally changing network conditions. An alternative to using hand-crafted routing strategies is to use Reinforcement Learning (RL) to learn adaptive multi-hop routing strategies automatically. RL focuses on the design of intelligent agents: an RL agent interacts with its environment to learn a policy, i.e., which actions to take in different environmental states. By using function approximation like deep neural networks (DNNs) as in deep reinforcement learning (DeepRL) to approximate the policy, the RL agent can learn to generalize from its training experience to unseen network conditions and scale the learned routing strategy to larger networks. The PIs will continue their current practice of involving under-represented groups in research, and will use the project research to promote teaching and training through postdoctoral mentoring, course development, and outreach activities.The goal of this project is to use DeepRL to develop a universal multi-hop routing strategy for mobile wireless networks that is scalable, generalizable, and adaptive. Specifically, this project will build a novel routing framework that uses hierarchical DeepRL to design an option hierarchy, comprised of multiple layers of routing decisions working together to achieve the overall goals of the network. To enable the same routing strategy to be used at different devices and in unseen network scenarios, the framework will use relational features combined with novel neural network models to handle mobility and perform feature estimation. To further enhance generalizability, the framework will use continual learning to ensure that the routing behaviors learned for more recently seen network scenarios do not dominate the learned routing policy. The developed routing strategies will be thoroughly evaluated using both simulation and experimental testbeds. Through the use of hierarchical DeepRL, this project will provide a significant step forward in developing RL-based routing strategies, and will facilitate development of adaptive strategies for a wide range of mobile wireless networks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在移动无线网络中使用多跳路由正变得越来越普遍，正如这些网络变得更加密集、动态和异构一样。然而，由于需要无缝地调整路由行为以适应空间多样性和时间变化的网络条件，为移动无线网络设计通用的多跳路由策略是具有挑战性的。使用手工制作路由策略的另一种选择是使用强化学习（RL）来自动学习自适应多跳路由策略。RL专注于智能代理的设计：RL代理与环境交互以学习策略，即在不同的环境状态下采取哪些行动。通过在深度强化学习（DeepRL）中使用像深度神经网络（dnn）这样的函数近似来近似策略，RL代理可以学习将其训练经验推广到未知的网络条件，并将学习到的路由策略扩展到更大的网络。pi将继续他们目前的做法，让代表性不足的群体参与研究，并将利用项目研究通过博士后指导、课程开发和外展活动来促进教学和培训。该项目的目标是使用DeepRL为移动无线网络开发一种通用的多跳路由策略，该策略具有可扩展性、通用性和适应性。具体来说，该项目将构建一个新的路由框架，该框架使用分层DeepRL来设计一个选项层次结构，由多层路由决策组成，共同实现网络的总体目标。为了使相同的路由策略能够在不同的设备和不可见的网络场景中使用，该框架将使用关系特征与新型神经网络模型相结合来处理移动性并执行特征估计。为了进一步增强可泛化性，框架将使用持续学习来确保为最近看到的网络场景学习的路由行为不会主导学习的路由策略。开发的路由策略将通过仿真和实验测试平台进行全面评估。通过使用分层DeepRL，该项目将在开发基于rl的路由策略方面迈出重要一步，并将促进为广泛的移动无线网络开发自适应策略。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Hierarchical Prototype Networks for Continual Graph Representation Learning

• DOI: 10.1109/tpami.2022.3186909
• 发表时间: 2021-11
• 期刊: IEEE Transactions on Pattern Analysis and Machine Intelligence
• 影响因子: 23.6
• 作者: Xikun Zhang;Dongjin Song;D. Tao

Sparsified Subgraph Memory for Continual Graph Representation Learning

• DOI: 10.1109/icdm54844.2022.00177
• 发表时间: 2022-11
• 期刊: 2022 IEEE International Conference on Data Mining (ICDM)
• 作者: Xikun Zhang;Dongjin Song;D. Tao

Learning an adaptive forwarding strategy for mobile wireless networks: resource usage vs. latency

• DOI: 10.1007/s10994-024-06601-3
• 发表时间: 2024-08-07
• 期刊: MACHINE LEARNING
• 影响因子: 7.5
• 作者: Manfredi，Victoria;Wolfe，Alicia P.;Wang，Bing
• 通讯作者: Wang，Bing