ERI: Distributed Learning in Regulation of UAV Communication Networks with Dynamic UAV Lineup

ERI：动态无人机阵容的无人机通信网络调节中的分布式学习

• 批准号: 2412393
• 负责人: Ran Zhang
• 金额: $ 19.28万
• 依托单位: University of North Carolina at Charlotte
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-11-15 至 2024-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2412393&HistoricalAwards=false
• 关键词: ERI; Distributed; Learning; Regulation; UAV

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).Unmanned aerial vehicles, or drones, have been demonstrating impressive potentials in next generation wireless communications. Compared to the terrestrial cellular base stations, drones equipped with wireless transceivers can serve as mobile base stations, and stand out in providing highly on-demand services with flexible 3D mobility, better wireless connectivity with higher chance of Line-of-Sight links, and much lower deployment cost with almost infrastructure-free network construction. Promising as it is, drone based communication networks still face fundamental regulation challenges: i) as drones are highly mobile, the dynamically changing network topology may make it unpractical for a centralized control unit to collect complete network information and make collective decisions; ii) the environment in which drones operate may be dynamically changing or unexplored without a priori knowledge of the environment modeling, making the conventional optimization or rule-based methods hardly applicable; iii) the research is still embryonic on how to optimally regulate the drone network when the lineup of the serving drones dynamically change, which, however, will be a common event in realistic implementation. To this end, this proposal is aimed to crack the nut of the above identified issues, and develop an effective framework for distributed network regulation solutions that are environment-model-free and well adaptive to the dynamic drone lineup. The research outcomes are expected to provide valuable inspirations and benchmarking to the distributed, scalable, and artificial-intelligence powered management of aerial access communication networks under a dynamic network setup. Such networks will be embraced as a key component in the larger-scope Space-Air-Ground Integrated Networks for the beyond-5G mobile telecommunications. The success of the project will potentially contribute to the leadership and competence of the United States worldwide in future generation mobile telecommunications as well as elevating national communication welfare with more integrated and on-demand communication infrastructure.In this project, multi-agent reinforcement learning will be applied to establish a distributed and model-free network regulation framework. The framework will feature strong capability in making sequential decisions in complex time-varying environments. Under the developed framework, the project aims to investigate how the drone communication networks should responsively handle and further proactively control the dynamic change of the drone lineup in a distributed yet coordinated manner. Specifically, responsive strategies will be first designed for a general drone communication network. The strategies will jointly optimize the radio resource management and trajectory design for the drones when the drone lineup change dynamically. The learning algorithm design will be investigated with different levels of inter-drone information exchange. The learning exploration will be promoted by adopting the structure of asynchronous parallel computing. The network will be prototyped leveraging on programmable drone products and simple-yet-effective communication protocols. To move one step further, proactive control strategies will be derived for the solar-powered self-sustainable drone communication network, which proactively control the quit and join-in of the drones by pre-shaping their solar-charging plan. The strategies will consider dynamic user spatial and traffic distributions by combining Fourier analysis, Long-Short Term Memory and Gaussian process regression for distribution prediction, and enabling predicting while learning to significantly reduce the reinforcement learning complexity. The hybrid cooperative-compete relationship among individual drones will be handled by exploiting Nash Q learning and correlated Q learning. The anxiety on the high-dimension state-action space in learning will be relieved by adopting problem decomposition techniques. The proposed project will advance the research on autonomous regulation of drone communication networks by filling the gaps of missing control strategy design to responsively handle and proactively control the drone lineup change. In addition, the introduction of game theory into the distributed framework makes the research more realistic with diversified autonomy in individual drones. The prototyping plan will complement the simulation evaluation on the time complexity and communication overhead/latency in the real-world implementation.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该奖项是根据2021年《美国救援计划法》（第117-2）的全部或部分资助的。无人机或无人机在下一代无线通信中表现出了令人印象深刻的潜力。与陆地蜂窝基站相比，配备了无线收发器的无人机可以用作移动基站，并在提供具有灵活的3D移动性，更好的无线连接性以及更高的视线链路链路以及几乎无基础结构网络网络构建方面的较高的部署成本方面脱颖而出。尽管如此，基于无人机的通信网络仍然面临着基本的法规挑战：i）由于无人机是高度移动的，因此动态变化的网络拓扑可能使集中式控制单元收集完整的网络信息并做出集体决策是不切实际的； ii）无人机操作的环境可能会动态更改或未探索，而没有对环境建模的先验知识，从而使常规优化或基于规则的方法几乎不适用； iii）该研究仍然是关于如何在服务无人机的阵容动态变化时如何最佳调节无人机网络的胚胎，但是，这将是现实实现的常见事件。为此，该提案的目的是破解上述问题的螺母，并为无环境模型且适应动态无人机阵容的分布式网络调节解决方案开发有效的框架。预计研究结果将为在动态网络设置下的空中访问通信网络的分布式，可扩展和人工智能的管理提供宝贵的灵感和基准测试。此类网络将被作为超出5G移动电信的大型空间空间集成网络中的关键组成部分。该项目的成功将有可能为全球在未来的移动电信中的领导和能力做出贡献，并通过更整合和按需通信基础架构提高国家通信福利。在该项目中，多阶级加固学习将用于建立分布式和模型网络调节框架。该框架将具有强大的能力，可以在复杂的时变环境中做出顺序决策。在开发的框架下，该项目旨在调查无人机通信网络应如何响应地处理并进一步积极地以分布式但协调的方式积极地控制无人机阵容的动态变化。具体而言，响应式策略将首先是为一般无人机通信网络设计的。当无人机阵容动态变化时，这些策略将共同优化无人机的无线电资源管理和轨迹设计。学习算法设计将通过不同级别的无人机间信息交换进行研究。学习探索将通过采用异步平行计算的结构来促进。该网络将原型用于可编程无人机产品和简单效果的通信协议。为了更进一步，将为太阳能自动可持续的无人机通信网络提供积极主动的控制策略，该策略通过预先塑造其太阳能收费计划来主动控制无人机的戒烟和加入。这些策略将通过结合傅立叶分析，长期记忆和高斯流程回归来考虑动态的用户空间和流量分布，以进行分配预测，并可以预测，同时学会显着降低加强学习的复杂性。单个无人机之间的混合合作竞争关系将通过利用NASH Q学习和相关的Q学习来处理。通过采用问题分解技术，将减轻对学习高维状态空间的焦虑。拟议的项目将通过填补缺失的控制策略设计的空白来响应地处理和主动控制无人机阵容变化，从而推动对无人机通信网络自主调节的研究。此外，将游戏理论引入分布式框架使研究更现实，而单个无人机中的自主权多样化。原型计划将补充对现实世界实施中时间复杂性和沟通的开销/延迟的模拟评估。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子和更广泛影响的评估来审查标准的评估值得支持的。