Multi-Agent Reinforcement Learning for the Low-Level Control of a Quadrotor UAV-猫眼课题宝

DOI：

10.48550/arxiv.2311.06144

发表时间：

2023-11

期刊：

ArXiv

影响因子：

通讯作者：

中科院分区：

其他

文献类型：

作者：

Beomyeol Yu;Taeyoung Lee

通过利用四旋翼动力学的底层结构，我们提出了多智能体强化学习框架，以创新四旋翼的低级别控制，其中独立的智能体协同操作以实现共同的目标。虽然单智能体强化学习已成功应用于四旋翼控制，但训练大型单片网络通常是数据密集型和耗时的。此外，由于四旋翼动力学的强耦合性质，实现敏捷偏航控制仍然是一个重大挑战。为了解决这个问题，我们将四旋翼动力学分解为平移和偏航组件，并为每个部分分配协作强化学习代理，以促进更有效的训练。此外，我们引入正则化项来减轻稳态误差并防止过度机动。基准研究，包括SIM到SIM传输验证，表明我们提出的训练方案大大提高了训练的收敛速度，同时提高了飞行控制性能和稳定性相比，传统的单智能体方法。

By leveraging the underlying structures of the quadrotor dynamics, we propose multi-agent reinforcement learning frameworks to innovate the low-level control of a quadrotor, where independent agents operate cooperatively to achieve a common goal. While single-agent reinforcement learning has been successfully applied in quadrotor controls, training a large monolithic network is often data-intensive and time-consuming. Moreover, achieving agile yawing control remains a significant challenge due to the strongly coupled nature of the quadrotor dynamics. To address this, we decompose the quadrotor dynamics into translational and yawing components and assign collaborative reinforcement learning agents to each part to facilitate more efficient training. Additionally, we introduce regularization terms to mitigate steady-state errors and prevent excessive maneuvers. Benchmark studies, including sim-to-sim transfer verification, demonstrate that our proposed training schemes substantially improve the convergence rate of training, while enhancing flight control performance and stability compared to traditional single-agent approaches.

Multi-Agent Reinforcement Learning for the Low-Level Control of a Quadrotor UAV

文献摘要

相似文献