SBIR Phase I: Drone Control in Turbulence via Reinforcement Learning

这项小企业创新研究（SBIR）第一阶段项目的更广泛影响是改善无人机控制。目前的控制器在特定环境下是有效的，但在湍流等环境和飞行条件下表现不佳，从而缩小了无人机的使用范围。该项目将推进即插即用解决方案，用户可以专注于特定于其用例的高级任务，如避障和路线规划。这种新的控制器在商业和军事环境中都有广泛的应用：它可以在各种环境中稳定飞行，在湍流条件下扩大飞行包线，并且由于控制效率的提高，可以执行更长时间的任务。该小企业创新研究（SBIR）第一阶段项目通过开发基于强化学习的飞行控制器来解决无人机在湍流中的控制问题。该项目将扩大四轴飞行器的设计范围，并为测试可应用于其他控制问题的强化学习算法提供系统和环境。现代系统依赖于比例积分导数（PID）控制器作为稳定飞行的重要组成部分。这些PID控制器依赖于整体调谐的静态功能，将机动命令转换为无人机电机的电压变化，以满足转子的目标旋转速度。代替静态定义的PID方程，这种新型控制器使用强化学习算法，这是机器学习的一个子集，其中智能体被训练以选择在环境中最大化奖励的行为。这种技术已经在各种不同的控制和博弈论任务中取得了比人类更好的表现，但人们对这些技术在取代PID控制系统时的表现知之甚少。与更简单的PID控制器相比，开发强化学习控制器的主要优势是用户能够在更高的抽象层次上查看无人机控制，从而减少了关注复杂任务飞行控制细节的需要。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The broader impact of this Small Business Innovation Research (SBIR) Phase I project is to improve drone control. Current controllers are effective in specific environments but perform poorly in environments and flight conditions such as turbulence, thereby narrowing the scope in which drones can be used. This project will advance a plug-and-play solution in which users can focus on higher-level tasks specific to their use case, like obstacle avoidance and route planning. This new controller has broad applications in both commercial and military settings: it enables stable flight across a wide array of environments, expands the flight envelope in turbulent conditions, and allows for longer missions due to increased control efficiency.This Small Business Innovation Research (SBIR) Phase I project addresses the problem of drone control in turbulence through the development of a reinforcement learning-based flight controller. The project will enlarge the design envelope for quadcopters as well as provide a system and environment for testing reinforcement learning algorithms that can be applied to other control problems. Contemporary systems rely on Proportional Integrative Derivative (PID) controllers as an essential part of stable flight. These PID controllers rely on holistically tuned, static functions to convert maneuvering commands into voltage changes across drone motors to meet the rotor’s targeted rotation speed. In lieu of statically defined PID equations, this novel controller uses a reinforcement learning algorithm, which is a subset of machine learning where an agent is trained to select actions that maximize a reward across an environment. This technique has led to greater-than-human performance across a variety of different control and game theory tasks, but little is known about how these techniques fare when replacing PID control systems. The primary advantage the development of a reinforcement learning controller would have over simpler PID controllers is the ability for the user to view drone control at a higher level of abstraction, thus mitigating the need to focus on the minutiae of flight control for complex missions.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.