Glider soaring via reinforcement learning in the field
Glider soaring via reinforcement learning in the field
复制标题
DOI：
10.1038/s41586-018-0533-0
复制
发表时间：
2018-10-11
期刊：
影响因子：
64.8
通讯作者：
Vergassola, Massimo
中科院分区：
文献类型：
--
作者：
Reddy, Gautam;Wong-Ng, Jerome;Vergassola, Massimo
翱翔的鸟类在寻找猎物或长距离迁徙时通常依靠大气中上升的热羽流（热气流）(1-4)。对流的景观崎岖不平，并且随着热气流不断形成、分解或被风吹走而在几分钟内发生变化(5,6)。翱翔的鸟类如何在这片复杂的景观中找到并导航热气流尚不清楚。强化学习（7）提供了一个适当的框架，可以在其中将有效的导航策略识别为响应环境线索而做出的一系列决策。在这里，我们使用强化学习来训练滑翔机在现场自主导航大气热气流。我们为一架翼展两米的滑翔机配备了飞行控制器，可以精确控制倾斜角和俯仰角，并每隔一段时间对其进行调节，以期获得尽可能多的升力。导航策略完全根据滑翔机在几天的现场收集的经验来确定。该策略依靠机载方法来准确估计当地垂直风加速度和滑翔机上的横摇扭矩，作为导航提示。我们通过现场实验、数值模拟和大气湍流引起的测量噪声估计来确定所学飞行策略的有效性。我们的结果强调了垂直风加速度和横摇扭矩作为翱翔鸟类的有效机械感觉线索的作用，并提供了直接适用于自动翱翔车辆开发的导航策略。
Soaring birds often rely on ascending thermal plumes (thermals) in the atmosphere as they search for prey or migrate across large distances(1-4). The landscape of convective currents is rugged and shifts on timescales of a few minutes as thermals constantly form, disintegrate or are transported away by the wind(5,6). How soaring birds find and navigate thermals within this complex landscape is unknown. Reinforcement learning(7) provides an appropriate framework in which to identify an effective navigational strategy as a sequence of decisions made in response to environmental cues. Here we use reinforcement learning to train a glider in the field to navigate atmospheric thermals autonomously. We equipped a glider of two-metre wingspan with a flight controller that precisely controlled the bank angle and pitch, modulating these at intervals with the aim of gaining as much lift as possible. A navigational strategy was determined solely from the glider's pooled experiences, collected over several days in the field. The strategy relies on on-board methods to accurately estimate the local vertical wind accelerations and the roll-wise torques on the glider, which serve as navigational cues. We establish the validity of our learned flight policy through field experiments, numerical simulations and estimates of the noise in measurements caused by atmospheric turbulence. Our results highlight the role of vertical wind accelerations and roll-wise torques as effective mechanosensory cues for soaring birds and provide a navigational strategy that is directly applicable to the development of autonomous soaring vehicles.