Learning to Map Natural Language Instructions to Physical Quadcopter Control using Simulated Flight
Learning to Map Natural Language Instructions to Physical Quadcopter Control using Simulated Flight
复制标题
DOI：
--
复制
发表时间：
2019-10
期刊：
影响因子：
--
通讯作者：
Valts Blukis;Yannick Terme;Eyvind Niklasson;Ross A. Knepper;Yoav Artzi
中科院分区：
文献类型：
--
作者：
Valts Blukis;Yannick Terme;Eyvind Niklasson;Ross A. Knepper;Yoav Artzi
我们提出了一个联合模拟和现实世界学习框架，用于映射导航说明和原始的第一人称观察以进行连续控制。我们的模型估计了对环境探索的需求，可以预测执行过程中访问环境职位的可能性，并控制代理人探索和访问高样的位置。我们引入了监督的强化异步学习（传播）。学习同时使用模拟和实际环境，而无需在训练过程中在物理环境中进行自主飞行，并结合了监督的学习，以预测访问和强化学习以进行连续控制的职位。我们使用物理四轮驱动器评估了自然语言指导遵守任务的方法，并展示了有效的执行和探索行为。
We propose a joint simulation and real-world learning framework for mapping navigation instructions and raw first-person observations to continuous control. Our model estimates the need for environment exploration, predicts the likelihood of visiting environment positions during execution, and controls the agent to both explore and visit high-likelihood positions. We introduce Supervised Reinforcement Asynchronous Learning (SuReAL). Learning uses both simulation and real environments without requiring autonomous flight in the physical environment during training, and combines supervised learning for predicting positions to visit and reinforcement learning for continuous control. We evaluate our approach on a natural language instruction-following task with a physical quadcopter, and demonstrate effective execution and exploration behavior.