Influencing Reinforcement Learning through Natural Language Guidance-猫眼课题宝

DOI：

10.32473/flairs.v34i1.128472

发表时间：

2021-04

期刊：

ArXiv

影响因子：

通讯作者：

中科院分区：

其他

文献类型：

作者：

Tasmia Tasrin;Md Sultan Al Nahian;Habarakadage Perera;Brent Harrison

交互式强化学习（IRL）代理使用人类反馈或指令来帮助他们在复杂的环境中学习。通常，这种反馈以正或负的离散信号的形式出现。虽然信息丰富，但这些信息本身可能很难概括。在这项工作中，我们探索如何通过扩展策略塑造（一种众所周知的 IRL 技术），使用自然语言建议向强化学习代理提供更丰富的反馈信号。通常，策略制定会采用人类反馈策略来帮助代理更多地了解如何实现其目标。在我们的例子中，我们用基于自然语言建议生成的策略替换了这种人类反馈策略。我们的目标是检查生成的自然语言推理是否为深度强化学习代理提供支持，以在任何给定环境中成功决定其行动。因此，我们用三个网络设计模型：第一个是体验驱动的，第二个是建议生成器，第三个是建议驱动的。经验驱动的强化学习智能体选择的行动受到环境奖励的影响，而建议驱动的神经网络则通过建议生成器为任何新状态生成反馈，选择其行动来帮助强化学习智能体更好地制定政策。

Interactive reinforcement learning (IRL) agents use human feedback or instruction to help them learn in complex environments. Often, this feedback comes in the form of a discrete signal that’s either positive or negative. While informative, this information can be difficult to generalize on its own. In this work, we explore how natural language advice can be used to provide a richer feedback signal to a reinforcement learning agent by extending policy shaping, a well-known IRL technique. Usually policy shaping employs a human feedback policy to help an agent to learn more about how to achieve its goal. In our case, we replace this human feedback policy with policy generated based on natural language advice. We aim to inspect if the generated natural language reasoning provides support to a deep RL agent to decide its actions successfully in any given environment. So, we design our model with three networks: first one is the experience driven, next is the advice generator and third one is the advice driven. While the experience driven RL agent chooses its actions being influenced by the environmental reward, the advice driven neural network with generated feedback by the advice generator for any new state selects its actions to assist the RL agent to better policy shaping.

Influencing Reinforcement Learning through Natural Language Guidance

文献摘要

相似文献