Permutation based task transfer for genetic programming

本研究提案的总体背景是将遗传规划（GP）应用于在延迟支付环境中操作的代理（或强化学习）的学习决策策略。该提案的具体重点在于开发一个框架，以便系统地将GP扩展到比以前考虑的更困难的具有延迟回报的任务版本。我们特别感兴趣的场景是，一个简单的初始“源”任务的解决方案，然后“转移”到一个更困难但相关的（目标）任务；或者是一种迁移学习。这项工作的洞察力是利用GP的能力来识别利用状态变量子集的解决方案。然后，这为重新部署在源任务下发现的解决方案提供了基础，从而可以解决更困难的任务。采用这种方法的潜在好处是：1)不需要不断地从头开始重新发现政策；2)提高最终目标任务的成功率或更好的解决方案；3)相对于寻找每个任务的解决方案，计算开销更低。

The general context for this research proposal is that of genetic programming (GP) as applied to learning decision making policies for agents operating in environments with delayed payoff (or reinforcement learning). The specific focus of the proposal lies in developing a framework for systematically scaling GP to more difficult versions of tasks with delayed payoff than have previously been considered. In particular we are interested in scenarios in which solutions for a simpler initial `source' task are then `transferred' to a more difficult but related (target) task; or a form of transfer learning. The insight of this work is to make use of the capability of GP to identify solutions that make use of subsets of state variables. This then provides the basis for redeploying solutions discovered under the source task such that more difficult tasks can be solved. The potential benefits of adopting such an approach are that: 1) it is not necessary to continuously rediscover policies from scratch; 2) increased success in / or better solutions to the ultimate target task; and 3) lower computational overhead as measured against finding solutions to each task.