Cooperative Coevolution of Neural Networks in Sequential Decision Tasks

顺序决策任务中神经网络的协同协同进化

• 批准号: 0083776
• 负责人: Risto Miikkulainen
• 金额: $ 41.91万
• 依托单位: University of Texas at Austin
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2000
• 资助国家: 美国
• 起止时间: 2000-09-15 至 2004-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0083776&HistoricalAwards=false
• 关键词: Cooperative; Coevolution; Neural; Networks; Sequential

In sequential decision tasks such as resource optimization, robot control, and game playing, several decisions must be made before the outcome can be evaluated. Such reinforcement feedback depends on the entire sequence of decisions, and it is difficult to determine which of the decisions were responsible for the outcome. This project aims at developing better techniques for learning in domains with such sparse feedback, based on evolving neural networks with genetic algorithms. The goal is both to be able to solve existing problems faster, and to be able to solve problems that have not been feasible as sequential decision tasks before. Our previous work showed that neuroevolution is most powerful when individual neurons are evolved to cooperate and form good networks. In this project, such cooperative coevolution methods are studied in depth. The research aims at answering three main questions: Where does the power of cooperative coevolution come from and what are the best ways of making use of it? How do the evolutionary reinforcement learning methods differ from the traditional value function methods in learning sequential decision tasks? Does evolutionary reinforcement learning have the accuracy and flexibility required in real-world applications? If successful, the project will result in cooperative coevolution algorithms that will solve existing sequential decision tasks faster, and will allow solving more difficult tasks than before. We will know how to decide between evolutionary and value function methods for a given reinforcement learning task, and also how to use each method most effectively. Finally, the project will demonstrate how learning in general, and cooperative coevolution of neural networks in particular, can be used to save resources and achieve complex behavior in challenging real-world tasks.

在诸如资源优化、机器人控制和游戏等顺序决策任务中，在评估结果之前必须做出多个决策。这种强化反馈依赖于整个决策序列，很难确定哪些决策对结果负责。 该项目旨在开发更好的技术，在这样的稀疏反馈域学习，基于进化神经网络与遗传算法。目标是能够更快地解决现有问题，并能够解决以前作为顺序决策任务不可行的问题。 我们之前的工作表明，当单个神经元进化到合作并形成良好的网络时，神经进化是最强大的。 本项目对这种协同进化方法进行了深入的研究。这项研究旨在回答三个主要问题： 合作共同进化的力量从何而来？利用它的最佳方式是什么？ 进化强化学习方法在学习序列决策任务时与传统的值函数方法有何不同？ 进化强化学习是否具有现实世界应用所需的准确性和灵活性？ 如果成功，该项目将产生合作的共同进化算法，将更快地解决现有的顺序决策任务，并将允许解决比以前更困难的任务。 我们将知道如何在给定的强化学习任务中选择进化方法和值函数方法，以及如何最有效地使用每种方法。 最后，该项目将展示如何学习，特别是神经网络的合作共同进化，可以用来节省资源，并在具有挑战性的现实世界任务中实现复杂的行为。