Cooperative Coevolution of Neural Networks in Sequential Decision Tasks

顺序决策任务中神经网络的协同协同进化

• 批准号: 0083776
• 负责人: Risto Miikkulainen
• 金额: $ 41.91万
• 依托单位: University of Texas at Austin
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2000
• 资助国家: 美国
• 起止时间: 2000-09-15 至 2004-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0083776&HistoricalAwards=false
• 关键词: Cooperative; Coevolution; Neural; Networks; Sequential

In sequential decision tasks such as resource optimization, robot control, and game playing, several decisions must be made before the outcome can be evaluated. Such reinforcement feedback depends on the entire sequence of decisions, and it is difficult to determine which of the decisions were responsible for the outcome. This project aims at developing better techniques for learning in domains with such sparse feedback, based on evolving neural networks with genetic algorithms. The goal is both to be able to solve existing problems faster, and to be able to solve problems that have not been feasible as sequential decision tasks before. Our previous work showed that neuroevolution is most powerful when individual neurons are evolved to cooperate and form good networks. In this project, such cooperative coevolution methods are studied in depth. The research aims at answering three main questions: Where does the power of cooperative coevolution come from and what are the best ways of making use of it? How do the evolutionary reinforcement learning methods differ from the traditional value function methods in learning sequential decision tasks? Does evolutionary reinforcement learning have the accuracy and flexibility required in real-world applications? If successful, the project will result in cooperative coevolution algorithms that will solve existing sequential decision tasks faster, and will allow solving more difficult tasks than before. We will know how to decide between evolutionary and value function methods for a given reinforcement learning task, and also how to use each method most effectively. Finally, the project will demonstrate how learning in general, and cooperative coevolution of neural networks in particular, can be used to save resources and achieve complex behavior in challenging real-world tasks.

在资源优化、机器人控制和玩游戏等顺序决策任务中，在评估结果之前必须做出几个决策。这种强化反馈依赖于整个决策序列，很难确定哪些决策对结果负有责任。这个项目的目的是开发更好的技术，在这种稀疏反馈的领域学习，基于进化的神经网络和遗传算法。我们的目标是既能够更快地解决现有问题，又能够解决以前作为顺序决策任务不可行的问题。我们之前的工作表明，当单个神经元进化成协作并形成良好的网络时，神经进化是最强大的。本课题对这种协作式协同进化方法进行了深入的研究。这项研究旨在回答三个主要问题：合作共同进化的力量来自哪里，利用这种力量的最佳方式是什么？进化强化学习方法在学习序列决策任务时与传统的值函数方法有何不同？进化强化学习在实际应用中是否具有所需的准确性和灵活性？如果成功，该项目将产生协作式协同进化算法，该算法将更快地解决现有的顺序决策任务，并将允许解决比以前更困难的任务。我们将知道如何为给定的强化学习任务在进化函数方法和价值函数方法之间做出选择，以及如何最有效地使用每种方法。最后，该项目将演示如何使用一般的学习，特别是神经网络的协作协同进化，在挑战现实世界的任务中节省资源和实现复杂的行为。