Bootstrapped fitness critics with bidirectional temporal difference
Bootstrapped fitness critics with bidirectional temporal difference
复制标题
具有双向时间差异的自举健身批评家
DOI：
10.1145/3520304.3528999
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Tumer, Kagan
中科院分区：
文献类型：
--
作者：
Rockefeller, Golden;Tumer, Kagan
进化算法（EA）非常适合解决许多具有全局、长期反馈的现实世界多智能体协调问题。然而，当反馈变得稀疏且缺乏信息时，EA 就会陷入困境。在这种情况下，系统设计者可以使用Fitness Critics，这是一种功能模型，用于估计代理将稀疏域反馈转换为密集奖励信号的贡献值。然而，用于更新健身评价的现有方法没有利用有关何时收到奖励的时间信息。理想情况下，时间差分 (TD) 方法可以利用有关稀疏反馈信号的时间信息来引导健身评价。然而，由于健身评论家的结构，直接应用 TD 方法协同进化算法会导致健身评论家低估了在剧集早期收到的奖励。本文介绍了双向健身批评家（BFC），它利用一种新颖的双向时间差异方法，成功地利用时间奖励信息引导健身批评家的训练，而不会低估早期奖励。该论文证明了在多智能体协调域上与 BFC 共同进化的智能体的收敛性能显着提高。
Evolutionary algorithms (EAs) are well suited for solving many real-world multiagent coordination problems with global, long-term feedback. However, EAs struggle when the feedback becomes sparse and uninformative. In such cases, a system designer can useFitness Critics, which are functional models that estimate the value of an agent's contribution to transform the sparse domain feedback into a dense reward signal. However, existing methods for updating fitness critics do not leverage the temporal information about when a reward is received. Ideally, temporal difference (TD) methods can leverage temporal information about the sparse feedback signal to bootstrap Fitness Critics. Yet, due to the structure Fitness Critics, direct application of TD methods coevolutionary algorithms result in Fitness Critics that under-represent the rewards that are received earlier in the episode. This paper introducesBidirectional Fitness Critics(BFCs), which makes use of a novel, bidirectional temporal difference method, to successfully bootstrap the training of fitness critics with temporal reward information, without under-representing early rewards. The paper demonstrates a significant increase in the converged performance of agents coevolved with BFCs on a multiagent coordination domain.
DOI：
10.1109/cec.2002.1006261
发表时间：
2000-09
期刊：
Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600)
影响因子：
--
作者：
Y. Sano;H. Kita
通讯作者：
Y. Sano;H. Kita
适应度函数的进化神经估计
DOI：
--
发表时间：
2003
期刊：
Portuguese Conference on Artificial Intelligence
影响因子：
--
作者：
J. Aguilar;D. Mateos;D. S. Rodríguez
通讯作者：
D. S. Rodríguez