Adaptive Neuroevolution With Genetic Operator Control and Two-Way Complexity Variation
Adaptive Neuroevolution With Genetic Operator Control and Two-Way Complexity Variation
复制标题
DOI：
10.1109/tai.2022.3214181
复制
发表时间：
2023-12
期刊：
影响因子：
--
通讯作者：
A. Behjat;Nathan Maurer;Sharat Chidambaran;Souma Chowdhury
中科院分区：
文献类型：
--
作者：
A. Behjat;Nathan Maurer;Sharat Chidambaran;Souma Chowdhury
拓扑和权值演化人工神经网络算法同时优化人工神经网络的结构和权值。由此产生的网络通常用作解决控制和强化学习（RL）类型问题的策略模型。本文提出了一种神经进化算法，旨在解决其他神经进化算法中存在的典型停滞和缓慢收敛问题。这些问题往往是由种群多样性保护、勘探/开发平衡和搜索灵活性不足引起的。这种新的算法，称为自适应基因组进化的神经网络拓扑结构（代理），建立在神经进化的增强拓扑结构（NEAT）的概念。提出了适应选择和变异操作的新机制，以有利地控制种群多样性和探索/开发平衡。前者是建立在一个全新的方式量化多样性采取图论的角度来看人口的基因组和基因组间的差异。NEAT范式的进一步发展是通过将可变神经元特性和新的突变操作结合起来实现的，这些操作独特地允许ANN拓扑在进化过程中的增长和修剪。采用OpenAI Gym的基准控制问题的数值实验说明了AGENT与标准RL方法和自适应HyperNEAT的竞争性能，以及优于原始NEAT算法的优势。进一步的参数分析提供了对AGENT中新功能影响的关键见解。其次是评估的无人机避碰问题，其中机动规划模型学习的AGENT与33%的奖励改善超过15代。
Topology and weight evolving artificial neural network algorithms optimize the structure and weights of artificial neural networks (ANNs) simultaneously. The resulting networks are typically used as policy models for solving control and reinforcement learning (RL) type problems. This article presents a neuroevolution algorithm that aims to address the typical stagnation and sluggish convergence issues present in other neuroevolution algorithms. These issues are often caused by inadequacies in population diversity preservation, exploration/exploitation balance, and search flexibility. This new algorithm, called the adaptive genomic evolution of neural-network topologies (AGENT), builds on the neuroevolution of augmenting topologies (NEAT) concept. Novel mechanisms for adapting the selection and mutation operations are proposed to favorably control population diversity and exploration/exploitation balance. The former is founded on a fundamentally new way of quantifying diversity by taking a graph-theoretic perspective of the population of genomes and intergenomic differences. Further advancements to the NEAT paradigm occur through the incorporation of variable neuronal properties and new mutation operations that uniquely allow both the growth and pruning of ANN topologies during evolution. Numerical experiments with benchmark control problems adopted from the OpenAI Gym illustrate the competitive performance of AGENT against standard RL methods and adaptive HyperNEAT, and superiority over the original NEAT algorithm. Further parametric analysis provides key insights into the impact of the new features in AGENT. This is followed by evaluation on an unmanned aerial vehicle collision avoidance problem where maneuver planning models are learnt by AGENT with 33% reward improvement over 15 generations.