Applying Genetic Algorithms to Neural Network Optimization

本文将利用遗传算法来解决与神经网络相关的两类优化问题：(1)递归和前馈神经网络中权值的优化问题；(2)神经网络连通性的优化问题。为了建立衡量成功的具有挑战性的标准，它将建立和测试雷达信号脉冲探测的大型网络。使用遗传算法来定义神经网络的连通性直到去年才成为一个研究领域。一种遗传算法已被用来为小问题提供网络连通性，产生的网络学习速度比标准的“完全连接”网络快得多，而且在学习所需的数据表示数量上惊人地一致。这项工作将通过将用于权重优化的遗传算法代码与我们当前优化连接的方法的变体相结合来扩展当前的方法。这种组合应该允许同时优化连接性和大型问题的权重。最后，使用遗传算法成功优化前馈网络是非常重要的，因为这些算法不会对要求遗传算法评估的权重(参数)的实际使用方式做出任何假设。遗传算法应该可以直接应用于递归网络优化问题，当用真正的梯度下降学习技术训练时，这些问题往往具有有限的存储和计算要求。

This work will use genetic algorithms to solve two classes of optimization problems related to neural networks: (1) the optimization of weights in recurrent and feed forward neural networks and (2) the optimization of the connectivity of neural networks. To establish a challenging criteria for measuring success, it will build and test large networks for radar signal pulse detection. The use of genetic algorithms for defining neural network connectivity has emerged as an area of research only in the last year. A genetic algorithm has been used to fefine network connectivities for small problems, producing nets which learn much faster than standard "fully-connected" nets and which are suprisingly consistent in the number of data presentations required for learning. This work will extend the current approach by combining the genetic algorithm code for weight optimization with a variation on our current approach to optimizing the connectivity. This combination should allow optimization of connectivity and weights simultaneously on large problems. Finally, the successful optimization of feed forward networks using genetic algorithms is significant because these algorithms make no assumptions about how the weights (parameters) that the genetic algorithm is asked to evaluate are actually used. It should be possible to applied the genetic algorithms directly to recurrent network optimization problems which tend to have restrictive storage and computational requirements when trained with true gradient descent learning techniques.