Deep learning with evolutionary synaptic pruning

在本财政年度，我们开发了一种专门的进化策略优化算法，该算法在优化过程中修剪深度神经网络模型。该方法结合了质量多样性算法和修剪算子，实现了相当于最先进的基于梯度的修剪技术的模型压缩。因此，这项工作表明了无梯度优化方法作为机器学习修剪方法的替代方法来解决强化学习问题的有效性。结果提交并发表在遗传与进化计算会议（GECCO 2022），一个顶级的国际会议。为了进一步了解修剪和参数优化之间的相互作用，我们设计了一个实验，实现了基于种群的遗传算法。初始种群是用随机修剪的神经模型初始化的，这些神经模型在上述实验中成功地学习了任务。我们试图找到信息理论测量和子网络之间的相关性，这些相关性迅速收敛到给定任务的解决方案。

During this fiscal year, we developed a specialized Evolutionary Strategy optimization algorithm that prunes deep neural network models during the optimization process. The proposed approach combines quality diversity algorithms and pruning operators that achieve a model compression equivalent to state-of-the-art gradient-based pruning techniques. This work thus suggests the validity of gradient-free optimization methods as an alternative to machine learning pruning approaches to reinforcement learning problems. Results were submitted and published at The Genetic and Evolutionary Computation Conference (GECCO 2022), a top-tier international conference.To further understand the interaction between pruning and parameters optimization, we devised an experiment implementing a population-based genetic algorithm. The initial population is initialized with randomly pruned neural models that successfully learn the task in the experiments described above.We are attempting to find correlations between information-theoretic measures and sub-network that rapidly converge to a solution to the given task.