Deep learning with evolutionary synaptic pruning

具有进化突触修剪的深度学习

• 批准号: 22KF0142
• 负责人: オベル加藤 ナタナエル
• 金额: $ 1.41万
• 依托单位: Ochanomizu University
• 依托单位国家: 日本
• 项目类别: Grant-in-Aid for JSPS Fellows
• 财政年份: 2023
• 资助国家: 日本
• 起止时间: 2023-03-08 至 2025-03-31
• 项目状态: 未结题
• 来源: https://kaken.nii.ac.jp/grant/KAKENHI-PROJECT-22KF0142/
• 关键词: Evolutionary Strategy; Synaptic Pruning; Quality-Diversity

During this fiscal year, we developed a specialized Evolutionary Strategy optimization algorithm that prunes deep neural network models during the optimization process. The proposed approach combines quality diversity algorithms and pruning operators that achieve a model compression equivalent to state-of-the-art gradient-based pruning techniques. This work thus suggests the validity of gradient-free optimization methods as an alternative to machine learning pruning approaches to reinforcement learning problems. Results were submitted and published at The Genetic and Evolutionary Computation Conference (GECCO 2022), a top-tier international conference.To further understand the interaction between pruning and parameters optimization, we devised an experiment implementing a population-based genetic algorithm. The initial population is initialized with randomly pruned neural models that successfully learn the task in the experiments described above.We are attempting to find correlations between information-theoretic measures and sub-network that rapidly converge to a solution to the given task.

在本财政年度，我们开发了一种专门的进化策略优化算法，该算法在优化过程中修剪深度神经网络模型。该方法结合了质量多样性算法和修剪算子，实现了相当于最先进的基于梯度的修剪技术的模型压缩。因此，这项工作表明了无梯度优化方法作为机器学习修剪方法的替代方法来解决强化学习问题的有效性。结果提交并发表在遗传与进化计算会议（GECCO 2022），一个顶级的国际会议。为了进一步了解修剪和参数优化之间的相互作用，我们设计了一个实验，实现了基于种群的遗传算法。初始种群是用随机修剪的神经模型初始化的，这些神经模型在上述实验中成功地学习了任务。我们试图找到信息理论测量和子网络之间的相关性，这些相关性迅速收敛到给定任务的解决方案。

项目成果

Synaptic pruning with MAP-elites

• DOI: 10.1145/3520304.3528813
• 发表时间: 2022-07
• 期刊: Proceedings of the Genetic and Evolutionary Computation Conference Companion
• 作者: Federico Da Rold;Olaf Witkovski;N. Aubert-Kato