AS-NAS: Adaptive Scalable Neural Architecture Search With Reinforced Evolutionary Algorithm for Deep Learning
AS-NAS: Adaptive Scalable Neural Architecture Search With Reinforced Evolutionary Algorithm for Deep Learning
复制标题
AS-NAS：采用深度学习强化进化算法的自适应可扩展神经架构搜索
DOI：
10.1109/tevc.2021.3061466
复制
发表时间：
2021-02
影响因子：
14.3
通讯作者：
Chen C. L. Philip
中科院分区：
文献类型：
--
作者：
Zhang Tong;Lei Chunyu;Zhang Zongyan;Meng Xian-Bing;Chen C. L. Philip
神经结构搜索（NAS）由于其非凸性而成为深度学习设计中的一个难题。为了解决这一问题，提出了一种基于强化易经占卜进化算法和变结构编码策略的自适应可扩展NAS方法。首先，与典型的基于强化学习（RL）和进化算法（EA）的NAS方法不同，开发了一种简化的RL算法，并将其作为自适应选择IDEA有效算子的增强算子控制器。在简化强化学习的基础上，去掉复杂的关键因素部分，以较低的计算成本提高了原EA的搜索效率。其次，提出了一种可变结构编码策略，将神经结构编码为固定长度的二进制字符串。通过同时考虑可变层、通道和不同卷积层之间的连接，深度神经结构可以扩展。通过将增强的IDEA和变结构编码策略相结合，使深度神经网络的结构设计具有自适应可扩展性。最后，将所提出的AS-NAS与${L}_{1/2}$正则化相结合，提高了优化后神经网络结构的稀疏性。实验和比较验证了该方法的有效性和优越性。
Neural architecture search (NAS) is a challenging problem in the design of deep learning due to its nonconvexity. To address this problem, an adaptive scalable NAS method (AS-NAS) is proposed based on the reinforced I-Ching divination evolutionary algorithm (IDEA) and variable-architecture encoding strategy. First, unlike the typical reinforcement learning (RL)-based and evolutionary algorithm (EA)-based NAS methods, a simplified RL algorithm is developed and used as the reinforced operator controller to adaptively select the efficient operators of IDEA. Without the complex actor–critic parts, the reinforced IDEA based on simplified RL can enhance the search efficiency of the original EA with lower computational cost. Second, a variable-architecture encoding strategy is proposed to encode neural architecture as a fixed-length binary string. By simultaneously considering variable layers, channels, and connections between different convolution layers, the deep neural architecture can be scalable. Through the integration with the reinforced IDEA and variable-architecture encoding strategy, the design of the deep neural architecture can be adaptively scalable. Finally, the proposed AS-NAS are integrated with the ${L}_{1/2}$ regularization to increase the sparsity of the optimized neural architecture. Experiments and comparisons demonstrate the effectiveness and superiority of the proposed method.