Selecting the best optimizers for deep learning-based medical image segmentation.
Selecting the best optimizers for deep learning-based medical image segmentation.
复制标题
DOI：
10.3389/fradi.2023.1175473
复制
发表时间：
2023
期刊：
影响因子：
--
通讯作者：
Bagci, Ulas
中科院分区：
文献类型：
--
作者：
Mortazi, Aliasghar;Cicek, Vedat;Keles, Elif;Bagci, Ulas
关键词：
这项工作的目标是在医学图像分割的背景下探索深度学习的最佳优化器，并为如何设计具有有效优化策略的分割网络提供指导。大多数成功的深度学习网络都使用两种类型的随机梯度下降（SGD）算法进行训练：自适应学习和加速方案。自适应学习通过从较大的学习率（LR）开始并逐渐降低它来帮助快速收敛。动量优化器在加速方案类别中快速优化神经网络时特别有效。通过揭示这两种类型的算法之间的潜在相互作用[LR和动量优化器或简称为动量率（MR）]，在本文中，我们将在单个设置中探索SGD算法的两种变体。我们建议使用循环学习作为基础优化器，并整合学习率和动量率的最佳值。在这项工作中提出的新的优化功能是基于Nesterov加速梯度优化器，这是更有效的计算，并具有更好的泛化能力相比，其他自适应优化。我们研究了LR和MR的关系下的一个重要问题的医学图像分割的心脏结构的MRI和CT扫描。我们使用来自MICCAI 2017的ACDC挑战的心脏成像数据集进行了实验，四种不同的架构被证明可以成功解决心脏图像分割问题。我们的综合评估表明，与深度学习文献中的其他优化器相比，所提出的优化器在单对象和多对象分割设置中具有相似或更低的计算成本，从而实现了更好的结果（骰子度量提高了2%以上）。我们假设，加速和自适应优化方法的组合可以在医学图像分割性能产生巨大的影响。为此，我们提出了一种新的循环优化方法（循环学习/动量率），以解决基于深度学习的医学图像分割的效率和准确性问题。所提出的策略产生了更好的推广相比，自适应优化。
The goal of this work is to explore the best optimizers for deep learning in the context of medical image segmentation and to provide guidance on how to design segmentation networks with effective optimization strategies. Most successful deep learning networks are trained using two types of stochastic gradient descent (SGD) algorithms: adaptive learning and accelerated schemes. Adaptive learning helps with fast convergence by starting with a larger learning rate (LR) and gradually decreasing it. Momentum optimizers are particularly effective at quickly optimizing neural networks within the accelerated schemes category. By revealing the potential interplay between these two types of algorithms [LR and momentum optimizers or momentum rate (MR) in short], in this article, we explore the two variants of SGD algorithms in a single setting. We suggest using cyclic learning as the base optimizer and integrating optimal values of learning rate and momentum rate. The new optimization function proposed in this work is based on the Nesterov accelerated gradient optimizer, which is more efficient computationally and has better generalization capabilities compared to other adaptive optimizers. We investigated the relationship of LR and MR under an important problem of medical image segmentation of cardiac structures from MRI and CT scans. We conducted experiments using the cardiac imaging dataset from the ACDC challenge of MICCAI 2017, and four different architectures were shown to be successful for cardiac image segmentation problems. Our comprehensive evaluations demonstrated that the proposed optimizer achieved better results (over a 2% improvement in the dice metric) than other optimizers in the deep learning literature with similar or lower computational cost in both single and multi-object segmentation settings. We hypothesized that the combination of accelerated and adaptive optimization methods can have a drastic effect in medical image segmentation performances. To this end, we proposed a new cyclic optimization method (Cyclic Learning/Momentum Rate) to address the efficiency and accuracy problems in deep learning–based medical image segmentation. The proposed strategy yielded better generalization in comparison to adaptive optimizers.
登录
查看更多内容
影响因子：
10.9
作者：
LaLonde R;Xu Z;Irmakci I;Jain S;Bagci U
通讯作者：
Bagci U
影响因子：
17.6
作者：
Bizopoulos, Paschalis;Koutsouris, Dimitrios
通讯作者：
Koutsouris, Dimitrios
DOI：
10.1109/tpami.2016.2572683
发表时间：
2017-04-01
影响因子：
23.6
作者：
Shelhamer, Evan;Long, Jonathan;Darrell, Trevor
通讯作者：
Darrell, Trevor
影响因子：
7.7
作者：
Huang, Qing;Sun, Jinfeng;Wang, Guangzhi
通讯作者：
Wang, Guangzhi
影响因子：
7.8
作者：
Qian, N
通讯作者：
Qian, N