Continual Learning Objective for Analyzing Complex Knowledge Representations.
Continual Learning Objective for Analyzing Complex Knowledge Representations.
复制标题
分析复杂知识表示的持续学习目标。
DOI：
10.3390/s22041667
复制
发表时间：
2022-02-21
期刊：
影响因子：
--
通讯作者：
Werghi N
中科院分区：
文献类型：
--
作者：
Khan AM;Hassan T;Akram MU;Alghamdi NS;Werghi N
人类倾向于从快速变化的环境中逐步学习，而不会包括或忘记已经学习的表示。虽然深度学习也有可能在某种程度上模仿这种人类行为，但它会遭受灾难性的遗忘，因为它在学习新知识时，已经学习的任务的性能会急剧下降。许多研究者已经提出了有希望的解决方案，以消除这种灾难性的遗忘在知识蒸馏过程中。然而，据我们所知，迄今为止还没有文献利用这些解决方案之间的复杂关系，并利用它们进行跨越多个数据集甚至多个领域的有效学习。在本文中，我们提出了一个持续学习的目标，包括相互蒸馏损失，以理解这种复杂的关系，并允许深度学习模型有效地保留先验知识，同时适应新的类，新的数据集，甚至新的应用程序。在跨越三个应用程序的九个公开可用的多供应商和多模式数据集上对所提出的目标进行了严格的测试，并实现了0.9863%的前1准确率和0.9930的F1分数。
Human beings tend to incrementally learn from the rapidly changing environment without comprising or forgetting the already learned representations. Although deep learning also has the potential to mimic such human behaviors to some extent, it suffers from catastrophic forgetting due to which its performance on already learned tasks drastically decreases while learning about newer knowledge. Many researchers have proposed promising solutions to eliminate such catastrophic forgetting during the knowledge distillation process. However, to our best knowledge, there is no literature available to date that exploits the complex relationships between these solutions and utilizes them for the effective learning that spans over multiple datasets and even multiple domains. In this paper, we propose a continual learning objective that encompasses mutual distillation loss to understand such complex relationships and allows deep learning models to effectively retain the prior knowledge while adapting to the new classes, new datasets, and even new applications. The proposed objective was rigorously tested on nine publicly available, multi-vendor, and multimodal datasets that span over three applications, and it achieved the top-1 accuracy of 0.9863% and an F1-score of 0.9930.
登录
查看更多内容
DOI：
10.1109/jbhi.2020.2982914
发表时间：
2021-01-01
影响因子：
7.7
作者：
Hassan, Taimur;Akram, Muhammad Usman;Nazir, Muhammad Noman
通讯作者：
Nazir, Muhammad Noman
影响因子：
13.7
作者：
Farsiu S;Chiu SJ;O'Connell RV;Folgar FA;Yuan E;Izatt JA;Toth CA;Age-Related Eye Disease Study 2 Ancillary Spectral Domain Optical Coherence Tomography Study Group
通讯作者：
Age-Related Eye Disease Study 2 Ancillary Spectral Domain Optical Coherence Tomography Study Group
影响因子：
2.8
作者：
Mery, Domingo;Riffo, Vladimir;Carrasco, Miguel
通讯作者：
Carrasco, Miguel
关注病变：用于视网膜光学相干断层扫描图像分类的病变感知卷积神经网络
影响因子：
10.6
作者：
Fang, Leyuan;Wang, Chong;Liu, Zhimin
通讯作者：
Liu, Zhimin
影响因子：
8
作者：
Akcay, Samet;Breckon, Toby
通讯作者：
Breckon, Toby