Triple-Sigmoid Activation Function for Deep Open-Set Recognition
Triple-Sigmoid Activation Function for Deep Open-Set Recognition
复制标题
用于深度开集识别的三重 Sigmoid 激活函数
DOI：
10.1109/access.2022.3192621
复制
发表时间：
2022
期刊：
影响因子：
3.9
通讯作者：
JOO-HO LEE
中科院分区：
文献类型：
--
作者：
DINH TUAN TRAN ;NOBUTAKA SHIMADA;JOO-HO LEE
用于各种机器学习问题（如图像分类）的传统模型只有在封闭集的假设下才能表现良好。这意味着输入必须属于模型被训练的类。在现实世界中收集的数据可能不属于任何有限的类集，并且用无限数量的类训练模型显然是不可能的。学习模型必须识别和拒绝这些数据样本，或者请求人工帮助标记它们，而不是错误地将属于未知类的离群输入分类为模型所训练的类之一的成员。例如，当自动驾驶汽车检测到不熟悉的场景或物体时，它必须通知驾驶员并移交控制权。已经提出了各种模型来解决开集问题。然而，现有的模型通常是复杂的或使用复杂的技术，如生成对抗网络和自编码器，它们的效率与它们的复杂性不成比例。在这项研究中，我们提出了一个简单的激活函数，由三个Sigmoid函数组成。在任何深度神经网络模型的最后一层激活中使用triple - s - moid使模型能够识别异常值。虽然三重sigmoid可以应用于各种机器学习问题，包括半监督学习、主动学习和增量学习，但我们在这项工作中只研究了开集识别问题。大量的实验结果证明，用Triple-Sigmoid替代Softmax和Sigmoid等传统激活方法不仅在闭集环境下保持了性能，而且在开集环境下也显著提高了性能。此外，这些结果表明，Triple-Sigmoid明显优于现有的最先进的方法。建议模型的源代码可从https://github.com/dinhtuantran/triple-sigmoid获得。
Traditional models for various machine learning problems such as image classification perform well only under the assumption of a closed set. This implies that inputs must belong to the classes for which the models were trained. Data collected in the real world may not belong to any finite set of classes, and training a model with an infinite number of classes would obviously be impossible. Rather than incorrectly classifying outlier inputs that belong to unknown classes as members of one of the classes on which the model was trained, learning models must recognize and reject such data samples, or request human assistance in labeling them. For example, when a self-driving vehicle detects unfamiliar scenes or objects, it must notify the driver and hand over control. Various models have been proposed to address the open-set problem. However, the existing models are generally complex or use complicated techniques such as generative adversarial networks and auto-encoders, and their efficiency is not proportional to their complexity. In this study, we propose Triple-Sigmoid as a simple activation function comprised of three Sigmoid functions. Using Triple-Sigmoid in the last activation layer of any deep neural network model enables the model to recognize outliers. Although Triple-Sigmoid can be applied to a variety of machine learning problems, including semi-supervised learning, active learning, and incremental learning, we only investigated the open-set recognition problem in this work. The results of numerous experiments are presented to validate that substituting Triple-Sigmoid for conventional activations such as Softmax and Sigmoid not only retained performance in closed-set settings, but also significantly improved performance in open-set configurations. Furthermore, these results demonstrate that Triple-Sigmoid significantly outperformed existing state-of-the-art methods. Source code for the proposed model is available at https://github.com/dinhtuantran/triple-sigmoid.