Deep Active Ensemble Sampling For Image Classification
Deep Active Ensemble Sampling For Image Classification
复制标题
DOI：
10.48550/arxiv.2210.05770
复制
发表时间：
2022-10
期刊：
影响因子：
6.6
通讯作者：
S. Mohamadi;Gianfranco Doretto;D. Adjeroh
中科院分区：
文献类型：
--
作者：
S. Mohamadi;Gianfranco Doretto;D. Adjeroh
传统的主动学习（AL）框架旨在通过主动请求对信息量最大的数据点进行标记来降低数据注释的成本。然而，将AL引入数据饥渴的深度学习算法一直是一个挑战。一些建议的方法包括基于不确定性的技术，几何方法，基于不确定性和几何方法的隐式组合，以及最近，基于半/自监督技术的框架。在本文中，我们解决这方面的两个具体问题。首先是需要有效的开发/勘探权衡在AL的样本选择。为此，我们提出了一个创新的整合，最近的进展，在不确定性为基础的和几何框架，使一个有效的探索/开发权衡样本选择策略。为此，我们建立在一个计算效率的近似汤普森采样的关键变化作为不确定性表示的后验估计。我们的框架提供了两个优点：（1）准确的后验估计，以及（2）计算开销和更高精度之间的可调权衡。第二个问题是需要改进深度AL中的训练协议。为此，我们使用半/自监督学习的思想来提出一种独立于所使用的特定AL技术的通用方法。综上所述，我们的框架显示出了显着的改进，在国家的最先进的，与相同的设置下的监督学习的性能相媲美的结果。我们展示了我们框架的实证结果，以及在四个数据集上与最先进的性能进行比较，即MNIST，CIFAR 10，CIFAR 100和ImageNet，以在两种不同的设置中建立新的基线。
Conventional active learning (AL) frameworks aim to reduce the cost of data annotation by actively requesting the labeling for the most informative data points. However, introducing AL to data hungry deep learning algorithms has been a challenge. Some proposed approaches include uncertainty-based techniques, geometric methods, implicit combination of uncertainty-based and geometric approaches, and more recently, frameworks based on semi/self supervised techniques. In this paper, we address two specific problems in this area. The first is the need for efficient exploitation/exploration trade-off in sample selection in AL. For this, we present an innovative integration of recent progress in both uncertainty-based and geometric frameworks to enable an efficient exploration/exploitation trade-off in sample selection strategy. To this end, we build on a computationally efficient approximate of Thompson sampling with key changes as a posterior estimator for uncertainty representation. Our framework provides two advantages: (1) accurate posterior estimation, and (2) tune-able trade-off between computational overhead and higher accuracy. The second problem is the need for improved training protocols in deep AL. For this, we use ideas from semi/self supervised learning to propose a general approach that is independent of the specific AL technique being used. Taken these together, our framework shows a significant improvement over the state-of-the-art, with results that are comparable to the performance of supervised-learning under the same setting. We show empirical results of our framework, and comparative performance with the state-of-the-art on four datasets, namely, MNIST, CIFAR10, CIFAR100 and ImageNet to establish a new baseline in two different settings.