Cost-Minimising Strategies for Data Labelling: Optimal Stopping and Active Learning
Cost-Minimising Strategies for Data Labelling: Optimal Stopping and Active Learning
复制标题
数据标记的成本最小化策略：最佳停止和主动学习
DOI：
--
复制
发表时间：
2007
期刊：
影响因子：
--
通讯作者：
Christian Savu
中科院分区：
文献类型：
--
作者：
Christos Dimitrakakis;Christian Savu
监督学习处理以观察空间 X 中的点为条件的输出或标签空间 Y 上的分布推断，给定 X × Y 中对的训练数据集 D。然而，在许多感兴趣的应用中，获取大量观察结果很容易，而生成标签的过程非常耗时或昂贵。解决这个问题的一种方法是主动学习，其中选择要标记的点，目的是创建一个比在相同数量的随机采样点上训练的模型具有更好性能的模型。在本文中，我们建议直接处理标签成本：学习目标被定义为成本最小化，该成本是预期模型性能和所用标签总成本的函数。这允许开发用于（a）最佳停止的通用策略和特定算法，其中预期成本决定标签获取是否应该继续（b）经验评估，其中成本用作推理、停止和采样方法的给定组合的性能指标。尽管本文的主要焦点是最优停止，但我们也旨在为主动学习相关领域的进一步发展和讨论提供背景。
Supervised learning deals with the inference of a distribution over an output or label space Y conditioned on points in an observation space X, given a training dataset D of pairs in X × Y. However, in a lot of applications of interest, acquisition of large amounts of observations is easy, while the process of generating labels is time-consuming or costly. One way to deal with this problem is active learning, where points to be labelled are selected with the aim of creating a model with better performance than that of an model trained on an equal number of randomly sampled points. In this paper, we instead propose to deal with the labelling cost directly: The learning goal is defined as the minimisation of a cost which is a function of the expected model performance and the total cost of the labels used. This allows the development of general strategies and specific algorithms for (a) optimal stopping, where the expected cost dictates whether label acquisition should continue (b) empirical evaluation, where the cost is used as a performance metric for a given combination of inference, stopping and sampling methods. Though the main focus of the paper is optimal stopping, we also aim to provide the background for further developments and discussion in the related field of active learning.