III: Small: RUI: Improving Data Quality and Data Mining Using Noisy Micro-Outsourcing

机器学习目前提供了构建预测模型的最具成本效益的方法之一（例如，用于对每天生成的数百万条消息、新闻文章和博客进行分类的分类器）。然而，机器学习方法在这样的设置中的有效使用受到训练语料库的可用性的限制（即，已经用相应类别标记的代表性实例集）。 在标记数据稀缺或获取昂贵的领域，迫切需要具有成本效益的方法来选择性地获取用于使用机器学习训练预测模型的数据样本的标签。该项目探索了新的技术，利用微外包的低成本，使用亚马逊的mechanical Turk等系统，吸引来自世界各地的大量工人来获取用于构建训练语料库的实例标签。目前很少有人了解如何利用多个嘈杂的标签获得微外包。需要有先进的技术来利用微型外包的低成本，以提高数据质量和根据现有数据建立的模型的质量。它探索了利用不同标签器给一个实例的多个标签的新方法。它还扩展了主动学习技术，用于主动选择要标记的样本，以考虑已经从标记器池中获得的多组标签。在微外包环境中主动选择要标记的数据实例的技术的进步可以显着提高用于在广泛的应用中构建预测模型的数据的质量，包括基因注释，图像注释，文本分类，情感分析和推荐系统，其中未标记的数据是丰富的，但标记的数据是稀疏的。该项目将为中央阿肯色州大学的学生提供研究机会，这是一所主要的本科院校，并有助于扩大STEM管道。有关该项目的更多信息，请访问：http://sun0.cs.uca.edu/~ssheng/。

Machine learning currently offers one of the most cost-effective approaches to building predictive models (e.g., classifiers for categorizing the millions of messages, news articles, and blogs that are generated every day). However, the effective use of machine learning methods in such settings is limited by the availability of a training corpus (i.e., a representative set of instances that have been labeled with the correponding categories). In domains where labeled data are scarce or expensive to acquire, there is an urgent need for cost-effective approaches to selectively acquiring labels for data samples used to train predictive models using machine learning. This project explores novel techniques that take advantage of the low cost of micro-outsourcing using systems such as Amazon's mechanical Turk, to engage a large number of workers from around the world for acquiring the labels of instances to be used to construct the training corpus. There is currently little understanding of how to utilize the multiple noisy labels obtained using micro-outsourcing. There is a need for advanced techniques for taking advantage of the low cost of micro-outsourcing in order to improve data quality and the quality of models built from the available data. It explores novel approaches for utilizing multiple labels given to an instance by different labelers. It also extends active learning techniques for active selection of samples to be labeled to take into account the multi-sets of labels that have been already obtained from a pool of labelers. Advances in techniques for active selection of data instances to be labeled in a micro-outsourcing setting can significantly improve the quality of data used to build predictive models in a broad range of applications, including gene annotation, image annotation, text classification, sentiment analysis, and recommender systems, where unlabeled data are plentiful yet labeled data are sparse. The project will provide research opportunities for students at University of Central Arkansas, a primarily undergraduate institution and help expand the STEM pipeline. Additional information about the project can be found at: http://sun0.cs.uca.edu/~ssheng/.