III: Small: RUI: Improving Data Quality and Data Mining Using Noisy Micro-Outsourcing

机器学习目前为构建预测模型提供了最具成本效益的方法之一（例如，用于对每天生成的数百万条消息、新闻文章和博客进行分类的分类器）。然而，在这种情况下，机器学习方法的有效使用受到训练语料库可用性的限制（即，已标记为相应类别的代表性实例集）。在标记数据稀缺或获取昂贵的领域，迫切需要一种经济有效的方法来选择性地获取用于使用机器学习训练预测模型的数据样本的标签。该项目探索利用低成本的微外包系统（如亚马逊的土耳其机械）的新技术，吸引来自世界各地的大量工人获取用于构建训练语料库的实例标签。目前，人们对如何利用微外包获得的多噪声标签知之甚少。需要先进的技术来利用微型外包的低成本优势，以提高数据质量和根据现有数据建立的模型的质量。它探索了利用不同标注者给实例的多个标签的新方法。它还扩展了主动学习技术，用于主动选择要标记的样本，以考虑已经从标记器池中获得的多组标签。在微外包环境中主动选择要标记的数据实例的技术进步可以显著提高用于在广泛应用中构建预测模型的数据质量，包括基因注释、图像注释、文本分类、情感分析和推荐系统，其中未标记的数据丰富，而标记的数据稀疏。该项目将为中阿肯色大学（University of Central Arkansas）的学生提供研究机会，这是一所主要的本科院校，并有助于扩大STEM管道。有关该项目的更多信息可以在http://sun0.cs.uca.edu/~ssheng/上找到。

Machine learning currently offers one of the most cost-effective approaches to building predictive models (e.g., classifiers for categorizing the millions of messages, news articles, and blogs that are generated every day). However, the effective use of machine learning methods in such settings is limited by the availability of a training corpus (i.e., a representative set of instances that have been labeled with the correponding categories). In domains where labeled data are scarce or expensive to acquire, there is an urgent need for cost-effective approaches to selectively acquiring labels for data samples used to train predictive models using machine learning. This project explores novel techniques that take advantage of the low cost of micro-outsourcing using systems such as Amazon's mechanical Turk, to engage a large number of workers from around the world for acquiring the labels of instances to be used to construct the training corpus. There is currently little understanding of how to utilize the multiple noisy labels obtained using micro-outsourcing. There is a need for advanced techniques for taking advantage of the low cost of micro-outsourcing in order to improve data quality and the quality of models built from the available data. It explores novel approaches for utilizing multiple labels given to an instance by different labelers. It also extends active learning techniques for active selection of samples to be labeled to take into account the multi-sets of labels that have been already obtained from a pool of labelers. Advances in techniques for active selection of data instances to be labeled in a micro-outsourcing setting can significantly improve the quality of data used to build predictive models in a broad range of applications, including gene annotation, image annotation, text classification, sentiment analysis, and recommender systems, where unlabeled data are plentiful yet labeled data are sparse. The project will provide research opportunities for students at University of Central Arkansas, a primarily undergraduate institution and help expand the STEM pipeline. Additional information about the project can be found at: http://sun0.cs.uca.edu/~ssheng/.