Crowdsourced Data Cleaning

点击翻译按钮获取中文摘要

Data cleaning is the process of detecting and correcting dirty (inconsistent, inaccurate, or incomplete) values from a database. Real-world data is often dirty. Analyses without data cleaning can be very risky, which may result in poor decision-making, and have a significant negative impact on applications. For example, in 2014, Statistics Canada under-reported the country’s job creation in July by more than 41000, based on an analysis of dirty data. This news was reported by hundreds of TV and Internet Media outlets around the world, resulting in various negative effects on Canada. Although there has already been a long line of work on machine-based data cleaning techniques, many cleaning tasks are too challenging for machine only solutions. Recently, the advance of crowdsourcing techniques and platforms (e.g., Amazon Mechanical Turk) provides a highly promising way to involve humans and computers in solving complex problems at low cost. In view of this great opportunity, this proposal will study crowdsourced data cleaning, intelligently combining humans and computers to address challenging data-cleaning problem. This work will not only open up a new research area in the database field, but also benefit a lot of other scientific fields, such as library science or sociology, which often require to conduct data analysis on real-world datasets. Furthermore, with the rise of big data, the world is moving towards a more data-driven environment. Data cleaning has long been considered as a bottleneck for extracting value from data. Crowdsourced data cleaning, which has the potential of significantly improving data quality at low cleaning cost, will have an increasing number of applications in this new environment, such as cleaning customer information for reliable market analysis, and cleaning patients’ medical history for accurate disease diagnosis.