Crowdsourced Data Cleaning

众包数据清理

• 批准号: RGPIN-2016-05555
• 负责人: Wang, Jiannan
• 金额: $ 2.62万
• 依托单位: Simon Fraser University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2020
• 资助国家: 加拿大
• 起止时间: 2020-01-01 至 2021-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=709264
• 关键词: Crowdsourced; Data; Cleaning

Data cleaning is the process of detecting and correcting dirty (inconsistent, inaccurate, or incomplete) values from a database. Real-world data is often dirty. Analyses without data cleaning can be very risky, which may result in poor decision-making, and have a significant negative impact on applications. For example, in 2014, Statistics Canada under-reported the country's job creation in July by more than 41000, based on an analysis of dirty data. This news was reported by hundreds of TV and Internet Media outlets around the world, resulting in various negative effects on Canada. Although there has already been a long line of work on machine-based data cleaning techniques, many cleaning tasks are too challenging for machine only solutions. Recently, the advance of crowdsourcing techniques and platforms (e.g., Amazon Mechanical Turk) provides a highly promising way to involve humans and computers in solving complex problems at low cost. In view of this great opportunity, this proposal will study crowdsourced data cleaning, intelligently combining humans and computers to address challenging data-cleaning problem. This work will not only open up a new research area in the database field, but also benefit a lot of other scientific fields, such as library science or sociology, which often require to conduct data analysis on real-world datasets. Furthermore, with the rise of big data, the world is moving towards a more data-driven environment. Data cleaning has long been considered as a bottleneck for extracting value from data. Crowdsourced data cleaning, which has the potential of significantly improving data quality at low cleaning cost, will have an increasing number of applications in this new environment, such as cleaning customer information for reliable market analysis, and cleaning patients' medical history for accurate disease diagnosis.

数据清理是从数据库中检测和纠正脏（不一致，不准确或不完整）值的过程。真实世界的数据往往是肮脏的。没有数据清理的分析可能非常危险，可能导致决策失误，并对应用程序产生重大负面影响。例如，2014年，加拿大统计局根据对肮脏数据的分析，少报了该国7月份创造的就业机会超过4.1万个。这一消息被世界各地数百家电视和网络媒体报道，对加拿大造成了各种负面影响。 尽管基于机器的数据清理技术已经有很长一段时间的工作，但许多清理任务对于仅使用机器的解决方案来说太具有挑战性了。最近，众包技术和平台的进步（例如，Amazon Mechanical Turk）提供了一种非常有前途的方式，让人类和计算机以低成本解决复杂问题。鉴于这个巨大的机会，该提案将研究众包数据清理，智能地将人类和计算机结合起来，以解决具有挑战性的数据清理问题。 这项工作不仅将在数据库领域开辟一个新的研究领域，而且还将有益于许多其他科学领域，如图书馆学或社会学，这些领域通常需要对真实世界的数据集进行数据分析。此外，随着大数据的兴起，世界正在走向一个更加数据驱动的环境。数据清洗一直被认为是从数据中提取价值的瓶颈。众包数据清理有可能以较低的清理成本显著提高数据质量，在这种新环境中将有越来越多的应用，例如清理客户信息以进行可靠的市场分析，清理患者的病史以进行准确的疾病诊断。