Improving Data Quality in Protected and Dynamic Data Environments

点击翻译按钮获取中文摘要

Poor data quality is a mainstream issue for organizations, and is a pervasive problem in databases, datawarehousing, and Web data. Many data processing tasks such as information integration, information retrieval, and recent initiatives such as Big Data analysis, require various forms of data preparation that assume the data conforms to ideal data distributions. In reality however, this is not often the case. Real data often contains erroneous, incomplete, and inconsistent values. These inconsistencies are magnified by the increasingly large, heterogeneous, and dynamic datasets available today. In order for tools and applications to glean insights from this data, the ability to handle uncertainties and inconsistencies in the data is crucial for building high quality data stores. My work to date has focused on developing techniques that resolve inconsistencies by performing repairs to the data or to the violated constraints in static data environments. In the proposed work, I will study transformative solutions to address the increased need for dynamic, privacy aware data cleaning. Given the increasing size of data stores, and the proliferation of personal data available in modern datasets, my goal is to understand and develop techniques for online data cleaning, while respecting the privacy policies of the data. The proposed techniques will be applicable in organizational and online settings where there may be restricted access to portions of the data, but global data cleaning solutions are required. That is, given limited views of the data, my goal is to find repairs which resolve the inconsistencies, with respect to the full view of the data, by sharing the private information in an obfuscated manner. This work aligns with privacy aware Big Data initiatives, and contributes towards the goal of automated data quality management.