Improving Data Quality in Protected and Dynamic Data Environments

提高受保护的动态数据环境中的数据质量

• 批准号: 435477-2013
• 负责人: Chiang, Fei
• 金额: $ 1.46万
• 依托单位: McMaster University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2019
• 资助国家: 加拿大
• 起止时间: 2019-01-01 至 2020-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=677180
• 关键词: Improving; Data; Quality; Protected; Dynamic

Poor data quality is a mainstream issue for organizations, and is a pervasive problem in databases, datawarehousing, and Web data. Many data processing tasks such as information integration, information retrieval, and recent initiatives such as Big Data analysis, require various forms of data preparation that assume the data conforms to ideal data distributions. In reality however, this is not often the case. Real data often contains erroneous, incomplete, and inconsistent values. These inconsistencies are magnified by the increasingly large, heterogeneous, and dynamic datasets available today. In order for tools and applications to glean insights from this data, the ability to handle uncertainties and inconsistencies in the data is crucial for building high quality data stores. My work to date has focused on developing techniques that resolve inconsistencies by performing repairs to the data or to the violated constraints in static data environments. In the proposed work, I will study transformative solutions to address the increased need for dynamic, privacy aware data cleaning. Given the increasing size of data stores, and the proliferation of personal data available in modern datasets, my goal is to understand and develop techniques for online data cleaning, while respecting the privacy policies of the data. The proposed techniques will be applicable in organizational and online settings where there may be restricted access to portions of the data, but global data cleaning solutions are required. That is, given limited views of the data, my goal is to find repairs which resolve the inconsistencies, with respect to the full view of the data, by sharing the private information in an obfuscated manner. This work aligns with privacy aware Big Data initiatives, and contributes towards the goal of automated data quality management.

数据质量差是组织的主流问题，也是数据库、数据仓库和Web数据中普遍存在的问题。许多数据处理任务，如信息集成，信息检索，以及最近的举措，如大数据分析，需要各种形式的数据准备，假设数据符合理想的数据分布。然而，在现实中，情况并非如此。真实的数据通常包含错误、不完整和不一致的值。这些不一致性被当今日益庞大、异构和动态的数据集放大。 为了让工具和应用程序从这些数据中收集见解，处理数据中的不确定性和不一致性的能力对于构建高质量的数据存储至关重要。 到目前为止，我的工作主要集中在开发技术，通过对静态数据环境中的数据或违反的约束进行修复来解决不一致性。 在拟议的工作中，我将研究变革性的解决方案，以满足对动态、隐私感知数据清理的日益增长的需求。 考虑到数据存储的规模越来越大，以及现代数据集中可用的个人数据的激增，我的目标是了解和开发在线数据清洗技术，同时尊重数据的隐私政策。 拟议的技术将适用于组织和在线设置，其中可能存在对部分数据的限制访问，但需要全局数据清理解决方案。也就是说，鉴于数据的视图有限，我的目标是通过以模糊的方式共享私有信息，找到解决数据完整视图不一致的修复方法。 这项工作与隐私意识大数据计划保持一致，并有助于实现自动化数据质量管理的目标。