Looking Back to Look Forward: Explaining and Exploring Changes in Data

回顾过去展望未来：解释和探索数据的变化

• 批准号: RGPIN-2020-05711
• 负责人: Chiang, Fei
• 金额: $ 2.99万
• 依托单位: McMaster University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=750426
• 关键词: Looking; Back; Look; Forward; Explaining

Databases are collecting data at an unprecedented rate, storing information such as financial and e-commerce transactions, environmental sensor readings, and social network interactions.  Applications in finance, retail, healthcare, and environmental monitoring operate in highly dynamic data environments that capture user and environmental changes. Data is not static, and naturally evolve according to such activities that elicit change in the data and meta-data (schema). Although existing database systems are designed for storage, management, and efficient querying of data, features to manage and understand the cause of changes to the data are limited to database transaction logs, triggers that react to specific value changes, and statistics that are re-generated for query optimization.   By understanding the underlying cause of changes in the data, data analysts are able to discover change patterns, identify events that trigger large-scale changes, and explain the lineage of data and schema evolution. This provides analysts with vital information to help detect abnormal and malicious activity, assess trust for data quality, or to identify trends and correlations to predict future changes. Unfortunately, users are often unaware of the types of changes they are seeking, the criteria for abnormality, and have limited tools to understand how, when and why data changes occur.  The proposed research program aims to address the above challenges, and make significant impact along three lines. First, it will advance the state-of-the-art in developing models for change management, prediction, and data cleaning from anomalous and unfair updates.  This enables change analysis that increase the predictive accuracy for resource allocation, inventory planning, and system monitoring applications; improving the efficiency and increasing the global competitiveness of Canadian firms.  Second, it will develop techniques to identify spatial and temporal interactions between changes, their origins, and their propagation. Third, it will develop a system to enable users to measure and understand the consequential effect of changes in their system.  The long-term goal of the research program is to co-develop the foundations and the computational models for data change. The short-term goal is to design a change exploration and explanation system to improve data quality.  With almost half of new records containing at least one critical error, understanding data change and its impact on data quality are urgently needed.  Our system will help users and businesses understand how data evolves, to derive insights from their data faster, and with greater accuracy and timeliness.

数据库正在以前所未有的速度收集数据，存储金融和电子商务交易、环境传感器读数和社会网络互动等信息。金融、零售、医疗保健和环境监测领域的应用程序在高度动态的数据环境中运行，这些数据环境可以捕捉用户和环境变化。数据不是静态的，它会根据引起数据和元数据（模式）变化的活动自然演变。尽管现有的数据库系统是为数据的存储、管理和高效查询而设计的，但用于管理和理解数据更改原因的功能仅限于数据库事务日志、对特定值更改作出反应的触发器以及为查询优化而重新生成的统计信息。通过了解数据中变化的潜在原因，数据分析人员能够发现变化模式，识别触发大规模变化的事件，并解释数据和模式演变的沿袭。这为分析人员提供了重要的信息，以帮助检测异常和恶意活动，评估数据质量的信任，或识别趋势和相关性，以预测未来的变化。不幸的是，用户通常不知道他们正在寻找的变化类型、异常的标准，并且只有有限的工具来理解数据变化是如何、何时以及为什么发生的。拟议的研究计划旨在解决上述挑战，并在三个方面产生重大影响。首先，它将在开发变更管理、预测和从异常和不公平更新中清理数据的模型方面推进最先进的技术。这使得变更分析能够提高资源分配、库存计划和系统监控应用程序的预测准确性；提高加拿大企业的效率和全球竞争力。其次，它将开发技术来识别变化，它们的起源和传播之间的空间和时间相互作用。第三，它将开发一个系统，使用户能够度量和理解系统变更的相应影响。该研究项目的长期目标是共同开发数据变化的基础和计算模型。短期目标是设计一个变化探索和解释系统，以提高数据质量。由于几乎一半的新记录包含至少一个关键错误，因此迫切需要了解数据更改及其对数据质量的影响。我们的系统将帮助用户和企业了解数据是如何演变的，从而更快、更准确、更及时地从数据中获得见解。