Automatic Generation of Normalized Relational Schemas from Nested Key-Value Data
Automatic Generation of Normalized Relational Schemas from Nested Key-Value Data
复制标题
DOI：
10.1145/2882903.2882924
复制
发表时间：
2016-06
期刊：
影响因子：
--
通讯作者：
M. DiScala;D. Abadi
中科院分区：
文献类型：
--
作者：
M. DiScala;D. Abadi
随着应用程序开发人员选择避免关系模型带来的僵化，自描述键值数据格式（例如 JSON）变得越来越流行。为这些自描述格式设计的数据库系统（例如 MongoDB）鼓励用户使用非规范化的、高度嵌套的数据模型，以便不需要预定义或标准化记录和其他模式信息之间的关系。此类数据模型会增加长期开发的复杂性，因为它们缺乏明确的实体和关系跟踪，给不熟悉数据集的新开发人员带来了负担。此外，此类数据布局中存在的大量数据重复可能会导致更新异常和扫描性能不佳，从而降低数据分析的质量和性能。在本文中，我们提出了一种算法，可以自动将 NoSQL 系统中常见的非规范化嵌套数据转换为可以存储在标准 RDBMS 中的传统关系数据。此过程包括模式生成算法，该算法发现非规范化数据集的属性之间的关系，以便将这些属性组织到关系表中。它还包括一个匹配算法，用于发现表示重叠实体的属性集并将这些属性集合并在一起。这些算法减少了数据重复，允许使用针对关系数据的数据分析工具，加速对数据的扫描密集型算法，并帮助用户获得对复杂嵌套数据集的语义理解。
Self-describing key-value data formats such as JSON are becoming increasingly popular as application developers choose to avoid the rigidity imposed by the relational model. Database systems designed for these self-describing formats, such as MongoDB, encourage users to use denormalized, heavily nested data models so that relationships across records and other schema information need not be predefined or standardized. Such data models contribute to long-term development complexity, as their lack of explicit entity and relationship tracking burdens new developers unfamiliar with the dataset. Furthermore, the large amount of data repetition present in such data layouts can introduce update anomalies and poor scan performance, which reduce both the quality and performance of analytics over the data. In this paper we present an algorithm that automatically transforms the denormalized, nested data commonly found in NoSQL systems into traditional relational data that can be stored in a standard RDBMS. This process includes a schema generation algorithm that discovers relationships across the attributes of the denormalized datasets in order to organize those attributes into relational tables. It further includes a matching algorithm that discovers sets of attributes that represent overlapping entities and merges those sets together. These algorithms reduce data repetition, allow the use of data analysis tools targeted at relational data, accelerate scan-intensive algorithms over the data, and help users gain a semantic understanding of complex, nested datasets.