Data Integration: from NoSQL to Main Memory Data Sources

考虑到现代企业将可以访问大量由内部操作信息系统管理或可以通过Web上的外部站点访问的结构化数据源。这些源代码将提供各种接口，从存储CSV文件等人工产物的低级文件系统，到管理非常大量的操作数据的中级键值系统，再到通过SQL访问结构化数据的高级关系数据库系统。*还考虑到，企业将在许多情况下必须由开发人员编写代码，以便在运行时访问许多这样的数据源，以满足对其现有信息系统的新的集成要求，或满足当前正在开发的新信息系统的要求。这种情况是数据集成的一般问题的例子，数据集成是本研究议程的重点。*数据集成是寻找和执行查询计划的问题，这些查询计划显示了与数据源交互的有效方式，以便获得关于其结构化数据的知识，并以尽可能高效地产生对开发人员编码的用户查询的答案的方式组合这种交互的结果。当用户查询以高级非过程性语言(如SQL)表达而不是逻辑设计时，这个问题就变得特别具有挑战性。这种设计有时被称为中介模式或本体，是对数据的特定于问题或领域的评价，完全不需要了解数据源本身。提供这种开发人员界面的数据集成技术提高了开发人员的生产率，这一点现在是毋庸置疑的；数据集成中有三个子问题：(1)选择合适的语言来捕获所谓的物理设计(物理设计利用驻留在各种数据源处的数据的源模式来扩充逻辑设计，具有将逻辑设计与源模式相关联的映射规则，并且具有数据源本身的能力的知识)，(2)选择用于表达用户查询和查询计划的合适的语言，以及(3)在给定物理设计和用户查询的情况下寻找查询计划的问题。*本项目是我在每个子问题上正在进行的工作的继续，因此也采用了这样的假设，即一阶逻辑(FOL)和由SQL标准假定的所谓的封闭世界语义是数据集成的基本理论和技术的适当起点或基础。事实上，FOL位于关系数据库系统和数据的关系模型的核心，并通过关系代数对SQL查询语言的发展产生了深远的影响。

Consider that a modern enterprise will have access to a large number of structured data sources that are managed by internal operational information systems or that can be accessed through external sites on the web. The sources will provide interfaces that can range from low-level file systems storing such artifacts as CSV files, to intermediate-level key-value systems managing very large volumes of operational data, through to high-level relational database systems providing access to structured data via SQL.***Also consider that the enterprise will have an abundance of circumstances where code must to be written by developers to access many of these data sources at runtime in order to satisfy emerging integration requirements for its existing information systems, or to satisfy the requirements of new information systems currently under development. Such circumstances are instances of the general problem of data integration, the focus of this research agenda.***Data integration is the problem of finding and executing query plans that manifest efficient ways to interact with data sources so as to obtain knowledge of their structured data, and to combine the results of this interaction in a way that, as efficiently as possible, yields the answers to user queries coded by developers. The problem becomes particularly challenging when user queries are expressed in a high-level non-procedural language, such as SQL, over a logical design. Such a design, sometimes called a mediated schema or ontology, is a problem or domain specific appreciation of the data entirely devoid of any knowledge of the data sources themselves. The increase in developer productivity enabled by data integration technology that provides such a developer interface is now indisputable; witness that relational technology continues to be a growing multi-billion dollar industry.***There are three subproblems in data integration: (1) the selection of suitable languages for capturing so-called physical designs (physical designs augment logical designs with the source schemas of data residing at various data sources, with mapping rules relating the logical design to source schemas, and with knowledge of the capabilities of the data sources themselves), (2) the selection of suitable languages for expressing user queries and query plans, and (3) the problem of finding query plans given physical designs and user queries.***This project is a continuation of my ongoing work on each of these subproblems, and therefore also adopts the hypothesis that first order logic (FOL) and so-called closed world semantics assumed by the SQL standard are an appropriate starting point or foundation for an underlying theory and technology of data integration. Indeed, FOL lies at the heart of relational database systems and the relational model of data, and has had a profound influence, via relational algebra, on the development of the SQL query language.**