Structured Data Integration with Referring Expressions and Types

任何企业都可以访问由其业务信息系统管理的大量结构化数据源，或者可以通过网络上的外部站点访问这些数据源。这些源代码将提供各种接口，从存储CSV文件等构件的低级文件系统，到管理大量操作数据的中级键值系统，再到为其自己的结构化数据提供SQL接口的高级关系数据库系统和NewSQL系统。企业还将在许多情况下，必须由其IT人员、数据工程师和数据科学家编写代码，以便在运行时访问许多这些数据源，以满足对其现有信息系统的新集成要求，或满足当前正在开发的新信息系统的要求。这些情况都是结构化数据集成的一般问题的例子，这是本研究计划的重点。结构化数据集成是查找和执行查询计划的问题，查询计划显示了与数据源交互以获取其结构化数据的知识的有效方式，并以尽可能高效地为开发人员编码的用户查询提供答案的方式组合了这种交互的结果。当在集成模式上用高级非过程性语言(如SQL)表示用户查询时，这个问题就变得特别具有挑战性。这种模式有时被称为中介模式或本体，它完全针对企业的领域进行定制，因此缺乏对数据源本身的任何了解。提供这样的开发人员界面的数据集成技术提高了开发人员的工作效率，这一点现在是毋庸置疑的。在数据集成中有两个子问题是本研究的重点：(1)选择合适的语言来捕获所谓的物理设计(物理设计利用驻留在不同数据源处的数据的源模式来扩充逻辑设计，使用将逻辑设计与源模式相关联的映射规则，以及利用数据源本身的能力的知识)；(2)选择合适的内部语言来表达用户查询和查询计划，以及寻找给定的物理设计和用户查询的查询计划的问题。该研究计划是我在每个子问题上正在进行的工作的继续，因此特别关注结构化数据集成中的标识问题，特别是数据源如何编码对企业感兴趣的实体的引用，以及企业本身如何选择这种引用方法。最终目标是开发实用工具，用于通过集成模式将SQL查询编译为有效的访问结构化数据源的查询计划，目前也在顺利进行中。

Any enterprise will have access to a large number of structured data sources that are managed by its operational information systems or that can be accessed through external sites on the web. The sources will provide interfaces that can range from low-level file systems storing such artifacts as CSV files, to intermediate-level key-value systems managing very large volumes of operational data, through to high-level relational database systems and NewSQL systems providing an SQL interface to their own structured data. The enterprise will also have an abundance of circumstances where code must be written by its IT personal, its data engineers and its data scientists to access many of these data sources at runtime in order to satisfy emerging integration requirements for its existing information systems, or to satisfy the requirements of new information systems currently under development. Such circumstances are instances of the general problem of structured data integration, the focus of this research program. Structured data integration is the problem of finding and executing query plans that manifest efficient ways to interact with data sources to obtain knowledge of their structured data, and to combine the results of this interaction in a way that, as efficiently as possible, provides answers to user queries coded by developers. The problem becomes particularly challenging when user queries are expressed in a high-level non-procedural language, such as SQL, over an integrating schema. Such a schema, sometimes called a mediated schema or ontology, is tailored entirely to the domain of the enterprise, and is therefore devoid of any knowledge of the data sources themselves.  The increase in developer productivity enabled by data integration technology that provides such a developer interface is now indisputable. There are two subproblems in data integration that are the focus of this research: (1) the selection of suitable languages for capturing so-called physical designs (physical designs augment logical designs with the source schemas of data residing at various data sources, with mapping rules relating the logical design to source schemas, and with knowledge of the capabilities of the data sources themselves),  and (2) the selection of suitable internal languages for expressing user queries and query plans together with the problem of finding query plans given physical designs and user queries. The research program is a continuation of my ongoing work on each of these subproblems, and therefore has a particular focus on the issue of identification in structured data integration, in particular, on how data sources have encoded references to entities of interest to the enterprise and on how the enterprise itself has chosen such means of reference. The development of practical tools for compiling SQL queries over an integrating schema to efficient query plans accessing structured data sources is the ultimate goal and is also well underway.