Structured Data Integration with Referring Expressions and Types

任何企业都可以访问大量的结构化数据源，这些数据源由其运营信息系统管理，或者可以通过Web上的外部站点访问。这些数据源将提供接口，从存储CSV文件等工件的低级文件系统，到管理大量操作数据的中级键值系统，再到高级关系数据库系统和为自己的结构化数据提供SQL接口的NewSQL系统。企业还将有大量的情况，其中代码必须由其IT人员、其数据工程师和其数据科学家编写，以在运行时访问这些数据源中的许多数据源，以满足其现有信息系统的新兴集成需求，或者满足当前正在开发的新信息系统的需求。这种情况是结构化数据集成一般问题的实例，也是本研究计划的重点。结构化数据集成是寻找和执行查询计划的问题，这些查询计划表明与数据源交互以获得其结构化数据的知识的有效方式，并且以尽可能有效地为开发人员编码的用户查询提供答案的方式联合收割机组合这种交互的结果。当用户查询在一个集成模式上用高级非过程语言（如SQL）表示时，这个问题变得特别具有挑战性。这样的模式，有时称为中介模式或本体，完全针对企业的领域定制，因此缺乏数据源本身的任何知识。 通过提供这种开发人员界面的数据集成技术，开发人员生产力的提高现在是无可争议的。数据集成中有两个子问题是本研究的重点：（1）选择合适的语言来捕捉所谓的物理设计（物理设计利用驻留在各种数据源处的数据的源模式、利用将逻辑设计与源模式相关联的映射规则以及利用数据源自身的能力的知识来增强逻辑设计），以及（2）选择合适的内部语言来表达用户查询和查询计划，以及在给定物理设计和用户查询的情况下找到查询计划的问题。该研究计划是我正在进行的工作，对这些子问题的延续，因此有一个特别关注的问题，在结构化数据集成的识别，特别是，数据源如何编码引用实体感兴趣的企业和企业本身如何选择这样的参考手段。开发实用的工具，用于在集成模式上编译SQL查询，以有效地访问结构化数据源的查询计划，是最终目标，也正在进行中。

Any enterprise will have access to a large number of structured data sources that are managed by its operational information systems or that can be accessed through external sites on the web. The sources will provide interfaces that can range from low-level file systems storing such artifacts as CSV files, to intermediate-level key-value systems managing very large volumes of operational data, through to high-level relational database systems and NewSQL systems providing an SQL interface to their own structured data. The enterprise will also have an abundance of circumstances where code must be written by its IT personal, its data engineers and its data scientists to access many of these data sources at runtime in order to satisfy emerging integration requirements for its existing information systems, or to satisfy the requirements of new information systems currently under development. Such circumstances are instances of the general problem of structured data integration, the focus of this research program. Structured data integration is the problem of finding and executing query plans that manifest efficient ways to interact with data sources to obtain knowledge of their structured data, and to combine the results of this interaction in a way that, as efficiently as possible, provides answers to user queries coded by developers. The problem becomes particularly challenging when user queries are expressed in a high-level non-procedural language, such as SQL, over an integrating schema. Such a schema, sometimes called a mediated schema or ontology, is tailored entirely to the domain of the enterprise, and is therefore devoid of any knowledge of the data sources themselves.  The increase in developer productivity enabled by data integration technology that provides such a developer interface is now indisputable. There are two subproblems in data integration that are the focus of this research: (1) the selection of suitable languages for capturing so-called physical designs (physical designs augment logical designs with the source schemas of data residing at various data sources, with mapping rules relating the logical design to source schemas, and with knowledge of the capabilities of the data sources themselves),  and (2) the selection of suitable internal languages for expressing user queries and query plans together with the problem of finding query plans given physical designs and user queries. The research program is a continuation of my ongoing work on each of these subproblems, and therefore has a particular focus on the issue of identification in structured data integration, in particular, on how data sources have encoded references to entities of interest to the enterprise and on how the enterprise itself has chosen such means of reference. The development of practical tools for compiling SQL queries over an integrating schema to efficient query plans accessing structured data sources is the ultimate goal and is also well underway.