Structured Data Integration with Referring Expressions and Types

点击翻译按钮获取中文摘要

Any enterprise will have access to a large number of structured data sources that are managed by its operational information systems or that can be accessed through external sites on the web. The sources will provide interfaces that can range from low-level file systems storing such artifacts as CSV files, to intermediate-level key-value systems managing very large volumes of operational data, through to high-level relational database systems and NewSQL systems providing an SQL interface to their own structured data. The enterprise will also have an abundance of circumstances where code must be written by its IT personal, its data engineers and its data scientists to access many of these data sources at runtime in order to satisfy emerging integration requirements for its existing information systems, or to satisfy the requirements of new information systems currently under development. Such circumstances are instances of the general problem of structured data integration, the focus of this research program. Structured data integration is the problem of finding and executing query plans that manifest efficient ways to interact with data sources to obtain knowledge of their structured data, and to combine the results of this interaction in a way that, as efficiently as possible, provides answers to user queries coded by developers. The problem becomes particularly challenging when user queries are expressed in a high-level non-procedural language, such as SQL, over an integrating schema. Such a schema, sometimes called a mediated schema or ontology, is tailored entirely to the domain of the enterprise, and is therefore devoid of any knowledge of the data sources themselves.  The increase in developer productivity enabled by data integration technology that provides such a developer interface is now indisputable. There are two subproblems in data integration that are the focus of this research: (1) the selection of suitable languages for capturing so-called physical designs (physical designs augment logical designs with the source schemas of data residing at various data sources, with mapping rules relating the logical design to source schemas, and with knowledge of the capabilities of the data sources themselves),  and (2) the selection of suitable internal languages for expressing user queries and query plans together with the problem of finding query plans given physical designs and user queries. The research program is a continuation of my ongoing work on each of these subproblems, and therefore has a particular focus on the issue of identification in structured data integration, in particular, on how data sources have encoded references to entities of interest to the enterprise and on how the enterprise itself has chosen such means of reference. The development of practical tools for compiling SQL queries over an integrating schema to efficient query plans accessing structured data sources is the ultimate goal and is also well underway.