Termite: a system for tunneling through heterogeneous data
Termite: a system for tunneling through heterogeneous data
复制标题
DOI：
10.1145/3329859.3329877
复制
发表时间：
2019-03
期刊：
影响因子：
--
通讯作者：
R. Fernandez;S. Madden
中科院分区：
文献类型：
--
作者：
R. Fernandez;S. Madden
数据驱动的分析在几乎每个现代组织中都很重要。然而，大多数数据未得到充分利用，因为它们仍然被锁定在组织内部的孤岛中；大型组织拥有数千个数据库和数十亿个文件，这些数据库和文件没有集成到一个可查询的存储库中。尽管数据库社区进行了40多年的持续努力，但数据集成仍然是一项开放的挑战。在本文中，我们倡导一种不同的方法：我们的目标不是试图推断一个共同的模式，而是为不同的、不同的数据找到另一个共同的表示形式。具体地说，我们主张将所有实体、行、列和段落表示为点的嵌入(即向量空间)。在嵌入中，点之间的距离表示它们之间的关联程度。我们展示了Termite，这是我们构建的一个原型，目的是从数据中学习最佳嵌入方式。因为可以学习最佳表示，所以这使Termite可以避免与传统数据集成任务相关的大量人工工作。在Termite之上，我们实现了Termite-Join操作符，它允许人们识别相关概念，即使这些概念存储在具有不同模式的数据库和非结构化数据(如文本文件、网页等)中。最后，我们通过用户研究显示了我们的原型的初步评估结果，并描述了我们确定的未来方向。
Data-driven analysis is important in virtually every modern organization. Yet, most data is underutilized because it remains locked in silos inside of organizations; large organizations have thousands of databases, and billions of files that are not integrated together in a single, queryable repository. Despite 40+ years of continuous effort by the database community, data integration still remains an open challenge. In this paper, we advocate a different approach: rather than trying to infer a common schema, we aim to find another common representation for diverse, heterogeneous data. Specifically, we argue for an embedding (i.e., a vector space) in which all entities, rows, columns, and paragraphs are represented as points. In the embedding, the distance between points indicates their degree of relatedness. We present Termite, a prototype we have built to learn the best embedding from the data. Because the best representation is learned, this allows Termite to avoid much of the human effort associated with traditional data integration tasks. On top of Termite, we have implemented a Termite-Join operator, which allows people to identify related concepts, even when these are stored in databases with different schemas and in unstructured data such as text files, webpages, etc. Finally, we show preliminary evaluation results of our prototype via a user study, and describe a list of future directions we have identified.