Integrating third-party and open data with internal corporate databases

将第三方和开放数据与内部企业数据库集成

• 批准号: 542303-2019
• 负责人: Rafiei, Davood
• 金额: $ 3.64万
• 依托单位: University of Alberta
• 依托单位国家: 加拿大
• 项目类别: Collaborative Research and Development Grants
• 财政年份: 2021
• 资助国家: 加拿大
• 起止时间: 2021-01-01 至 2022-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=720659
• 关键词: Integrating; third; party; open; data

Many companies want to better understand the needs of their customers, in an effort to help them reach their goals and to maintain a positive mutual relationship. In a typical setting, information about customers (such as name and address) and their transactions with a company is stored in a dedicated database managed by the company. However, additional information is often available through other sources and the company may have access to some of these resources (e.g. Environics, Wealth, Insurance, etc.) and also to the data that is available publicly. Utilizing third party and public data (collectively referred to as external data) is expected to bring greater value, helping the company move closer to some of its objectives, but also introduces challenges. First, external data is not expected to follow the same set of rules governing the internal data, in terms of the schema, naming of the tables and columns and integrity constraints. Information about an entity can be spread over multiple (and sometimes inconsistent) sources. Second, the identifying detail of an entity can differ between sources (e.g. due to spelling and other variations) and it may not be possible to join the data from such sources with certainty. Third, data may have some spatial or temporal attributes. For example, the content may be associated to a geographical location. Understanding these attributes may be important in resolving some of the ambiguities.The primary objective of this research is to study the challenges in integrating third-party and open data with data residing inside an organization. We will investigate robust system architectures and efficient and scalable algorithms that support integration at different stages including data cleaning, schema mapping, and query processing. This project will be developed in partnership with Servus Credit Union, an Alberta-based company that deals with the aforementioned integration and cleaning challenges to support down-the-stream data analytics applications and services the company runs.

许多公司希望更好地了解客户的需求，以帮助他们实现目标并保持积极的相互关系。在典型的设置中，有关客户的信息(如姓名和地址)以及他们与公司的交易信息存储在公司管理的专用数据库中。但是，其他信息通常可以通过其他来源获得，公司可能可以访问其中一些资源(例如，环境、财富、保险等)。以及公开可用的数据。利用第三方和公共数据(统称为外部数据)有望带来更大的价值，帮助公司更接近其一些目标，但也带来了挑战。首先，在模式、表和列的命名以及完整性约束方面，外部数据不应遵循管理内部数据的相同规则集。有关实体的信息可以分布在多个(有时是不一致的)来源上。其次，不同来源的实体的识别细节可能不同(例如，由于拼写和其他变化)，并且可能无法确定地连接来自这些来源的数据。第三，数据可能具有一些空间或时间属性。例如，内容可以与地理位置相关联。了解这些属性对于解决一些歧义可能很重要。本研究的主要目标是研究将第三方和开放数据与驻留在组织内部的数据进行集成的挑战。我们将研究健壮的系统架构和支持不同阶段集成的高效且可扩展的算法，包括数据清理、模式映射和查询处理。该项目将与Servus Credit Union合作开发，Servus Credit Union是一家总部位于艾伯塔省的公司，负责应对上述集成和清理挑战，以支持该公司运营的下游数据分析应用程序和服务。