Scaling-Out Streaming Graph Processing

横向扩展流图处理

• 批准号: 538924-2019
• 负责人: Ozsu, MTamer
• 金额: $ 6.86万
• 依托单位: University of Waterloo
• 依托单位国家: 加拿大
• 项目类别: Collaborative Research and Development Grants
• 财政年份: 2019
• 资助国家: 加拿大
• 起止时间: 2019-01-01 至 2020-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=692517
• 关键词: Scaling; Out; Streaming; Graph; Processing

Graphs are natural representation for complex, inter-structured data arising in many domains and applications ranging from bioinformatics to real-time fraud-detection. Many of these real-world graphs are massive in size and continuously evolve overtime - edges are streaming at a very high rate. These streaming graphs pose unique data management challenges as they cannot be feasibly stored locally, and it is difficult to answer queries in real-time. The existing research and commercial systems are not able to handle these graphs and workloads. The primary objective of this research is to investigate robust system architectures and design a Streaming Graph Management System (SGMS) that can process a large number of persistent queries on large streaming graphs. In particular, we intend to study load balancing and distributed query processing techniques for scale-out architectures as well as single-pass, non-blocking operators for persistent query execution.

图是复杂的，结构化的数据在许多领域和应用程序中产生的自然表示，从生物信息学到实时欺诈检测。这些真实世界的图中有许多是巨大的，并且随着时间的推移不断演变-边缘以非常高的速率流动。这些流图带来了独特的数据管理挑战，因为它们无法在本地存储，并且难以实时回答查询。现有的研究和商业系统无法处理这些图形和工作负载。本研究的主要目标是研究健壮的系统架构，并设计一个流图管理系统（SGMS），可以处理大量的持久查询大型流图。特别是，我们打算研究负载平衡和分布式查询处理技术的横向扩展架构，以及单通道，非阻塞操作符的持久查询执行。