SHF: Small: MIGS -- Efficiently Evaluating Multiple Iterative Graph Queries

由于图形可以很容易地表示实体和它们之间的关系，因此它们被广泛用于表示从社交网络到生物和大脑网络的领域的大量数据。挖掘大型图以从数据中开发见解通常采取评估迭代图分析查询的形式，这些查询源自图中的不同点，并计算可能依赖于整个图的结构的全局属性值。这些算法都是计算和数据密集型的;由于现实世界图的尺寸大且结构高度不规则，因此优化它们是一项挑战。虽然已经开发了许多用于评估迭代图查询的系统，但它们的重点是最小化单个查询的执行时间，尽管在实践中需要评估许多查询。这项研究的目标是显着提高评估时间的一组查询协同评估他们利用观察，评估个别查询涉及图遍历和计算，大大重叠彼此。这项研究有助于发现新的技术，用于识别和利用这种重叠，以减少冗余的成本和开销与单机和多机平台上的并行执行。构建如此强大的系统可以加速使用图形分析的领域的新发现。此外，更广泛的影响，也导致从培训学生在国家need.Specifically领域，本研究开发系统级优化和算法的创新，一起工作，大大提高了同时评估多个迭代查询的可扩展性。为了保证所产生的技术的多功能性，新的优化和算法正在共享内存多核机器以及多台机器的分布式集群的上下文中实现。通过支持不同类型的查询（点到点和点对所有）和不同形式的图（固定图和流或动态图），也可以实现多功能性。每种查询类型和图形类型都对开发正确并实现高性能的算法提出了独特的挑战。为了利用不同查询的评估过程中执行的工作的重叠，正在开发基于重用的算法，而更新查询的结果，以响应图中的变化，正在设计增量算法。这些方法的目标是在共享内存机器上实现高性能和高系统利用率。为了进一步将性能扩展到大批量查询，正在开发分布式算法以利用分布式集群中多台机器的资源。评估利用来自KONNECT和SNAP等公共数据库的大型图形数据集。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Since graphs can readily represent entities and relationships among them, they are widely used to represent large volumes of data from domains ranging from social networks to biological and brain networks. Mining of large graphs for developing insights from data often takes the form of evaluating iterative graph analytics queries that originate at different points in the graph and compute global property values that are potentially dependent upon the structure of the entire graph. Such algorithms are both compute- and data-intensive; optimizing them is a challenge due to the large size and highly irregular structure of real-world graphs. While many systems for evaluating iterative graph queries have been developed, their focus is on minimizing the execution time of a single query though in practice many queries need to be evaluated. The goal of this research is to dramatically improve evaluation times of a group of queries by synergistically evaluating them to exploit the observation that evaluating individual queries involves graph traversals and computations that greatly overlap with each other. This research helps discover novel techniques for identifying and exploiting such overlap to reduce redundant costs and overheads associated with parallel executions on single-machine and multiple-machine platforms. Building such powerful systems accelerates new discoveries in fields that employ graph analytics. In addition, broader impact also results from training students in an area of national need.Specifically, this research develops both system level optimizations and algorithmic innovations that work together to greatly increase the scalability of simultaneously evaluating multiple iterative queries. To guarantee versatility of techniques produced, the new optimizations and algorithms are being realized both in the context of a shared-memory multicore machine as well as a distributed cluster of multiple machines. Versatility is also achieved by supporting different kinds of queries (point-to-point and point-to-all) and different forms of graphs (fixed graphs and streaming or dynamic graphs). Each query type and graph type pose unique challenges to developing algorithms that are both correct and achieve high-performance. To exploit the overlap of work performed during evaluation of different queries, reuse-based algorithms are being developed while to update the results of queries in response to changes in the graph, incremental algorithms are being devised. These approaches are aimed at enabling high-performance and high system utilization on a shared-memory machine. To further scale performance to large batches of queries, distributed algorithms are being developed to utilize the resources from multiple machines in a distributed cluster. The evaluations utilize large graph data sets from public repositories such as KONNECT and SNAP. The software developed is made available to other researchers over the course of this project.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.