EAGER: Developing scalable benchmark mini-apps for graph engine comparison

在过去的十年中，我们看到了超大型、非结构化和动态数据集的增长，这些数据被称为大数据。然而，人们越来越希望不仅提取这些事实的集合的特定属性，而且还提取该数据中的底层实体之间的关系。例子来自现代生活的广泛样本：生物信息学、金融、推荐系统、网络和国家安全以及社交网络。图已经成为表达这些问题的有价值和有生产力的范例，其中图是一组对象（顶点）的集合，其中一些对象对通过表示两者之间的某种关系的链接（边）连接。在过去的十年中，对图的支持出现了爆炸式的增长，执行模型和目标适用性有很大的不同。尽管已经提出了许多图形基准，但只有一个基准对来自多个平台的性能数据进行了严格的积累（www.graph500.org）。计算是在整个静态图上进行的，而在真实的世界中，应用程序中的更新数据会流入大型持久化图中，并且可能会同时进行非常多的小目标查询。考虑到使用图编程范式比传统编程（特别是对于并行系统）预期的生产率提高，拥有可用于跨范式比较的通用迷你应用程序变得越来越重要。此外，考虑到图形大小的持续增长，了解底层图形引擎如何在目标图形的大小和类型以及它们可以支持的并行和并发的数量和组合方面进行扩展是很重要的。与商业和政府研究实验室合作，主要目标是定义一组迷你应用程序，这些应用程序反映了比今天的简单基准更复杂的复杂现实应用程序，将这些迷你应用程序转换为现有的主要图形包，然后在各种并行系统上运行它们。更广泛的影响可能是重大的。识别相关的迷你应用程序以及它们在不同系统中的表现将有助于了解如何以更可扩展的方式编写更完整的图形应用程序，以及哪些编程系统和平台最适合。还可以预期的是，并非所有的迷你应用程序都可以在所有当前的范例中表达，这为这些范例的开发人员提供了关于可表达性问题的见解。考虑到这种图形包的相对婴儿期，这种洞察力现在可以从根本上提高它们在未来真实的应用中的适用性。

The last decade has seen the growth of extremely large, unstructured, and dynamic data sets, loosely termed Big Data. However, there is a growing desire to extract not just specific properties of collections of such facts, but also relationships between the underlying entities in that data. Examples come from a broad swatch of modern life: bioinformatics, financial, recommendation systems, cyber and national security, and social networks. Graphs have emerged as a valuable and productive paradigm for expressing such problems, where a graph is a collection of a set of objects (vertices) where some pairs of objects are connected by links (edges) that represent some relation between the two. In the last decade there has been an explosion in support for graphs, with widely differing execution models and targeted applicability. Although numerous graph benchmarks have been proposed, only one has had a rigorous accumulation of performance data from multiple platforms (www.graph500.org). Computation is over a whole static graph, whereas the real world sees applications where update data is streaming into large persistent graphs, and very many small targeted queries may be in progress at once.Given the expected productivity increase of using a graph programming paradigm over conventional programming, especially for parallel systems, it is of growing importance to have common mini-apps that can be used for cross-paradigm comparisons. Also, given the continued increase in graph sizes, it is important to understand how the underlying graph engines scale both in the size and type of the target graphs and in the amount and mix of parallelism and concurrency they can support.This project addresses this need. In collaboration with commercial and government research labs, the primary objective is on defining a set of mini-apps that reflect complex real-world applications more sophisticated than today's simple benchmarks, converting these mini-apps to the existing major graph packages, and then running them on a wide range of parallel systems. The wider impact can be significant. Identification of relevant mini-apps and how they perform across different systems will provide insight into both how to write more complete graph applications in more scalable ways, and which aspects of which programming systems and platforms are best suited. It is also expected that not all mini-apps will be expressible in all the current paradigms, providing insight to the developers of those paradigms on expressibility issues. Given the relative infancy of such graph packages such insight now can radically improve their applicability to real applications in the future.