EAGER: Developing scalable benchmark mini-apps for graph engine comparison

在过去的十年里，我们看到了巨大的、非结构化的和动态的数据集的增长，这些数据集被松散地称为大数据。然而，人们越来越希望不仅提取这些事实集合的特定属性，还提取数据中底层实体之间的关系。例子来自现代生活的广泛领域：生物信息学、金融、推荐系统、网络和国家安全，以及社交网络。图已经成为表达这些问题的一种有价值和富有成效的范例，其中图是一组对象（顶点）的集合，其中一些对象对通过表示两者之间某种关系的链接（边）连接起来。在过去十年中，对图的支持出现了爆炸式增长，执行模型和目标适用性存在很大差异。尽管已经提出了许多图形基准测试，但只有一个具有来自多个平台的性能数据的严格积累（www.graph500.org）。计算是在整个静态图上进行的，而在现实世界中，应用程序的更新数据会流到大型持久图中，并且可能会同时进行许多小的目标查询。考虑到使用图形编程范式比传统编程的预期生产率提高，特别是对于并行系统，拥有可用于跨范式比较的通用迷你应用程序变得越来越重要。此外，考虑到图形大小的持续增长，了解底层图形引擎如何在目标图形的大小和类型以及它们所能支持的并行性和并发性的数量和混合方面进行扩展是很重要的。本项目解决了这一需求。在与商业和政府研究实验室的合作中，主要目标是定义一组反映复杂的现实世界应用程序的迷你应用程序，这些应用程序比今天的简单基准更复杂，将这些迷你应用程序转换为现有的主要图形包，然后在广泛的并行系统上运行它们。更广泛的影响可能是显著的。识别相关的小应用程序以及它们在不同系统中的表现将有助于我们了解如何以更具可扩展性的方式编写更完整的图形应用程序，以及哪些编程系统和平台最适合哪些方面。我们也希望不是所有的小应用都能在所有当前的范例中表达，这为那些范例的开发者提供了关于可表达性问题的见解。考虑到这种图形包的相对婴儿期，这种洞察力现在可以从根本上提高它们在未来实际应用中的适用性。

The last decade has seen the growth of extremely large, unstructured, and dynamic data sets, loosely termed Big Data. However, there is a growing desire to extract not just specific properties of collections of such facts, but also relationships between the underlying entities in that data. Examples come from a broad swatch of modern life: bioinformatics, financial, recommendation systems, cyber and national security, and social networks. Graphs have emerged as a valuable and productive paradigm for expressing such problems, where a graph is a collection of a set of objects (vertices) where some pairs of objects are connected by links (edges) that represent some relation between the two. In the last decade there has been an explosion in support for graphs, with widely differing execution models and targeted applicability. Although numerous graph benchmarks have been proposed, only one has had a rigorous accumulation of performance data from multiple platforms (www.graph500.org). Computation is over a whole static graph, whereas the real world sees applications where update data is streaming into large persistent graphs, and very many small targeted queries may be in progress at once.Given the expected productivity increase of using a graph programming paradigm over conventional programming, especially for parallel systems, it is of growing importance to have common mini-apps that can be used for cross-paradigm comparisons. Also, given the continued increase in graph sizes, it is important to understand how the underlying graph engines scale both in the size and type of the target graphs and in the amount and mix of parallelism and concurrency they can support.This project addresses this need. In collaboration with commercial and government research labs, the primary objective is on defining a set of mini-apps that reflect complex real-world applications more sophisticated than today's simple benchmarks, converting these mini-apps to the existing major graph packages, and then running them on a wide range of parallel systems. The wider impact can be significant. Identification of relevant mini-apps and how they perform across different systems will provide insight into both how to write more complete graph applications in more scalable ways, and which aspects of which programming systems and platforms are best suited. It is also expected that not all mini-apps will be expressible in all the current paradigms, providing insight to the developers of those paradigms on expressibility issues. Given the relative infancy of such graph packages such insight now can radically improve their applicability to real applications in the future.