Flexible Data Aggregation for Performance Profiling
Flexible Data Aggregation for Performance Profiling
复制标题
用于性能分析的灵活数据聚合
DOI：
--
复制
发表时间：
2017
期刊：
影响因子：
--
通讯作者：
M. Schulz
中科院分区：
文献类型：
--
作者：
David Böhme;D. Beckingsale;M. Schulz
HPC领域中的几乎所有性能分析工具都执行某种形式的聚合，以计算一系列性能测量的汇总信息，从求和到更复杂的运算(如直方图)。聚合不仅减少了数据量，进而减少了存储空间需求和管理费用，而且对于从记录的测量数据中提取洞察力也至关重要。然而，在当前的工具中，控制聚集的大多数方面，例如要减少的数据维度，都是针对工具开发人员标识的一组特定用例在工具中硬编码的，并且不能由用户扩展或修改。这限制了它们的灵活性，导致用户不得不学习和使用具有不同聚合选项的多个工具来满足其性能分析需求。提出了一种基于灵活键的性能数据聚合的新方法：具有用户定义属性的值数据模型，用户可以用简单的描述语言定义自定义的聚合方案。这不仅为用户提供了部署他们所需的特定数据聚合的控制权，还为使用传统分析工具无法实现的特定于应用程序的数据维度的聚合打开了大门。我们展示了我们的方法如何应用于运行时的性能分析、跨进程数据聚合和交互式数据分析，并通过实际代码驱动的几个案例研究演示了它的功能。
Almost all performance analysis tools in the HPC space perform some form of aggregation to compute summary information of a series of performance measurements, from summations to more complex operations like histograms. Aggregation not only reduces data volumes and consequently storage space requirements and overheads, but is also crucial to extract insights from recorded measurement data. In current tools, however, most aspects that control the aggregation, such as the data dimensions to be reduced, are hard-coded in the tool for a set of particular use cases identified by the tool developer and cannot be extended or modified by the user. This limits their flexibility and often results in users having to learn and use multiple tools with different aggregation options for their performance analysis needs.We present a novel approach for performance data aggregation based on a flexible key:value data model with user-defined attributes, where users can define custom aggregation schemes in a simple description language. This not only gives users the control to deploy the particular data aggregation they need, but also opens the door for aggregations along application-specific data dimensions that cannot be achieved with traditional profiling tools. We show how our approach can be applied for performance profiling at runtime, cross-process data aggregation, and interactive data analysis and demonstrate its functionality with several case studies driven by real world codes.