SHF: Small: Scalable Trace-Based Tools for In-Situ Data Analysis of HPC Applications (ScalaJack)

点击翻译按钮获取中文摘要

Production codes on supercomputers are struggling to remain scalableeach time the processor core count increases by a factor of 10, eventhough they run efficiently at smaller scale.But root cause diagnosis fails at petascale since (1) symptoms ofperformance problems can be subtle, (2) only fewmetrics can be efficiently collected and (3) tools can only feasibly recorda small subset of even these metrics.This work addresses these problems by creating a framework that allowsapplication developers to focus on data analysis that drives customizeddata extraction combined with on-the-fly analysis specifically gearedto their individual problems. This is accomplished by combining traceanalysis and in-situ data analysis techniques at runtime, therebylifting data reduction to a new level where it IS analysis. With thisapproach, modular measurement and analysis components are combined toselectively extract representative data from production codes in aproblem-specific manner, which enables root cause analysis.The work demonstrates the feasibility of customized dataextraction and analysis at scale for root cause analysis on currentand forthcoming multi-petascale supercomputers. It thus contributesto sustain scalable scientific computing into the future up to the largestscales. Results of this work will be contributed as open-source codeto the research community and beyond as done, allowing other groups tonot only build tools on top of our framework but also contribute theirown components.