Process-Oriented Performance Engineering Service Infrastructure for Scientific Software at German HPC Centers

ProPE项目将部署一个原型HPC用户支持基础设施，作为几个2/3级中心的分布式跨站点协作工作，并补充HPC专业知识。在ProPE中，科学软件的代码优化和并行化被视为一个结构化的、定义良好的过程，具有可持续的结果。ProPE的核心组成部分是结构化性能工程(PE)过程的改进、基于过程的实施和传播。该PE过程将代码优化和并行化定义和驱动为面向目标的结构化过程。首先识别应用热点，然后在迭代周期中进行优化/并行化：从分析算法、代码和目标硬件开始，基于性能模式和模型提出了性能限制因素的假设。性能测量验证或指导假设的迭代适应。在验证硬件瓶颈之后，部署适当的代码更改并重新启动PE周期。PE流程的详细程度可以根据潜在问题的复杂性和HPC分析师的经验进行调整。目前，这一过程是由专家在原型层面上应用的。ProPE将对PE过程进行形式化和文档化，并将其应用于各种场景(单核/节点优化、分布式并行、IO密集型问题)。体育过程的不同抽象级别将通过用户支持项目、教学活动和网络文档实施并传播给HPC分析员和应用程序开发人员。将PE流程集成到具有不同HPC支持专业知识的多个中心的现代IT基础设施将是第二个项目重点。PE流程的所有组成部分将在合作地点之间进行协调和标准化。这样，ProPE内部完整的高性能计算专业知识就可以在全国范围内作为连贯的服务提供。正在进行的支持项目可以在参与中心之间轻松转移。为了在系统级别识别低性能应用程序、表征应用程序负载并量化PE活动的好处，ProPE将为HPC群集部署系统监控基础设施。该工具将根据PE流程的要求量身定做，设计用于在2/3级中心轻松部署和使用。相关的ProPE合作伙伴将确保嵌入到德国HPC基础设施中，并在算法选择方面提供基本的PE专业知识，完美地补充ProPE的代码优化和并行化工作。

The ProPE project will deploy a prototype HPC user support infrastructure as a distributed cross-site collaborative effort of several tier-2/3 centers with complementing HPC expertise. Within ProPE code optimizing and parallelization of scientific software is seen as a structured, well-defined process with sustainable outcome.The central component of ProPE is the improvement, process-based implementation, and dissemination of a structured performance engineering (PE) process. This PE process defines and drives code optimization and parallelization as a target-oriented, structured process. Application hot spots are identified first and then optimized/parallelized in an iterative cycle: Starting with an analysis of the algorithm, the code, and the target hardware a hypothesis of the performance-limiting factors is proposed based on performance patterns and models. Performance measurements validate or guide the iterative adaption of the hypothesis. After validation of the hardware bottleneck, appropriate code changes are deployed and the PE cycle restarts. The level of detail of the PE process can be adapted to the complexity of the underlying problem and the experience of the HPC analyst. Currently this process is applied by experts and at the prototype level. ProPE will formalize and document the PE process and apply it to various scenarios (single core/node optimization, distributed parallelization, IO-intensive problems). Different abstraction levels of the PE process will be implemented and disseminated to HPC analysts and application developers via user support projects, teaching activities, and web documentation. The integration of the PE process into modern IT infrastructure across several centers with different HPC support expertise will be the second project focus. All components of the PE process will be coordinated and standardized across the partnering sites. This way the complete HPC expertise within ProPE can be offered as coherent service on a nationwide scale. Ongoing support projects can be transferred easily between participating centers. In order to identify low-performing applications, characterize application loads, and quantify benefits of the PE activities at a system level, ProPE will employ a system monitoring infrastructure for HPC clusters. This tool will be tailored to the requirements of the PE process and designed for easy deployment and usage at tier-2/3 centers.The associated ProPE partners will ensure the embedding into the German HPC infrastructure and provide basic PE expertise in terms of algorithmic choices, perfectly complementing the code optimization and parallelization efforts of ProPE.