Profiling Toolkit for High Performance Computing

高性能计算(HPC)已经成为许多科学学科的标准研究工具。在自然科学和工程科学中，没有至少支持HPC计算的研究正变得越来越少。最重要的是，新的学科正在发现HPC是他们研究的一项资产，例如在生物信息学和社会科学领域。这意味着越来越多的科学家开始使用HPC资源，而不是很好地了解这种系统的工作原理。另一方面，高性能计算资源的复杂性增加，从而增加了这种知识差距。这尤其涉及到高性能计算作业的性能参数和性能工程的重要性。具有初级或中等HPC知识水平的科学家，一旦他们的研究问题可以在可接受的时间框架内在可用的系统上得到解决，他们往往会感到满意，即使这意味着在准确性或解决的问题数量上做出妥协。这些用户大多使用其本地的第3层计算中心，通常缺乏足够的人力资源来单独处理应用程序性能，这一事实加剧了这种情况。我们还需要承认，至少在科学研究的开始阶段，大多数用户并不关心系统资源的最佳利用或应用程序的性能，因为可以理解的是，他们的主要目标是尽快产生科学成果。这还会导致锁定，研究人员将无法将他们的工作转移到Tier-2或Tier-1计算资源，即使这是必需的，只是因为他们无法充分扩展他们的计算。随着在第3层中心部署异类和更复杂的系统，对性能方面的认知需求被视为优化使用第3层和第2层资源上的计算和存储资源的挑战，如电话会议所概述的那样。我们的目标是在所有HPC用户群体中提高对性能参数和问题的认识，并使所有级别的HPC用户能够获得和了解有关其工作负载性能的信息。然后，生成的信息适用于进一步的调查和性能工程措施，从而也降低了由于扩展不足而导致的第2层和第1层资源的障碍。为了实现这些目标，我们建议在现有的概况分析解决方案的基础上，实施一套概况分析工具，它可以自动收集每个工作岗位的绩效指标，并以易于理解的摘要形式提供给研究人员。

High-Performance-Computing (HPC) has become a standard research tool in many scientific disciplines. In the natural and engineering sciences research without at least supporting HPC calculations is becoming increasingly rare. On top of that new disciplines are discovering HPC as an asset to their research, for example in the areas of bioinformatics and social sciences. This means that more and more scientists start using HPC resources, without having a good understanding of the working of such systems. On the other hand, the complexity of HPC resources increases, thereby increasing this knowledge gap. This especially pertains to the performance parameters of HPC jobs and the importance of performance engineering. Scientists with beginning or intermediate HPC knowledge levels are often content once their research problem can be solved on an available system in an acceptable time frame, even if that means compromising on accuracy or the amount of questions addressed. The situation is exacerbated by the fact that these users mostly use their local Tier-3 compute center, which typically lacks sufficient human resources to work with them individually on application performance. We also need to concede that, at least in the beginning of their scientific research, most users do not concern themselves with optimal use of system resources or application performance, as, understandably, their primary objective is to generate scientific output as fast as possible. This also leads to a lock-in where researchers will be unable to transfer their work to Tier-2 or Tier-1 compute resources, even if that would be required, simply because they are not able to scale their calculations sufficiently. With the deployment of heterogeneous, and more complex systems at Tier-3 centers the need of awareness for performance aspects is seen as a challenge for the optimal use of compute and storage resources on Tier-3 and Tier-2 resources, as outlined by the call. We aim to raising awareness for performance parameters and issues across all HPC user communities and to enable HPC users at all levels of experience to obtain and understand information on the perfor-mance of their workloads. The resulting information are then suitable for further investigation and performance engineering measures,thereby also lowering barriers to Tier-2 and Tier-1 resources due to insufficient scaling. In order to achieve these goals we propose to implement a profiling tool set, based on existing profiling solutions, which automatically collects per job performance metrics and presents them to researchers in an understandable summary.