Collaborative Research: Integrated HPC Systems Usage and Performance of Resources Monitoring and Modeling (SUPReMM)

当今的高性能计算系统是软件、处理器、存储器、网络和存储系统的复杂组合，其特征在于频繁的破坏性技术进步。在这种环境下，系统管理人员、用户和赞助者发现，如果不是不可能的话，也很难知道基础设施是否实现了最佳性能，或者甚至所有子组件是否正常运行。这种系统的用户往往从事极端的科学，系统的不确定性可能会大大延迟甚至混淆科学调查。关键的是，对于基于开源软件系统的系统，其中包括大部分XSEDE资源，使用和管理这些复杂系统所需的数据和信息是不可用的。HPC中心及其用户在某种程度上是盲目的，对系统行为没有清晰的理解。异常行为必须通过不完整和稀疏的数据进行诊断和补救。用户很难评估他们利用现有资源创造科学知识的有效性。NSF缺乏一个全面的知识库来评估其在HPC系统投资的有效性。该奖项将通过创建一套全面的工具来开发所需的知识库来解决这个问题。这将通过以下方式实现：在XSEDE项目的技术审计服务（TAS）下，布法罗大学和德克萨斯大学/德克萨斯州高级计算中心（TACC）目前正在进行的HPC系统监测和报告工作的基础上，并将其与现有监测和分析工具的许多元素结合起来，作为Ranger技术插入工作的一部分。PI将提供了解XSEDE当前运营所需的知识基础，以增强和提高XSEDE所有利益相关者（服务提供商，用户和赞助商）的生产力，并最终提供开源工具，以大大提高HPC系统的运营效率和生产力。

Todays high-performance computing systems are a complex combination of software, processors, memory, networks, and storage systems characterized by frequent disruptive technological advances. In this environment, system managers, users and sponsors find it difficult if not impossible to know if optimal performance of the infrastructure is being realized, or even if all subcomponents are functioning properly. Users of such systems are often engaged in science at the extreme where system uncertainties can significantly delay or even confound the scientific investigations. Critically, for systems based on open source software systems which includes a large fraction of XSEDE resources, the data and information necessary to use and manage these complex systems is not available. HPC centers and their users, are to some extent flying blind, without a clear understanding of system behavior. Anomalous behavior has to be diagnosed and remedied with incomplete and sparse data. It is difficult for users to assess the effectiveness with which they are using the available resources to generate knowledge in their sciences. NSF lacks a comprehensive knowledge base to evaluate the effectiveness of its investments in HPC systems.This award will address this problem through the creation of a comprehensive set of tools for developing the needed knowledge bases. This will be accomplished by building on and combining work on HPC systems monitoring and reporting currently underway at the University at Buffalo under the Technology Audit Service (TAS) of the XSEDE project and University of Texas/ Texas Advance Computing Center (TACC) as part of the Ranger Technology Insertion effort with many elements of existing monitoring and analysis tools. The PIs will provide the knowledge bases required to understand the current operations of XSEDE, to enhance and increase the productivity of all of the stakeholders of XSEDE (service providers, users and sponsors), and ultimately to provide open source tools to greatly increase the operational efficiency and productivity of HPC systems in general.