Collaborative Research: Integrated HPC Systems Usage and Performance of Resources Monitoring and Modeling (SUPReMM)

今天的高性能计算系统是软件、处理器、内存、网络和存储系统的复杂组合，其特征是频繁的破坏性技术进步。在这种环境中，系统管理人员、用户和发起人发现很难（如果不是不可能的话）知道基础结构的最佳性能是否正在实现，或者甚至所有子组件是否正常工作。这类系统的用户经常从事极端的科学研究，在这种情况下，系统的不确定性会大大延迟甚至混淆科学研究。关键的是，对于基于开源软件系统的系统（其中包含大量XSEDE资源），无法获得使用和管理这些复杂系统所需的数据和信息。HPC中心及其用户在某种程度上是盲目的，没有对系统行为的清晰理解。异常行为必须用不完整和稀疏的数据进行诊断和补救。使用者很难评估他们利用现有资源在其科学领域产生知识的有效性。NSF缺乏一个全面的知识库来评估其在高性能计算系统上的投资的有效性。该奖项将通过创建一套全面的工具来开发所需的知识库来解决这一问题。这将通过建立和结合目前在布法罗大学XSEDE项目的技术审计服务（TAS）和德克萨斯大学/德克萨斯高级计算中心（TACC）进行的HPC系统监测和报告工作来完成，作为Ranger技术插入工作的一部分，与现有的许多要素监测和分析工具相结合。pi将提供理解XSEDE当前操作所需的知识库，以增强和提高XSEDE所有利益相关者（服务提供商、用户和赞助者）的生产力，并最终提供开源工具，以大大提高一般HPC系统的操作效率和生产力。

Todays high-performance computing systems are a complex combination of software, processors, memory, networks, and storage systems characterized by frequent disruptive technological advances. In this environment, system managers, users and sponsors find it difficult if not impossible to know if optimal performance of the infrastructure is being realized, or even if all subcomponents are functioning properly. Users of such systems are often engaged in science at the extreme where system uncertainties can significantly delay or even confound the scientific investigations. Critically, for systems based on open source software systems which includes a large fraction of XSEDE resources, the data and information necessary to use and manage these complex systems is not available. HPC centers and their users, are to some extent flying blind, without a clear understanding of system behavior. Anomalous behavior has to be diagnosed and remedied with incomplete and sparse data. It is difficult for users to assess the effectiveness with which they are using the available resources to generate knowledge in their sciences. NSF lacks a comprehensive knowledge base to evaluate the effectiveness of its investments in HPC systems.This award will address this problem through the creation of a comprehensive set of tools for developing the needed knowledge bases. This will be accomplished by building on and combining work on HPC systems monitoring and reporting currently underway at the University at Buffalo under the Technology Audit Service (TAS) of the XSEDE project and University of Texas/ Texas Advance Computing Center (TACC) as part of the Ranger Technology Insertion effort with many elements of existing monitoring and analysis tools. The PIs will provide the knowledge bases required to understand the current operations of XSEDE, to enhance and increase the productivity of all of the stakeholders of XSEDE (service providers, users and sponsors), and ultimately to provide open source tools to greatly increase the operational efficiency and productivity of HPC systems in general.