SDCI HPC Improvement: Cactus Tools for Application Level Performance and Correctness Analysis (ALPACA)

尽管高端计算机的速度和性能在过去十年中有了显著的提高，但对这种并行计算机编程的难易程度并没有提高。开发和调试科学软件所需的时间和精力已经成为许多科学和工程领域的瓶颈。开发高性能软件的困难被认为是当今大规模计算机有效使用中最重大的挑战之一。Cactus框架是在过去几年里开发出来的，用于模拟许多科学领域的物理系统，比如广义相对论中的黑洞和中子星。与其他软件框架一样，应用程序是由单独开发和测试的组件构建的。SDCI高性能计算改进：Cactus应用级性能和正确性分析工具（ALPACA）项目将提供高级工具，允许开发人员和最终用户检查和验证应用程序的正确性，并帮助他们在生产环境中测量和改进其性能。这些工具本身就是组件，内置于应用程序中并与之交互。开发的软件还将有助于使应用程序耐受部分系统故障，这将成为未来的迫切需要。由数十万个节点组成的S架构。与现有的调试器和分析器相比，ALPACA工具将在更高的层次上工作，在应用程序实现的物理方程及其离散的层次上工作，而不是在单个代码行或变量的层次上工作。仅仅保证主内核的正确性和良好的可扩展性是不够的；整个应用程序？哪一个可能包含许多较小的模块？必须执行。我们的综合努力将带来经过良好测试和高效的应用程序，这些应用程序可以在更短的时间内开发并更可靠地执行。通过在生产环境中提供应用程序级别的交互式调试功能，以及在大型HPC系统上允许算法级别的交互式实验，ALPACA工具将显著减少从在单个工作站上使用孤立的应用程序组件到执行大规模HPC计算所需的时间和精力。ALPACA工具将与几个科学团体的科学家密切合作开发，确保其直接有用性和对现实世界问题的适用性。智力优势：ALPACA解决的问题对于所有规模的高性能计算系统的成功至关重要，并且ALPACA工具在许多科学领域的算法开发，性能分析和软件工程中将具有很高的价值。LSU小组在开发应用级HPC工具（包括Cactus框架）、在HPC和分布式HPC环境中开发自适应可扩展性算法以及开发应用程序方面处于领先地位；它在应用高性能计算作为解决爱因斯坦问题的工具方面处于世界领先地位？s方程。ALPACA项目同时解决物理和计算科学方面的问题，并将为科学家提供彻底改进的工具，帮助他们将问题带到机器上。更广泛的影响：ALPACA从根本上涉及对NSF重要的几个应用领域，并将影响许多其他领域。ALPACA通过提供软件基础设施，使开发人员和用户能够创建可扩展的应用程序，并以正确和有效的方式使用它们，有可能对计算科学做出巨大贡献。通过这些工具，我们希望团队能够更多地关注物理和数值，而不是更加复杂的计算环境中的计算细节。与此同时，许多其他社区正在出现，以解决复杂的问题，我们的想法和技术，旨在帮助任何学科的HPC软件开发，将对许多这些项目产生影响。因此，ALPACAtools将自然地扩展到社区中。该提案包括一个培训研讨会和一个博士后和一个研究生的培训。

Although the speed and performance of high end computers have increased dramatically over the last decade, the ease of programming such parallel computers has not progressed. The time and effort required to develop and debug scientific software has become the bottleneck in many areas of science and engineering. The difficulty of developing high-performance software is recognized as one of the most significant challenges today in the effective use of large scale computers.The Cactus framework for science applications has been developed over the last several years,tosimulate physical systems in many fields of science, such as black holes and neutron stars in general relativity. As in other software frameworks, applications are built from separately developed and tested components. The project SDCI HPC Improvement: Cactus Tools for Application Level Performance and Correctness Analysis (ALPACA) will provide high-level tools to allow developers and end-users to examine and validate the correctness of an application, and aid them in measuring and improving its performance in production environments. These tools will be components themselves, built into the application and interacting with it. The developed software will also help render applications tolerant against partial system failures, which is becoming a pressing need with tomorrow?s architectures consisting of hundreds of thousands of nodes. In contrast to existing debuggers and profilers, the ALPACA tools will work at a much higher level, at the level of the physical equations and their discretisations which are implemented by the application, not at the level of individual lines of code or variables. It is not enough for only the main kernels to be correct and show good scalability; the overall application ? which may contain many smaller modules ? must perform. Our integrative effort will lead to well-tested and highly efficient applications which are developed in a shorter time scale and execute more reliably. By providing interactive debugging abilities at the application level in production environments,and by allowing interactive experimentation at the algorithmic level on large HPC systems, the ALPACA tools will significantly reduce the time and effort required to take the steps from using isolated application components on single workstations to performing large-scale HPC calculations.The ALPACA tools will be developed in close conjunction with scientists from several scientific communities, ensuring their direct usefulness and applicability to real-world problems.Intellectual Merit: The issues addressed by ALPACA are critically important for the success ofHPC systems at all scales, and ALPACA tools will be highly valuable for algorithm development, performance analysis, and software engineering in many fields of science. The LSU group has beena leader in developing application level HPC tools, including the Cactus framework, in developing algorithms for adaptive scalability in HPC and distributed HPC environments, and in developing applications themselves; it is a world leader in applying HPC as a tool to solve Einstein?s equations.The ALPACA project simultaneously addresses problems in physics and computational science,and will provide scientists with radically improved tools to help them bringing their problems to the machine.Broader Impacts: ALPACA fundamentally involves several application areas important to NSF, and will impact many others. ALPACA has the potential to make a huge contribution to computational science by providing a software infrastructure that enables developers and users to create scalable applications and to use them in a correct and efficent manner. Through this tools, we expect groups to concentrate more on physics and numerics, and less on computational details in an ever more complex computing environment. At the same time, many other communities are emerging to solve complex problems, and our ideas and techniques, designed to help HPC software development in any disciplines, will have impact across many of these projects. The ALPACAtools will thus naturally spread out into the communities. This proposal includes a training workshop and the training of a postdoc and a graduate student.