Parrallel I/O Methodologies for I/O-Intensive Grand Challenge Applications

9318145 Messina大挑战应用小组竞赛为支持多学科科学家和工程师团队提供了一种机制，以实现1993财政年度高性能计算和通信（HPCC）计划的目标。理想的提案不仅提供了在以下方面取得重大进展的机会：(1)科学或工程中的基本问题，其解决方案可以通过应用高性能计算技术和资源来推进；(2)促进这些进展的使能技术，而且还提供了科学和计算活动之间的重要互动，通常涉及数学、计算机或计算科学家；这将对正在研究的特定科学或工程问题领域或学科之外的高性能计算活动产生影响。这个多学科的项目将研究和开发在计算科学和工程中有效实现I/O密集型应用的策略。可扩展的并行I/O方法将由一个由计算机科学家和应用科学家组成的团队来追求，他们将共同努力：*抽象和定义I/O模型（模板）*在大规模计算上实现和测试应用级I/O工具。Pablo性能分析环境将为性能检测和分析提供基础。应用程序已经在先进的体系结构系统上完全运行，它们的作者都是这个项目的共同研究者。主要使用的计算机将是加州理工学院的英特尔Touchstone Delta和Paragon系统，每台系统都有超过500个计算节点。五个应用领域将包括：流体动力学、化学、天文学、神经科学和材料处理等离子体的建模。这些应用程序的并行程序涵盖了一系列I/O模式和容量，本项目将开发的技术将与广泛的工程和科学应用相关。此外，通过克服当前的I/O限制，该奖项针对的特定应用将取得重大的新的科学和工程成果。到项目结束时，持续的teraFlops计算机将成为可能。该项目将利用当今先进的计算机设计和实施可扩展I/O的一般方法，立即应用这些方法在几个领域进行前所未有的应用，并利用项目前半部分获得的方法和经验来解决未来持续teraFlops计算机的I/O问题。该项目将由一组研究人员执行，他们在使用并行计算机进行大规模应用、测量和描述此类计算机上应用程序的行为以及创建促进此类系统使用的方法和工具方面具有多年经验。在这一丰富经验的基础上，该项目最初将集中于探索现有应用程序的I/O行为，并设计获得足够和可扩展I/O性能的方法。该合同的五个应用领域涵盖了广泛的I/O需求和行为，包括使用核心外方法、从高速仪器读取数据、建模中的交互式和不规则数据流，以及高速网络上分布式计算资源的数据流水线。该方法的一个关键部分是集成工作，该工作将集中于确保项目各组成部分之间的密切沟通和协同以及集成结果。该项目的主要目标是开发尽可能普遍适用的方法和见解。该奖项由高级项目研究局以及NSF工程和计算机科学项目提供支持。

9318145 Messina The Grand Challenge Application Groups competition provides one mechanism for the support of multidisciplinary teams of scientists and engineers to meet the goals of the High Performance Computing and Communications (HPCC) Initiative in Fiscal Year 1993. The ideal proposal provided not only the opportunity to achieve significant progress on (1) a fundamental problem in science or engineering whose solution could be advanced by applying high performance computing techniques and resources, (2) enabling technologies which facilitate those advances, but also significant interactions between scientific and computational activities, usually involving mathematical, computer or computational scientist, that would have impact in high performance computational activities beyond the specific scientific or engineering problem areas(s) or discipline being studied. This multi-disciplinary project will investigate and develop strategies for efficient implementation of I/O intensive applications in computational science and engineering. Scalable parallel I/O approaches will be pursued by a team of computer scientist and applications scientists who will work together to: * Characterize the I/O behavior of specific application programs running on large massively parallel computers * Abstract and define I/O models (templates) * Implement and test application-level I/O tools on large-scale computations The Pablo performance analysis environment will provide the foundation for the performance instrumentation and analysis. The application programs are already fully operational on advanced architecture systems and their authors are all co-investigators in this project. The principal computers used will be the Intel Touchstone Delta and Paragon systems at Caltech, each with over 500 computational nodes. Five application areas will be included: fluid dynamics, chemistry, astronomy, neuroscience, and modelling of m aterials-processing plasmas. The parallel programs for these applications cover a range of I/O patterns and volume, and the techniques that will be developed in this project will be of relevance to a broad spectrum of engineering and science applications. In addition, by overcoming their current I/O limitations, the specific applications targeted in this award will achieve significant new science and engineering results. By the end of the project, sustained teraFlops computers will become available. The project will devise and implement general methods for scalable I/O using today's advanced computers, immediately apply those methods to carry out unprecedented applications in several fields, and use the methods and experience gained in the first half of the project to tackle the I/O issues in future sustained teraFlops computers. This project will be carried out by a team of researchers who have many years of experience in using parallel computers for large-scale applications, in measuring and characterizing the behavior of applications on such computers, and in creating methods and tools that facilitate the use of such systems. Building on this considerable experience, the project will concentrate initially on exploring the I/O behavior of existing applications and on devising ways to get adequate and scalable I/O performance. The five application areas included in this award cover a wide range of I/O needs and behavior, including the use of out-of-core methods, reading data from high-speed instruments, interactive and irregular data flows in modelling, and pipelining of data from distributed computing resources on high-speed networks. A key part of the approach is an integration effort that will concentrate on ensuring close communication and synergy among the components of the project and on integrating the results. The primary objective of the project is to develop methods and insights that are as generally applicable as possible. This award is being suppo rted by the Advanced Projects Research Agency as well as NSF programs in engineering and computer science.