Collaborative Research: Compiler-Supported Simulation of Scalable Applications for Wide-Area Distributed Computing Systems

广域分布式计算系统代表了科学和商业计算的未来。这样的系统将使广泛的未来应用成为可能，具有巨大的经济和社会影响，如分布式多媒体服务、基于网络的协作、分布式超级计算和远程沉浸。然而，在实现这一愿景之前，需要在软件系统设计、实现和评估的几乎所有方面进行广泛的研究。离散事件仿真已经成为传统计算系统和应用设计和分析的重要工具。然而，由于必须模拟的软件、硬件和网络子系统的规模，在广域分布式系统上执行应用程序的实际仿真是一项具有挑战性的任务。然而，这些组件的复杂性及其复杂的闭环相互作用需要对组件及其相互作用进行足够详细的建模，以适当地预测它们对整体系统性能的影响。在最近的工作中，这些pi合作获得了一些令人兴奋但初步的结果，表明特定的编译器信息可以极大地提高消息传递程序仿真的效率和可扩展性。例如，在多达128个处理器上执行可扩展的ASCI内核基准测试应用程序SWEEP3D的模拟表明，通过将并行模拟与本建议的主题编译器优化类型一起使用，可以比实时更快地模拟。此外，编译器优化的模拟可以在数千个处理器上评估非常大的数据集：可以在多达10,000个处理器上模拟4000万个问题大小的Sweep3D的性能。通过使用编译器信息，可能有许多其他策略可以显著提高仿真的可扩展性和性能，但迄今为止还没有研究过这些策略。需要一个全面的研究计划来开发它们的潜力并评估它们对模拟现实世界应用的影响。本研究的重点是开发基于编译器的技术，以提高并行离散事件模拟的效率，并使用这些技术来评估广域分布式系统的应用程序和系统软件性能。该提案有三个关键组成部分：1。探索一系列编译器支持的策略，以高效地模拟动态，大规模系统和应用。通过扩展我们现有的编译器和仿真基础设施，实现和评估广域分布式系统的这些策略。这需要解决由这些系统的动态特性和缺乏良好定义的度量标准来衡量此类环境中有效的应用级性能所带来的额外挑战。为了评估这些策略对已经开发的真实世界分布式应用程序的有效性，例如分布式多媒体应用程序和紧密耦合应用程序的分布式版本，如asSweep3D和NAS并行基准。拟议的研究计划建立在pi研究小组之间数年的合作基础上，并汇集了大规模并行程序并行模拟，广域网并行模拟以及并行编译器及其用于支持性能评估的关键优势。这个研究项目还补充了正在进行的广域系统软件工作，旨在开发操作系统服务(例如。（如Globus、Legion和WebOS）和编程环境（如Legion、Globe和gradients）。因此，拟议的研究计划代表了成功的广域分布式系统开发和部署的软件支持的必不可少的第三条腿。

Wide-area distributed computing systems represent the future of scientificand commercial computing. Such systems will enable a wide range of futuristicapplications with the potential for enormous economic and social impactapplications such as distributed multimedia services, Web-based collaboration,distributed supercomputing, and teleimmersion. Before this vision can berealized, however, extensive research will be required in virtually allaspects of software system design, implementation, and evaluation.Discrete-event simulation has been an essential tool for the design andanalysis of traditional computational systems and applications. Realisticsimulation of applications executing on wide-area distributed systems,however, is a challenging task because of the scale of the software, hardwareand network subsystems that must be simulated. Nevertheless, the intricacy ofthese components and their complex, closed-loop interactions require thecomponents and their interactions to be modeled in sufficient detail toappropriately predict their impact on overall system performance.In recent work, the PIs have collaboratively obtained some exciting butpreliminary results showing that specific compiler information can greatlyenhance the efficiency and scalability of simulation of message-passingprograms. For instance, it was shown that simulation of a scalable ASCI kernelbenchmark application called SWEEP3D executing on up to 128 processors couldbe simulated faster than real-time by using parallel simulations togetherwith the type of compiler optimizations that are the subject of this proposal.Also, the compiler-optimized simulation can evaluate very large datasets on thousands of processors: it was possible to simulate the performanceof a 40 million-problem size Sweep3D for up to 10,000 processors. There arepotentially a number of other strategies to dramatically improve simulationscalability and performance by using compiler information, none of which havebeen studied so far. A comprehensive program of research is required todevelop their potential and evaluate their impact on simulation of real worldapplications.The focus of the current proposal is to develop compiler-based techniques forimproving the efficiency of parallel discrete event simulation, and to usethese techniques to evaluate application and system software performance forwide-area distributed systems. There are three key components to thisproposal:1. To explore a range of compiler-supported strategies for highly efficientsimulation of dynamic, large-scale systems and applications.2. To implement and evaluate these strategies for wide-area distributedsystems, by extending our existing compiler and simulation infrastructure.This requires addressing additional challenges raised by the dynamic natureof these systems and the lack of well-defined metrics to measure effectiveapplication level performance in such environments.3. To evaluate the effectiveness of these strategies for real world distributedapplications that have already been developed, such as a distributed multimediaapplication and distributed versions of tightly coupled applications such asSweep3D and the NAS parallel benchmarks.The proposed research program builds on a collaboration of several years betweenthe PIs' research groups, and brings together key strengths in parallelsimulation of large-scale parallel programs, parallel simulation of wide-areanetworks, and in parallelizing compilers and their use for supportingperformance evaluation.This program of research also complements the ongoing software efforts forwide-area systems that are aimed at developing operating system services(e.g., Globus, Legion, and WebOS) and programming environments (e.g., Legion,Globe, and GrADS). As such, the proposed research program represents anessential third leg of software support for the development and deploymentof successful wide-area distributed systems.