Collaborative Research: Compiler-Supported Simulation of Scalable Applications for Wide-Area Distributed Computing Systems

点击翻译按钮获取中文摘要

Wide-area distributed computing systems represent the future of scientific and commercial computing.Such systems will enable a wide range of futuristic applications with the potential for enormous economicand social impact applications such as distributed multimedia services, Web-based collaboration,distributed supercomputing, and teleimmersion. Before this vision can be realized, however, extensiveresearch will be required in virtually all aspects of software system design, implementation, and evaluation.Discrete-event simulation has been an essential tool for the design and analysis of traditional computationalsystems and applications. Realistic simulation of applications executing on wide-area distributed systems,however, is a challenging task because of the scale of the software, hardware and network subsystems thatmust be simulated. Nevertheless, the intricacy of these components and their complex, closed-loopinteractions require the components and their interactions to be modeled in sufficient detail to appropriatelypredict their impact on overall system performance.In recent work, the PIs have collaboratively obtained some exciting but preliminary results showing thatspecific compiler information can greatly enhance the efficiency and scalability of simulation of message-passing programs. For instance, it was shown that simulation of a scalable ASCI kernel benchmarkapplication called SWEEP3D executing on up to 128 processors could be simulated faster than real-time byusing parallel simulations together with the type of compiler optimizations that are the subject of thisproposal. Also, the compiler-optimized simulation can evaluate very large data sets on thousands ofprocessors: it was possible to simulate the performance of a 40 million-problem size Sweep3D for up to10,000 processors. There are potentially a number of other strategies to dramatically improve simulationscalability and performance by using compiler information, none of which have been studied so far. Acomprehensive program of research is required to develop their potential and evaluate their impact onsimulation of real world applications.The focus of the current proposal is to develop compiler-based techniques for improving the efficiency ofparallel discrete event simulation, and to use these techniques to evaluate application and system softwareperformance for wide-area distributed systems. There are three key components to this proposal:1. To explore a range of compiler-supported strategies for highly efficient simulation of dynamic, large-scale systems and applications.2. To use these strategies in developing a practical performance tool for wide-area distributed systems, byextending our existing compiler and simulation infrastructure. This requires addressing additionalchallenges raised by the dynamic nature of these systems and the lack of well-defined metrics tomeasure effective application level performance in such environments.3. To evaluate the effectiveness of these strategies for real world distributed applications such as SF-Express, a large-scale distributed interactive simulation (DIS) environment, and a distributed video-on-demand server (a distributed multimedia application).The proposed research program builds on a collaboration of several years between the PIs' research groups,and brings together key strengths in parallel simulation of large-scale parallel programs, parallel simulation of wide-area networks, and in parallelizing compilers and their use for supporting performance evaluation.This program of research also complements the ongoing software efforts for wide-area systems that areaimed at developing operating system services (e.g., Globus, Legion, and WebOS) and programmingenvironments (e.g., Legion, Globe, and GrADS). As such, the proposed research program represents anessential third leg of software support for the development and deployment of successful wide-areadistributed systems.