Collaborative Research: DOT -- Distributed Optical Testbed to Facilitate the Development of Techniques for Efficient Execution of Distributed Applications

作者：Taylor，ValerieChoudhary，Alok; Dinda，Peter A. Mambretti，Joel J.西北大学标题：CISE RR：（协作）DOT--分布式光学测试平台，以促进分布式应用程序的有效执行技术的发展（周日，02-24377）和芝加哥大学（Foster，02-24187），在五个站点获取数据节点和计算节点，有助于构建分布式光学测试床（DOT）。DOT系统是从在单个站点处的大型并行系统上运行的大规模应用到在分布式系统上运行的那些应用的范式转变的产物，其是由高速光网络（例如，Starlight、TeraGrid 40 Gb/s网络、PacificRail 10 Gb/s网络）。这种转变需要一些技术来允许应用程序有效地利用分布式系统。与并行系统相比，这些系统必须利用两个特性：资源（处理器和网络）的异构性和共享资源（特别是广域网）性能的动态变化。该系统由位于六个地理位置不同的站点的Linux群集组成，这些站点通过两个现有的研究DWDM网络（I-WIRE和OMNInet）互连，包括以下站点：阿贡国家实验室（ANL）、伊利诺伊理工学院（IIT）、国家超级计算机应用中心（NCSA）、西北大学芝加哥校区（NU-C）、西北大学埃文斯顿校区（NU-E）、和芝加哥大学（UC）。DOT将促进分布式应用领域的以下研究活动：动态负载平衡（Taylor）性能监测和预测（Dinda，Sun，Taylor）数据管理（Choudhary，Foster）第一项活动开发利用网络性能预测的技术，该技术考虑到分布式系统的处理器和网络的异质性，以动态平衡执行期间的负载。第二个扩展的性能监测，建模和预测技术，一直专注于并行系统和宽带网络的分布式系统与光网络和不同的拓扑结构。最后开发了管理分布式数据的技术，使得实际数据位置是透明的，并且可以有效地访问数据。这些研究活动是由三个应用程序驱动的，这些应用程序已经使用MPI并行化，以便应用程序可以很容易地移植到DOT：ENZO，一个自适应宇宙学应用程序，Cactus，一个用于求解爱因斯坦方程的开放框架，和AudioVoice，一个虚拟化的分布式音频应用程序，具有实时截止日期和不同的计算需求的物理模拟。每个应用程序都提出了挑战，其中包括自适应性，灵活的框架，并与实时截止日期的模拟。

EIA- 0224427Taylor, ValerieChoudhary, Alok; Dinda, Peter A.; Mambretti, Joel J.Northwestern University Title: CISE RR: (Collaborative) DOT--Distributed Optical Testbed to Facilitate the Development of Techniques for Efficient Execution of Distributed Applications This collaborative proposal with Illinois Institute of Technology (Sun, 02-24377) and the University of Chicago (Foster, 02-24187), acquiring data nodes and compute nodes at five sites, contributes to build a Distributed Optical Testbed (DOT). The DOT system, a product of the paradigm shift from large-scale applications running on large parallel systems at single sites to those running on distributed systems, has come about by the availability of high-speed optical networks (E.g., Starlight, TeraGrid 40 Gb/s network, the PacificRail 10 Gb/s network). This shift necessitates techniques that allow applications to efficiently utilize distributed systems. In contrast to parallel systems, these systems must exploit two characteristics:Heterogeneity of resources (processors and networks) and Dynamic changes in performance of shared resources, especially wide area networks. The system, consisting of Linux clusters at six geographically different sites interconnected via two existing research DWDM networks, I-WIRE and OMNInet, involves the following sites: Argonne National Laboratory (ANL), Illinois Institute of Technology (IIT), National Center for Supercomputer Applications (NCSA), Northwester University Chicago Campus (NU-C), Northwestern University Evanston Campus (NU-E), and the University of Chicago (UC). DOT will facilitate the following research activities in the area of distributed applications:Dynamic Load Balancing (Taylor)Performance Monitoring and Prediction (Dinda, Sun, Taylor)Data Management (Choudhary, Foster)The first activity develops techniques utilizing network performance predictions that take into consideration the heterogeneity of the processors and networks of distributed systems to dynamically balance the load during execution. The second extends performance monitoring, modeling and prediction techniques that have been focused on parallel systems and broadband network to distributed systems with optical networks and different topologies. The last develops techniques that manage the distributed data such that the actual data location is transparent and the data is accessed efficiently. These research activities are driven by three applications that have been parallelized using MPI, such that the applications can be easily ported to DOT:ENZO, an adaptive cosmological application,Cactus, an open framework used to solve Einstein's equations, andAudioVoice, a virtualized distributed audio application with physical simulations that have real-time deadlines and varying computational demands.Each application presents challenges, which include adaptivity, flexible framework, and simulations with real-time deadlines.