Collaborative Research: DOT -- Distributed Optical Testbed to Facilitate the Development of Techniques for Efficient Execution of Distributed Applications

EIA- 0224427Taylor, ValerieChoudhary, Alok；丁达，彼得A.；西北大学题目：CISE RR:（协作）DOT——分布式光学试验台促进分布式应用高效执行技术的发展这项与伊利诺伊理工学院（Sun, 02-24377）和芝加哥大学（Foster, 02-24187）的合作提案，在五个站点获取数据节点和计算节点，有助于建立一个分布式光学试验台（DOT）。DOT系统是由运行在单个站点的大型并行系统上的大规模应用程序向运行在分布式系统上的大规模应用程序的范式转变的产物，它是由高速光网络（例如Starlight， TeraGrid 40 Gb/s网络，PacificRail 10 Gb/s网络）的可用性产生的。这种转变需要允许应用程序有效利用分布式系统的技术。与并行系统相比，这些系统必须利用两个特点：资源（处理器和网络）的异构性和共享资源（尤其是广域网）性能的动态变化。该系统由位于六个地理位置不同的站点的Linux集群组成，通过两个现有的DWDM研究网络I-WIRE和OMNInet相互连接，涉及以下站点：阿贡国家实验室（ANL）、伊利诺伊理工学院（IIT）、国家超级计算机应用中心（NCSA）、西北大学芝加哥校区（NU-C）、西北大学埃文斯顿校区（NU-E）和芝加哥大学（UC）。DOT将促进分布式应用领域的以下研究活动：动态负载平衡（Taylor）性能监控和预测（Dinda, Sun, Taylor）数据管理（Choudhary, Foster）第一个活动开发利用网络性能预测的技术，考虑到分布式系统的处理器和网络的异构性，在执行期间动态平衡负载。第二部分将性能监控、建模和预测技术从并行系统和宽带网络扩展到具有光网络和不同拓扑结构的分布式系统。最后开发了管理分布式数据的技术，使实际数据位置是透明的，并且可以有效地访问数据。这些研究活动是由三个使用MPI并行化的应用程序驱动的，这样这些应用程序就可以很容易地移植到DOT:ENZO，一个自适应宇宙学应用程序，Cactus，一个用于解决爱因斯坦方程的开放框架，以及audiovoice，一个具有实时截止日期和不同计算需求的物理模拟的虚拟化分布式音频应用程序。每个应用程序都面临挑战，包括适应性、灵活的框架和具有实时截止日期的模拟。

EIA- 0224427Taylor, ValerieChoudhary, Alok; Dinda, Peter A.; Mambretti, Joel J.Northwestern University Title: CISE RR: (Collaborative) DOT--Distributed Optical Testbed to Facilitate the Development of Techniques for Efficient Execution of Distributed Applications This collaborative proposal with Illinois Institute of Technology (Sun, 02-24377) and the University of Chicago (Foster, 02-24187), acquiring data nodes and compute nodes at five sites, contributes to build a Distributed Optical Testbed (DOT). The DOT system, a product of the paradigm shift from large-scale applications running on large parallel systems at single sites to those running on distributed systems, has come about by the availability of high-speed optical networks (E.g., Starlight, TeraGrid 40 Gb/s network, the PacificRail 10 Gb/s network). This shift necessitates techniques that allow applications to efficiently utilize distributed systems. In contrast to parallel systems, these systems must exploit two characteristics:Heterogeneity of resources (processors and networks) and Dynamic changes in performance of shared resources, especially wide area networks. The system, consisting of Linux clusters at six geographically different sites interconnected via two existing research DWDM networks, I-WIRE and OMNInet, involves the following sites: Argonne National Laboratory (ANL), Illinois Institute of Technology (IIT), National Center for Supercomputer Applications (NCSA), Northwester University Chicago Campus (NU-C), Northwestern University Evanston Campus (NU-E), and the University of Chicago (UC). DOT will facilitate the following research activities in the area of distributed applications:Dynamic Load Balancing (Taylor)Performance Monitoring and Prediction (Dinda, Sun, Taylor)Data Management (Choudhary, Foster)The first activity develops techniques utilizing network performance predictions that take into consideration the heterogeneity of the processors and networks of distributed systems to dynamically balance the load during execution. The second extends performance monitoring, modeling and prediction techniques that have been focused on parallel systems and broadband network to distributed systems with optical networks and different topologies. The last develops techniques that manage the distributed data such that the actual data location is transparent and the data is accessed efficiently. These research activities are driven by three applications that have been parallelized using MPI, such that the applications can be easily ported to DOT:ENZO, an adaptive cosmological application,Cactus, an open framework used to solve Einstein's equations, andAudioVoice, a virtualized distributed audio application with physical simulations that have real-time deadlines and varying computational demands.Each application presents challenges, which include adaptivity, flexible framework, and simulations with real-time deadlines.