Collaborative Research: DOT -- Distributed Optical Testbed to Facilitate the Development of Techniques for Efficient Execution of Distributed Applications

这项与西北大学(泰勒，02-24427)和芝加哥大学(福斯特，02-24187)的合作计划，获取五个站点的数据节点和计算节点，有助于构建分布式光学测试床(DOT)。DOT系统是从运行在单个站点的大型并行系统上的大规模应用程序到运行在分布式系统上的大规模应用程序的范式转换的产物，它是由高速光网络(例如，Starlight、TeraGrid 40 Gb/S网络、太平洋铁路10 Gb/S网络)的可用性而产生的。这种转变需要能够让应用程序高效地利用分布式系统的技术。与并行系统相比，这些系统必须利用两个特点：资源(处理器和网络)的异构性和共享资源，特别是广域网络的性能的动态变化。该系统由六个不同地理位置的Linux集群组成，通过两个现有的研究DWDM网络I-Wire和OMNInet相互连接，涉及以下站点：阿贡国家实验室(ANL)、伊利诺伊理工学院(IIT)、国家超级计算机应用中心(NCSA)、西北大学芝加哥校区(NU-C)、西北大学埃文斯顿校区(NU-E)和芝加哥大学(UC)。DOT将促进分布式应用领域的下列研究活动：动态负载平衡(Taylor)性能监测和预测(Dinda，Sun，Taylor)数据管理(Choudhary，Foster)第一项活动开发利用网络性能预测的技术，这些预测考虑到分布式系统的处理器和网络的异构性，以在执行期间动态平衡负载。第二种是将并行系统和宽带网络的性能监测、建模和预测技术扩展到具有光网络和不同拓扑结构的分布式系统。后者开发了管理分布式数据的技术，以使实际数据位置是透明的，并有效地访问数据。这些研究活动是由三个使用MPI并行化的应用程序驱动的，因此这些应用程序可以很容易地移植到DOT：Enzo是一个自适应宇宙学应用程序，Cactus是一个用于求解爱因斯坦方程的开放框架，AudioVoice是一个虚拟的分布式音频应用程序，具有实时截止日期和不同的计算需求。每个应用程序都面临着挑战，包括自适应、灵活的框架和具有实时截止日期的模拟。

EIA- 0224377Sun, Xian-HeIllinois Institute of Technology Title: CISE RR: (Collaborative) DOT--Distributed Optical Testbed to Facilitate the Development of Techniques for Efficient Execution of Distributed Applications This collaborative proposal with Northwestern University (Taylor, 02-24427) and the University of Chicago (Foster, 02-24187), acquiring data nodes and compute nodes at five sites, contributes to build a Distributed Optical Testbed (DOT). The DOT system, a product of the paradigm shift from large-scale applications running on large parallel systems at single sites to those running on distributed systems, has come about by the availability of high-speed optical networks (E.g., Starlight, TeraGrid 40 Gb/s network, the PacificRail 10 Gb/s network). This shift necessitates techniques that allow applications to efficiently utilize distributed systems. In contrast to parallel systems, these systems must exploit two characteristics:Heterogeneity of resources (processors and networks) and Dynamic changes in performance of shared resources, especially wide area networks. The system, consisting of Linux clusters at six geographically different sites interconnected via two existing research DWDM networks, I-WIRE and OMNInet, involves the following sites: Argonne National Laboratory (ANL), Illinois Institute of Technology (IIT), National Center for Supercomputer Applications (NCSA), Northwester University Chicago Campus (NU-C), Northwestern University Evanston Campus (NU-E), and the University of Chicago (UC). DOT will facilitate the following research activities in the area of distributed applications:Dynamic Load Balancing (Taylor)Performance Monitoring and Prediction (Dinda, Sun, Taylor)Data Management (Choudhary, Foster)The first activity develops techniques utilizing network performance predictions that take into consideration the heterogeneity of the processors and networks of distributed systems to dynamically balance the load during execution. The second extends performance monitoring, modeling and prediction techniques that have been focused on parallel systems and broadband network to distributed systems with optical networks and different topologies. The last develops techniques that manage the distributed data such that the actual data location is transparent and the data is accessed efficiently. These research activities are driven by three applications that have been parallelized using MPI, such that the applications can be easily ported to DOT:ENZO, an adaptive cosmological application,Cactus, an open framework used to solve Einstein's equations, andAudioVoice, a virtualized distributed audio application with physical simulations that have real-time deadlines and varying computational demands.Each application presents challenges, which include adaptivity, flexible framework, and simulations with real-time deadlines.