SHF: Small: Understanding and Exploiting Software Defined Networks (SDN) in High Performance Computing (HPC) Environments

点击翻译按钮获取中文摘要

In modern society, many science and technology domains such as physics, biology, and meteorology rely on powerful supercomputers to perform simulations and data analysis to advance knowledge and to serve the society. These supercomputers use specialized hardware and are expensive to build and operate. This project aims at developing novel techniques to build high-performance and low-cost supercomputers by exploiting the capabilities of the Software Defined Networking (SDN) technology that has demonstrated its effectiveness in data centers to reduce cost of setup and management. If successful, this research will change how supercomputers are constructed, and lower the cost. This will make high-performance computing (HPC) accessible to a broad audience and help the advances in many domains. The project will also support education through a curriculum development effort that integrates research and education. SDN manages network traffic and resources in a logically centralized manner, enabling it to seek globally near-optimal resource management solutions for a given optimization objective. This can lead to improved performance over existing interconnection network technologies, including the proprietary ones in high-end HPC systems. Additionally, SDN's widespread commodity use will accelerate technological advances with improved performance and reduced cost in comparison to proprietary technologies. In this project, effective SDN techniques that can lead to higher performance than the current HPC systems are developed; the performance bounds of SDN under different constraints in the HPC environment are established. Additionally, practical techniques that can be deployed with the current technology are developed and evaluated. These techniques spread across different layers of the software stack including network, operating system, and run-time system, and work in concert to exploit the SDN capabilities. Finally, a working proof-of-concept SDN-capable HPC system that incorporates the developed techniques is implemented and deployed. Through analysis, design, development, and experimentation, the strengths and weaknesses of SDN in comparison to the networking technology in HPC systems are clearly identified.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.