CC-NIE Integration: High Performance Computing with Data and Networking Acceleration (HPCDNA)

人们越来越预计，未来的网络基础设施将围绕大数据、计算和高性能网络的集成展开。具有数据和网络加速的高性能计算(HPCDNA)项目在校园科学和计算需求的背景下关注这些问题。许多研究人员拥有特定于应用的数据和各种所需的计算环境，包括专门构建的小型实验室计算资源、中型校园高性能计算(HPC)、云计算和全国性的分布式或集中式资源。无法灵活且无缝地将数据获取到最合适的计算资源通常是决定在哪里运行计算以及执行什么计算的限制因素。HPCDNA项目正在开发技术，以极大地提高在这组不同的计算资源中利用公共数据集的能力。HPCDNA架构和技术包括基于部署在中大西洋十字路口区域网络核心的分布式高性能并行文件系统的网络嵌入式存储(NES)系统。该系统通过与校园HPC、校园网和广域网的紧密集成来增强高性能。正在开发一种NES客户端界面，该界面可将存储与动态网络资源调配相协调，并通过工作流流程实现各种高性能访问方法。预计HPCDNA系统将促进研究人员更广泛地使用高性能计算和云，这将允许采用更雄心勃勃的目标，因为他们的计算环境可以随着问题空间的扩大而扩大。

Future cyber-infrastructures are increasingly expected to revolve around the integration of big data, computation, and high performance networking. The High Performance Computing with Data and Networking Acceleration (HPCDNA) project focuses on these issues in the context of campus science and computational requirements. Many researchers have application specific data and a variety of desired compute environments, including purpose built small lab compute resources, medium scale campus High Performance Computing (HPC), cloud computing, and national scale distributed or centralized resources. The inability to flexibly and seamlessly get data to the most appropriate compute resource is often the limiting factor determining where the computation is run and what computation is performed. The HPCDNA project is developing technologies to greatly improve the ability to utilize common data sets across this diverse set of computational resources.The HPCDNA architecture and technologies include a Network Embedded Storage (NES) system based on a distributed high performance parallel file system deployed in the core of the Mid-Atlantic Crossroads regional network. This system is enhanced for high performance via tight integration with campus HPC, campus networks, and wide area networks. An NES client interface is being developed, which coordinates storage with dynamic network provisioning, and enables a variety of high performance access methods by workflow processes. It is expected that the HPCDNA system will facilitate expanded use of HPC and clouds by researchers, which will allow the adoption of more ambitious goals with the knowledge that their computing environments can scale up along with their problem space.