Collaborative Research: 100G Connectivity for Data-Intensive Computing at JHU

该项目建立了与约翰霍普金斯大学的100 G连接，以支持数据密集型科学。最近，美国国家科学基金会资助了几个区域中心，使他们的连接速度到Internet 2和NLR到100 G。2011年10月，美国能源部和Internet 2宣布了世界上第一个基于相干技术的100 G网络的跨洲部署。纽约、华盛顿、芝加哥和桑尼维尔之间的连接现在已经开通。这一全面的转型包括MidAtlantic Crossroads（MAX），它为JHU提供了与Internet2.Johns霍普金斯的连接，该公司也正在整个大学建立自己的高速数据覆盖研究网络，通过多个10 G连接连接整个大学的6个地点。这些将被聚合成一个单一的100 G的输出路径到MAX和更远的地方：到Teragrid，Internet 2和其他支持计算科学的国家资源。JHU已被授予NSF MRI资助，以建立5 PB的数据范围，一种新的仪器，以观察大型数据集。该系统不仅具有大容量存储，而且还具有极高的IO带宽（450 GBps）和基于GPU的处理能力（~ 200 TF），旨在实现当今其他任何地方都无法实现的高度数据密集型分析。待分析的数据集包括来自天体物理学（1 PB）、海洋环流模型（600 TB）、计算流体动力学（300 TB）、生物信息学和神经科学（各2- 300 TB）的非常大的模拟。这些挑战的主要困难是如何将数据集带到数据范围-其中大部分是外部生成的，如Teragrid，或橡树岭Jaguar和Kraken系统。这些数据量正在推动即使是10 G连接的极限。 展示移动PB数据并及时分析它们的能力将鼓励其他人效仿，并将改变当今所有科学领域处理大数据问题的方式。它还将把数据范围与整个美国社区联系起来，他们可以探索如何使用这种工具。它还可以作为一个本地中心，用于聚合数据集，以便快速上传到其他数据密集型的国家设施，如圣地亚哥的戈登系统和以芝加哥为中心的开放科学数据网格。如果没有高速网络，移动PB将非常具有挑战性：以有效的5Gbps传输1 PB数据集将需要18.5天。以100 Gbps的速度移动数据将把时间缩短到1天，这对PB来说是一个成败的决定。为了升级到100 G，该奖项的活动在MAX提供终端设备，在JHU提供相应的光学器件，在JHU提供思科光学器件，在JHU侧的Cisco Nexus 7 K交换机上增加一个支持100 G的卡，Nexus 7 K的6x 40 G卡可连接到大学周围的内部JHU研究网络中心，这些中心位于数据密集型资源（集群，仪器，存储）。这一战略将使大量的数据交付到JHU最关键的位置。拟议的高速连接将使JHU及其合作伙伴能够移动PB级的数据集，并使广泛的社区能够解决尖端问题，从传统的HPC和CFD到研究互联网连接的人。学生和研究人员可以对涉及非常大的数据集的主题进行研究，这是一个非常及时的领域，并获得以前在大学环境中闻所未闻的数据管理和分析技能。

The project establishes 100G connectivity to the Johns Hopkins University in support of data intensive science. Recently, the NSF has funded several regional centers to bring their connection speed to Internet2 and the NLR to 100G. In Oct 2011 DOE and Internet 2 have announced the world's first transcontinental deployment of a 100G network based on coherent technology. Connections are now operational between (among others) New York, Washington DC, Chicago, and Sunnyvale. This comprehensive transformation includes the MidAtlantic Crossroads (MAX) that provides the JHU connectivity to Internet2.Johns Hopkins is also in the process of establishing its own high-speed data overlay research network across the University, connecting 6 locations across the university with multiple 10G connections. These would be aggregated into a single 100G outgoing path to MAX and beyond: to the Teragrid, Internet2, and other national resources supporting computational science.JHU has been awarded an NSF MRI grant to build the 5PB Data-Scope, a novel instrument to observe large data sets. The system will not only have large storage, but it will have extreme IO bandwidth (450GBps) and GPU based processing capability (~200TF), aimed at highly data-intensive analyses simply not possible anywhere else today. Data sets to be analyzed include very large simulations from astrophysics (1PB), ocean circulation models (600TB), computational fluid dynamics (300TB), bioinformatics and neuroscience (2-300TB each). The main difficulty in these challenges is how to bring the data sets to the Data-Scope -- most of these are generated externally, like on the Teragrid, or at the Oak Ridge Jaguar and Kraken systems.These amounts of data are pushing the limits of even a 10G connection. Demonstrating the ability to move PB of data and analyzing them in a timely fashion would encourage others to follow, and would change the way large data problems are approached in all areas of science today. It would also connect the Data-Scope to the whole US community, who can explore how to use such an instrument. It can also serve as a local hub for aggregating data sets for fast uploads into other data intensive national facilities, like the Gordon system in San Diego and the Open Science Data Grid, centered in Chicago. Without the high speed network it will be quite challenging to move petabytes: transferring a 1PB data set at an effective 5Gbps would take 18.5 days. Moving data at 100Gbps would shrink the time to 1 day, a make or break difference for a PB.In order to upgrade to 100G the award activities provide end equipment at MAX, the corresponding optics at JHU, Cisco optics, the addition of a 100G capable card to the Cisco Nexus 7K switch at the JHU side, and a 6x40G-based card for the Nexus 7K to link to the internal JHU research network hubs around the University, where data-intensive resources (clusters, instruments, storage) are located. This strategy would enable the delivery of large amounts of data to the most critical locations at JHU.The proposed high-speed connectivity would enable JHU and its partners to move petabyte-scale data sets, and enable a wide community to tackle cutting edge problems, from the traditional HPC and CFD to people who study the connectivity of the Internet. Students and researchers could do research on topics involving very large datasets, a very current and timely area, and acquire data management and analysis skills on a scale previously unheard of in a university setting.