Collaborative Research: 100G Connectivity for Data-Intensive Computing at JHU

合作研究：JHU 数据密集型计算的 100G 连接

• 批准号: 1137045
• 负责人: Alexander Szalay
• 金额: $ 96.13万
• 依托单位: Johns Hopkins University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2012
• 资助国家: 美国
• 起止时间: 2012-01-01 至 2015-12-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1137045&HistoricalAwards=false
• 关键词: Collaborative; Research; 100G; Connectivity; Data

The project establishes 100G connectivity to the Johns Hopkins University in support of data intensive science. Recently, the NSF has funded several regional centers to bring their connection speed to Internet2 and the NLR to 100G. In Oct 2011 DOE and Internet 2 have announced the world's first transcontinental deployment of a 100G network based on coherent technology. Connections are now operational between (among others) New York, Washington DC, Chicago, and Sunnyvale. This comprehensive transformation includes the MidAtlantic Crossroads (MAX) that provides the JHU connectivity to Internet2.Johns Hopkins is also in the process of establishing its own high-speed data overlay research network across the University, connecting 6 locations across the university with multiple 10G connections. These would be aggregated into a single 100G outgoing path to MAX and beyond: to the Teragrid, Internet2, and other national resources supporting computational science.JHU has been awarded an NSF MRI grant to build the 5PB Data-Scope, a novel instrument to observe large data sets. The system will not only have large storage, but it will have extreme IO bandwidth (450GBps) and GPU based processing capability (~200TF), aimed at highly data-intensive analyses simply not possible anywhere else today. Data sets to be analyzed include very large simulations from astrophysics (1PB), ocean circulation models (600TB), computational fluid dynamics (300TB), bioinformatics and neuroscience (2-300TB each). The main difficulty in these challenges is how to bring the data sets to the Data-Scope -- most of these are generated externally, like on the Teragrid, or at the Oak Ridge Jaguar and Kraken systems.These amounts of data are pushing the limits of even a 10G connection. Demonstrating the ability to move PB of data and analyzing them in a timely fashion would encourage others to follow, and would change the way large data problems are approached in all areas of science today. It would also connect the Data-Scope to the whole US community, who can explore how to use such an instrument. It can also serve as a local hub for aggregating data sets for fast uploads into other data intensive national facilities, like the Gordon system in San Diego and the Open Science Data Grid, centered in Chicago. Without the high speed network it will be quite challenging to move petabytes: transferring a 1PB data set at an effective 5Gbps would take 18.5 days. Moving data at 100Gbps would shrink the time to 1 day, a make or break difference for a PB.In order to upgrade to 100G the award activities provide end equipment at MAX, the corresponding optics at JHU, Cisco optics, the addition of a 100G capable card to the Cisco Nexus 7K switch at the JHU side, and a 6x40G-based card for the Nexus 7K to link to the internal JHU research network hubs around the University, where data-intensive resources (clusters, instruments, storage) are located. This strategy would enable the delivery of large amounts of data to the most critical locations at JHU.The proposed high-speed connectivity would enable JHU and its partners to move petabyte-scale data sets, and enable a wide community to tackle cutting edge problems, from the traditional HPC and CFD to people who study the connectivity of the Internet. Students and researchers could do research on topics involving very large datasets, a very current and timely area, and acquire data management and analysis skills on a scale previously unheard of in a university setting.

该项目建立了与约翰霍普金斯大学的100G连接，以支持数据密集型科学。最近，NSF资助了几个区域中心，将它们的连接速度提高到Internet2和NLR的100G。2011年10月，美国能源部和互联网2宣布了世界上第一个基于相干技术的100G跨大陆网络的部署。现在，纽约、华盛顿、芝加哥和森尼维尔（其中包括）之间的连接已经开始运行。这种全面的转变包括中大西洋十字路口（MAX），它提供JHU到Internet2的连接。约翰霍普金斯大学也在建立自己的高速数据覆盖研究网络，通过多个10G连接连接大学的6个地点。这些数据将被聚合成一条100G的输出路径，到达MAX和更远的地方：到达Teragrid、Internet2和其他支持计算科学的国家资源。JHU已获得NSF核磁共振拨款，用于建造5PB数据范围，这是一种观察大型数据集的新型仪器。该系统不仅拥有大容量的存储空间，还将拥有极高的IO带宽（450GBps）和基于GPU的处理能力（~200TF），旨在进行高度数据密集型的分析，这是目前在其他任何地方都无法实现的。要分析的数据集包括来自天体物理学（1PB）、海洋环流模型（600TB）、计算流体动力学（300TB）、生物信息学和神经科学（2-300TB）的非常大的模拟。这些挑战的主要困难是如何将数据集引入data - scope，其中大多数数据集是在外部生成的，比如在Teragrid上，或者在Oak Ridge Jaguar和Kraken系统上。如此庞大的数据量甚至超过了10G连接的极限。展示移动PB数据并及时分析它们的能力将鼓励其他人效仿，并将改变当今所有科学领域处理大数据问题的方式。它还将把数据范围与整个美国社会联系起来，他们可以探索如何使用这种工具。它还可以作为一个本地中心，用于聚合数据集，以便快速上传到其他数据密集型国家设施，如圣地亚哥的戈登系统和以芝加哥为中心的开放科学数据网格。如果没有高速网络，移动pb将是相当具有挑战性的：以5Gbps的有效速度传输1PB数据集将需要18.5天。以100Gbps的速度移动数据将把时间缩短到1天，这对一个PB来说是生死攸关的。为了升级到100G，奖励活动提供了MAX的终端设备，JHU的相应光学器件，思科光学器件，在JHU侧的Cisco Nexus 7K交换机上添加100G功能卡，以及Nexus 7K的6x40g卡，用于连接到大学周围的内部JHU研究网络中心，其中数据密集型资源（集群，仪器，存储）位于。这一战略将使大量数据能够传送到JHU最关键的地点。提议的高速连接将使JHU及其合作伙伴能够移动pb级的数据集，并使广泛的社区能够解决从传统的HPC和CFD到研究互联网连接的人们的尖端问题。学生和研究人员可以对涉及非常大的数据集的主题进行研究，这是一个非常最新和及时的领域，并获得以前在大学环境中闻所未闻的规模的数据管理和分析技能。