Collaborative Research: CC-NIE Integration: A Data Movement Solution for Next-Generation Campus Cyberinfrastructure

研究进展通常严重依赖于长距离移动数据的能力：例如，从远程超级计算机，望远镜或基因组测序中心到研究人员的实验室或大学数据中心。然而，尽管广域网和园区网络有了重大改进，但由于防火墙和工具不足等障碍，数据传输仍然非常困难和缓慢。为了克服这个问题，该项目开发了一种易于使用的高性能数据传输和同步解决方案，可以在研究人员需要数据的任何地方运行：台式机和研究实验室、园区服务器和HPC设施、超级计算中心、科学仪器以及公共和私有云提供商。这种新的数据移动解决方案建立在广泛使用的Globus实现的GridFTP协议和较新的Globus Transfer托管的数据移动服务，以及印第安纳州大学实现的可扩展会话协议（XSP）的基础上，该协议通过OpenFlow等协议提供端到端电路的协商。它包含了许多新的功能，包括端到端电路的协商;网络地址转换（NAT）设备或防火墙后面的节点对之间的通信;新的数据移动功能，如有效的传输验证;通过Globus Transfer和GridFTP以及XSP开源的组合，将最终的软件解决方案交付给NSF社区软件分发。宇宙学、生物学、物理学、网络基础设施和其他领域的合作伙伴科学和工程项目正在协助评估。

Research progress often depends critically on the ability to move data over long distances: for example, from a remote supercomputer, telescope, or genome sequencing center to a researcher's laboratory or to a university data center. Yet despite major improvements in wide area and campus networks, data movement remains often frustratingly difficult and slow, due to obstacles such as firewalls and inadequate tools.To overcome this problem, this project develops an easy-to-use, high-performance data transfer and synchronization solution that can operate everywhere researchers need their data to be: desktops and research labs, campus servers and HPC facilities, supercomputing centers, scientific instruments, and public and private cloud providers. This new data movement solution builds on the widely used Globus implementation of the GridFTP protocol and the newer Globus Transfer hosted data movement service, plus the Indiana University implementation of the eXtensible Session Protocol (XSP), which provides for negotiation of end-to-end circuits via protocols such as OpenFlow. It incorporates numerous new capabilities, including negotiation of end-to-end circuits; communication between pairs of nodes that are both behind network address translation (NAT) devices or firewalls; new data movement capabilities such as efficient verification of transfers; and HTTP for communication with object stores.Resulting software solutions are delivered to the NSF community via a combination of Globus Transfer and the GridFTP and XSP open source software distributions. Partner science and engineering projects in cosmology, biology, physics, cyberinfrastructure, and other domains are assisting with evaluation.