CC*IIE Integration: Collaborative Research: EPSON: Embracing Parallel Networks and Storage for Predictable End-to-End Data Movement

地理上分散的科学界需要越来越复杂的数据传输机制，以应对在不同网络上共享大型数据集的挑战。这些挑战包括采用不同配置和协议的网络优化、高效读写并行存储的I/O机制以及差异很大的数据传输工作负载的不同需求。为了应对这些挑战，EPSON项目正在开发、实施和评估促进端到端并行数据传输的应用程序编程接口和工具。爱普生的研究人员专注于三个领域：(1)支持并行网络数据移动，通过使用专用电路和路径考虑共享网络和基础设施的并行网络特征的多样性，并有效地平衡路径之间的流量，以实现更可预测的性能；(2)开发GridFTP数据存储接口，实现进出并行文件系统的可扩展I/O--这对校园基础设施处理大规模数据集至关重要；以及(3)设计重叠网络传输与存储I/O并结合数据分段启发式的机制，匹配存储和网络能力之间的阻抗，以改进端到端数据传输。该项目涉及与应用程序科学家的密切合作，目标是提供先进的网络工具，以支持芝加哥大学和北伊利诺伊大学校园部署的应用程序的要求。

Geographically distributed scientific communities require increasingly sophisticated data transfer mechanisms that can handle the challenges of sharing large datasets over heterogeneous networks. These challenges include optimization of networks with different configurations and protocols, I/O mechanisms to efficiently read and write to parallel storage, and the varying demands of widely different data transfer workloads.To address these challenges, the EPSON project is developing, implementing, and evaluating application programming interfaces and tools that facilitate end-to-end parallel data transfers. EPSON researchers focus on three areas: (1) enabling parallel network data movement, by taking into account the diversity of parallel network characteristics of both shared networks and infrastructures with dedicated circuits and paths and effectively balancing the flows among paths for more predictable performance; (2) developing a GridFTP data storage interface, enabling scalable I/O to and from parallel filesystems -- critical for campus infrastructures to deal with large-scale datasets; and (3) devising mechanisms that overlap network transfers with storage I/O and incorporate data-staging heuristics, matching the impedance between storage and networking capabilities to improve end-to-end data transfers. The project involves close collaboration with application scientists, with the objective of providing advanced networking tools to support the requirements of the applications deployed at the University of Chicago and Northern Illinois University campuses.