The Evolution of Global Scale Filesystems for Scientific Software Distribution
The Evolution of Global Scale Filesystems for Scientific Software Distribution
复制标题
科学软件分发全球规模文件系统的演变
DOI：
--
复制
发表时间：
2015
期刊：
影响因子：
--
通讯作者：
D. Thain
中科院分区：
文献类型：
--
作者：
J. Blomer;P. Buncic;René Meusel;G. Ganis;I. Sfiligoi;D. Thain
在全球分布式系统中交付复杂的软件是高吞吐量科学计算的主要挑战。对于许多使用网格、云和分布式集群来满足其计算需求的科学社区来说，这个问题出现在不同的规模上。对于需要处理大量数据的高能物理（HEP）协作来说，这一挑战尤其严峻，这些数据依赖于分布在世界各地的数十万个核心进行数据处理。为了满足HEP社区的需求，进行了几次迭代，以创建一个可扩展的用户级文件系统，每天在全球范围内交付软件。该实现是在2006年设计的，旨在满足在数千台机器上运行的一个实验的需求。从那时起，这个想法演变成了一个新的生产全球规模的文件系统，为全球数十万台机器上的多个科学社区提供服务。
Delivering complex software across a worldwide distributed system is a major challenge in high-throughput scientific computing. The problem arises at different scales for many scientific communities that use grids, clouds, and distributed clusters to satisfy their computing needs. For high-energy physics (HEP) collaborations dealing with large amounts of data that rely on hundreds of thousands of cores spread around the world for data processing, the challenge is particularly acute. To serve the needs of the HEP community, several iterations were made to create a scalable, user-level filesystem that delivers software worldwide on a daily basis. The implementation was designed in 2006 to serve the needs of one experiment running on thousands of machines. Since that time, this idea evolved into a new production global-scale filesystem serving the needs of multiple science communities on hundreds of thousands of machines around the world.