The ATLAS Data Management System Rucio: Supporting LHC Run-2 and beyond
The ATLAS Data Management System Rucio: Supporting LHC Run-2 and beyond
复制标题
ATLAS 数据管理系统 Rucio：支持 LHC Run-2 及更高版本
DOI：
--
复制
发表时间：
2018
期刊：
影响因子：
--
通讯作者：
C. Serfon
中科院分区：
文献类型：
--
作者：
M. Barisits;T. Beermann;V. Garonne;Tomas;Javurek;M. Lassnig;C. Serfon
通过本文的贡献，我们介绍了高能物理实验 ATLAS 的数据管理系统 Rucio 的一些最新进展。 Rucio 已经管理着 300 PB 的官方和用户数据，在整个 LHC Run-2 中看到了逐步改进，目前正在为 HL-LHC 时代的 HEP 计算奠定基础。这一贡献的重点是（a）已经到位的自动化，例如数据重新平衡或用户数据的动态复制，以及它们的支持基础设施，例如实时网络指标或传输时间预测； (b) 采用灵活的方法纳入异构存储系统，包括对象存储，同时使用普遍可用的工具和协议统一潜在的访问路径； (c) 帮助估计传输吞吐量的机器学习方法； (d) 采用 Rucio 进行另外两个实验：AMS 和 Xenon1t。最后，我们提供了运行数据和数据来量化这些改进，并推断未来大型强子对撞机运行所需的变化和发展。
With this contribution we present some recent developments made to Rucio, the data management system of the High-Energy Physics Experiment ATLAS. Already managing 300 Petabytes of both official and user data, Rucio has seen incremental improvements throughout LHC Run-2, and is currently laying the groundwork for HEP computing in the HL-LHC era. The focus of this contribution are (a) the automations that have been put in place such as data rebalancing or dynamic replication of user data, as well as their supporting infrastructures such as real-time networking metrics or transfer time predictions; (b) the flexible approach towards inclusion of heterogeneous storage systems, including object stores, while unifying the potential access paths using generally available tools and protocols; (c) machine learning approaches to help with transfer throughput estimation; and (d) the adoption of Rucio for two other experiments, AMS and Xenon1t. We conclude by presenting operational numbers and figures to quantify these improvements, and extrapolate the necessary changes and developments for future LHC runs.