Managing ATLAS data on a petabyte-scale with DQ2
Managing ATLAS data on a petabyte-scale with DQ2
复制标题
使用 DQ2 管理 PB 级 ATLAS 数据
DOI：
10.1088/1742-6596/119/6/062017
复制
发表时间：
2008
期刊：
影响因子：
--
通讯作者：
T. Wenaus
中科院分区：
文献类型：
--
作者：
M. Branco;D. Cameron;B. Gaidioz;V. Garonne;B. Koblitz;M. Lassnig;R. Rocha;P. Salgado;T. Wenaus
欧洲核子研究中心大型强子对撞机的ATLAS探测器提出了前所未有的数据处理要求。从2008年开始，ATLAS分布式数据管理系统Don Quijote 2（DQ 2）每年必须管理数十PB的实验数据，这些数据通过LCG、OSG和NDGF计算网格（现在通常称为WLCG）分布在全球。自2005年成立以来，DQ 2一直在为ATLAS合作管理所有实验数据，该合作目前包括来自34个国家的150多所大学和实验室的3000多名科学家。为了满足其提供高度分布式、容错和可扩展架构的主要要求，DQ 2成功地从TB级数据管理升级到PB级数据管理。我们提出了改进和增强的基础上ATLAS数据管理的需求不断增长的DQ 2。我们描述了性能问题、架构更改和实施决策、测试和生产中的当前部署状态以及预期的未来改进。测试结果表明，DQ 2能够处理的数据达到或超过全面数据采集的要求。
The ATLAS detector at CERN's Large Hadron Collider presents data handling requirements on an unprecedented scale. From 2008 on the ATLAS distributed data management system, Don Quijote2 (DQ2), must manage tens of petabytes of experiment data per year, distributed globally via the LCG, OSG and NDGF computing grids, now commonly known as the WLCG. Since its inception in 2005 DQ2 has continuously managed all experiment data for the ATLAS collaboration, which now comprises over 3000 scientists participating from more than 150 universities and laboratories in 34 countries. Fulfilling its primary requirement of providing a highly distributed, fault-tolerant and scalable architecture DQ2 was successfully upgraded from managing data on a terabyte-scale to managing data on a petabyte-scale. We present improvements and enhancements to DQ2 based on the increasing demands for ATLAS data management. We describe performance issues, architectural changes and implementation decisions, the current state of deployment in test and production as well as anticipated future improvements. Test results presented here show that DQ2 is capable of handling data up to and beyond the requirements of full-scale data-taking.