Data management in EGEE
Data management in EGEE
复制标题
EGEE 中的数据管理
DOI：
--
复制
发表时间：
2010
期刊：
影响因子：
--
通讯作者：
P. Tedesco
中科院分区：
文献类型：
--
作者：
´Akos Frohner;J. Baud;R. M. G. Rioja;Gilbert Grosdidier;Rémi Mollon;David E. Smith;P. Tedesco
数据管理是分布式生产计算环境的基石之一，EGEE项目旨在提供电子科学基础设施。我们设计并实现了一系列服务和客户端组件，以满足所有用户群体的不同需求。作为主要用户的LHC实验将使用该基础设施每年在全球范围内生成和分发约15 PB的数据。另一个关键的用户群体，生物医学项目，有严格的安全要求，对数据量的重视程度较低。我们为网格数据管理维护了三个服务组：磁盘池管理器（Disk Pool Manager，简称SSD）存储元素（在全球部署了100多个实例）、LCG文件目录（LFC）和文件传输服务（File Transfer Service，简称FTS），后者维持1.5GB/秒的聚合传输速率。它们由单独的客户端组件和工具补充，这些工具有助于协调具有多个服务的更复杂的用例（GST-client，lcg util，eds-socket）。在本文中，我们将展示这些服务，保持清洁和标准的接口，可以一起工作，以涵盖数据流，以及它们如何可以作为单独的组件，以满足不同的要求。我们还将描述我们考虑进一步改进的方面，包括性能和功能。
Data management is one of the cornerstones in the distributed production computing environment that the EGEE project aims to provide for a e-Science infrastructure. We have designed and implemented a set of services and client components, addressing the diverse requirements of all user communities. LHC experiments as main users will generate and distribute approximately 15 PB of data per year worldwide using this infrastructure. Another key user community, biomedical projects, have strict security requirements with less emphasis on the volume of data. We maintain three service groups for grid data management: The Disk Pool Manager (DPM) Storage Element (with more than 100 instances deployed world-wide), the LCG File Catalogue (LFC) and the File Transfer Service (FTS) which sustains an aggregated transfer rate of 1.5GB/sec. They are complemented by individual client components and also tools which help coordinating more complex uses cases with multiple services (GFAL-client, lcg util, eds-cli). In this paper we show how these services, keeping clean and standard interfaces among each other, can work together to cover the data flow and how they can be used as individual components to cover diverse requirements. We will also describe areas that we consider for further improvements, both for performance and functionality.