Implementing a genomic data management system using iRODS in the Wellcome Trust Sanger Institute
Implementing a genomic data management system using iRODS in the Wellcome Trust Sanger Institute
复制标题
DOI：
10.1186/1471-2105-12-361
复制
发表时间：
2011-09-09
影响因子：
3
通讯作者：
Coates, Guy
中科院分区：
文献类型：
--
作者：
Chiang, Gen-Tao;Clapham, Peter;Coates, Guy
背景：威康信托桑格研究所(WTSI)正在产生越来越多的DNA测序数据。传统的文件系统很难处理这些不断增长的序列数据。因此，需要实施一个良好的数据管理系统，并将其整合到目前的WTSI基础设施中。这样的系统能够很好地管理测序管道的IT基础设施，并允许生物学家跟踪他们的数据。结果：我们选择了一个数据网格系统iRODS(Rule-Oriented Data Management Systems)作为WTSI的数据管理系统。IRODS提供了一种基于规则的系统管理方法，使数据复制变得更加容易，并提供额外的数据保护。与传统文件系统提供的元数据不同，iRODS的元数据系统是全面的，允许用户定制自己的应用层元数据。然后，WTSI中的用户和IT专家可以查询元数据以查找和跟踪数据。本文的目的是描述我们如何(从系统和用户的角度)设计和使用iRODS作为一个数据管理系统。详细介绍了实施iRODS时面临的问题和找到的解决方案。文中还介绍了一个简单的用例，描述了WTSI内部用户如何使用iRODS。结论：iRODS已经实现，可以作为WTSI测序流水线的生产系统。生物学家和IT专家现在都可以跟踪和管理数据，这是以前无法实现的。这种新颖的方法允许生物学家定义他们自己的元数据，并使用这些元数据查询基因组数据。
Background: Increasingly large amounts of DNA sequencing data are being generated within the Wellcome Trust Sanger Institute (WTSI). The traditional file system struggles to handle these increasing amounts of sequence data. A good data management system therefore needs to be implemented and integrated into the current WTSI infrastructure. Such a system enables good management of the IT infrastructure of the sequencing pipeline and allows biologists to track their data.Results: We have chosen a data grid system, iRODS (Rule-Oriented Data management systems), to act as the data management system for the WTSI. iRODS provides a rule-based system management approach which makes data replication much easier and provides extra data protection. Unlike the metadata provided by traditional file systems, the metadata system of iRODS is comprehensive and allows users to customize their own application level metadata. Users and IT experts in the WTSI can then query the metadata to find and track data. The aim of this paper is to describe how we designed and used (from both system and user viewpoints) iRODS as a data management system. Details are given about the problems faced and the solutions found when iRODS was implemented. A simple use case describing how users within the WTSI use iRODS is also introduced.Conclusions: iRODS has been implemented and works as the production system for the sequencing pipeline of the WTSI. Both biologists and IT experts can now track and manage data, which could not previously be achieved. This novel approach allows biologists to define their own metadata and query the genomic data using those metadata.