Compute and Storage Cluster

为了存储和高效地处理来自现代高吞吐量方法的数据，需要存储器和CPU计算集群等。申请的集群将主要存储和处理元基因组数据(给定样本中所有细菌的整个基因组)和单细胞研究数据(单细胞RNA序列和空间单细胞转录组学)。这两种数据类型目前是分子数据环境中最大的体积类型之一。2013年收购的原始计算集群旨在处理从微阵列技术获得的分子数据。这项技术现在已不再用于参赛组，并已完全被测序取代。然而，对数据进行排序需要更多的内存和计算工作量。生物信息学系与其合作伙伴，包括萨尔兰Helmholtz药物研究所(HIPS)和萨尔兰大学医院(UKS)的几个科室，每年产生约2000个元基因组，每个样本的深度为15千兆基。萨尔兰大学的人类遗传学和生物信息学系与他们的合作伙伴一起，每年使用所谓的Drop-Seq方法额外产生和处理大约200万个RNA单细胞图谱。在试点项目中，除RNA-SEQ外，目前还进行ATAC-SEQ，并以亚细胞分辨率收集RNA图谱。一个有50,000个细胞的单细胞实验--在2天内测序--需要大约3 TB的存储空间和20天的纯原始数据分析时间。在数据分析过程中，有时会生成比主数据本身更全面的中间结果。所要求的大规模系统应该能够并行和冗余地存储至少100个实验，将处理时间从20天减少到大约3天。要实现这一点，系统需要至少1,700 TB总存储容量(例如6个16 x 18 GB硬盘)和至少512个计算核心(例如16个32核处理器)，时钟频率在2.5 GHz到3 GHz之间。由于执行的分析通常占用大量内存，因此应该有8 TB的RAM可用。一个决定性因素是避免所谓的交换，即在RAM和硬盘之间频繁复制数据。因此，所有处理器总共至少需要16 TB的缓冲内存和100 TB的快速数据存储(固态硬盘；SSD)。此外，一个由网卡和相应的GB交换机组成的100 GB网络是必不可少的，这样才不会成为复制数据的瓶颈。此外，还需要一个所谓的元数据服务器，它可以将作业和进程以最佳方式分配给各个组件。元数据服务器应配备四个32核处理器和2TB RAM。

In order to store and efficiently process data from modern high-throughput methods, a memory and a CPU compute cluster are required, among other things. The cluster applied for will mainly store and process metagenomic data (the entire genome of all bacteria in a given sample) and single cell research data (single-cell RNA seq and spatial single cell transcriptomics). These two data types are currently among the largest volume types in the molecular data environment. The original compute cluster acquired in 2013 is designed to process molecular data obtained from microarray technology. This technology is now no longer used in the participating groups and has been completely replaced by sequencing. However, sequencing data requires significantly more memory and computational effort. Together with its partners, including the Helmholtz Institute for Pharmaceutical Research Saarland (HIPS) and several departments of Saarland University Hospital (UKS), the bioinformatics department generates about 2,000 metagenomes per year with a depth of 15 gigabases per sample. The human genetics and bioinformatics departments of Saarland University, together with their partners, generate and process an additional approximately 2 million RNA single-cell profiles annually using so-called Drop-seq methods. In pilot projects, ATAC-seq is currently performed in addition to RNA-seq and RNA profiles are collected at subcellular resolution. One single-cell experiment with 50,000 cells - sequenced in 2 days - requires about 3 TB of storage and 20 days of pure primary data analysis time. During the analysis of the data, intermediate results are sometimes generated that are more comprehensive than the primary data itself. The requested large-scale system should be able to store at least 100 experiments in parallel and redundantly, reducing the processing time from 20 days to about 3 days. To achieve this, the system requires at least 1,700 TB of gross storage capacity (for example, 6 x 16 x 18 GB HDDs) and at least 512 computational cores (for example, 16 x 32-core processors) with a clock frequency between 2.5 and 3 GHz. Since the analyses performed are generally memory-intensive, 8 TB of RAM should be available. A decisive factor is to avoid so-called swapping, i.e. the frequent copying of data between the RAM and the hard disk. Therefore, a total of at least 16 TB of buffer memory and 100 TB of fast data storage (solid state disks; SSDs) for all processors together are required. Furthermore, a 100 Gb network - consisting of network cards and a corresponding Gb switch - is essential so that copying the data does not become a bottleneck. In addition, a so-called metadata server is needed, which can optimally distribute jobs and processes to the individual components. The metadata server should be equipped with four 32-core processors and 2 TB RAM.