BIGDATA: F: DKA: DKM: Novel Out-of-core and Parallel Algorithms for Processing Biological Big Data

我们生活在一个科学和工程领域以低成本产生大量数据的时代。然而，分析工具的进步并没有赶上数据生成的速度。特别是，现有的工具需要花费太多的时间。一个主要原因是计算机的核心存储器不能容纳所有要分析的数据——大多数数据必须存储在二级存储器（ss）中，如固态驱动器和（旋转）磁盘。SSs的数据访问时间要比核心存储器多几个数量级。通过最小化来自ss的数据访问数量，可以获得巨大的速度提升。此外，尽管最近在生物问题的多核和GPU算法的发展方面有很多研究，但对于许多问题来说，只有顺序的核心算法是已知的。该项目旨在为生物大数据（BBD）分析开发新的核心外算法。提出的新型并行算法采用多种架构，包括多核异构集群和gpu，来解决BBD问题。所开发的新型可扩展算法可以处理pb级以上的数据，适用于各种数据集的数据挖掘。这个跨学科的项目为挖掘大量的生物和其他数据提供了一个新的计算套件。该项目为研究生和本科生提供了获得生物数据分析计算方面的第一手研究经验的教育机会。

We live in an era when vast amounts of data are being generated at a low cost in several domains of science and engineering. However, advances in analytics tools have not caught up with data generation. In particular, existing tools take too much time. A main reason is that core memories of computers cannot hold all the data to be analyzed -- most of the data have to be stored in secondary storages (SSs) such as solid state drives and (rotating) disks. Data access times from SSs are several orders of magnitude more than from core memories. Tremendous speedups can be obtained by minimizing the number of data accesses from SSs. Also, although there has been much recent research in the development of multicore and GPU algorithms for biological problems, for many of the problems only sequential in-core algorithms are known.This project is to develop novel out-of-core algorithms for biological big data (BBD) analytics. The proposed novel parallel algorithms employ various architectures including heterogeneous clusters of multicores and GPUs, to solve BBD problems. The developed novel scalable algorithms can handle petabytes of data and beyond for data mining applicable over varied datasets. This interdisciplinary project provides a new computation suite for mining voluminous biological and other data. This project provides educational opportunities to graduate and undergraduate students to get first-hand research experience in computational aspects of biological data analysis.