BIGDATA: F: DKA: DKM: Novel Out-of-core and Parallel Algorithms for Processing Biological Big Data

我们生活在一个在科学和工程的多个领域以低成本生成大量数据的时代。然而，分析工具的进步并没有赶上数据生成的速度。特别是，现有的工具需要太多的时间。一个主要原因是计算机的核心存储器无法保存所有要分析的数据-大多数数据必须存储在二级存储器（SS）中，如固态驱动器和（旋转）磁盘。从SS的数据访问时间是几个数量级以上的核心存储器。通过最小化来自SS的数据访问数量可以获得巨大的加速比。此外，虽然最近在开发多核和GPU算法以解决生物学问题方面进行了大量研究，但对于许多问题，只有顺序的核内算法是已知的。本项目旨在开发用于生物大数据（BBD）分析的新型核外算法。所提出的新的并行算法采用各种架构，包括异构集群的多核和GPU，解决BBD问题。所开发的新颖的可扩展算法可以处理PB级的数据，并超越适用于不同数据集的数据挖掘。这个跨学科的项目提供了一个新的计算套件，用于挖掘大量的生物和其他数据。该项目为研究生和本科生提供教育机会，以获得生物数据分析计算方面的第一手研究经验。

We live in an era when vast amounts of data are being generated at a low cost in several domains of science and engineering. However, advances in analytics tools have not caught up with data generation. In particular, existing tools take too much time. A main reason is that core memories of computers cannot hold all the data to be analyzed -- most of the data have to be stored in secondary storages (SSs) such as solid state drives and (rotating) disks. Data access times from SSs are several orders of magnitude more than from core memories. Tremendous speedups can be obtained by minimizing the number of data accesses from SSs. Also, although there has been much recent research in the development of multicore and GPU algorithms for biological problems, for many of the problems only sequential in-core algorithms are known.This project is to develop novel out-of-core algorithms for biological big data (BBD) analytics. The proposed novel parallel algorithms employ various architectures including heterogeneous clusters of multicores and GPUs, to solve BBD problems. The developed novel scalable algorithms can handle petabytes of data and beyond for data mining applicable over varied datasets. This interdisciplinary project provides a new computation suite for mining voluminous biological and other data. This project provides educational opportunities to graduate and undergraduate students to get first-hand research experience in computational aspects of biological data analysis.