XPS: FULL: DSD: End-to-end Acceleration of Genomic Workflows on Emerging Heterogeneous Supercomputers

这项研究利用并行性来加速检测遗传变异的普遍生物信息学工作流程。该工作流程确定了个体中存在的遗传变异，给出了DNA测序数据。变异检测工作流程是当前基因组数据分析的一个组成部分，一些研究已经将遗传变异与疾病联系起来。这种工作流程的典型实例目前需要几个小时到几天的时间才能完成最先进的软件，并且当前的算法和软件无法利用甚至适度水平的硬件并行性并从中受益。大多数之前的基因组数据分析管道并行化和性能调整方法都针对孤立的计算、I/O或网络数据传输瓶颈，因此它们可以实现的整体性能改进受到限制。该项目的目标是端到端的加速方法，并使用新兴的异构超级计算机来减少工作流程的完成时间。该项目侧重于整体方法，以加速遗传变异检测工作流程中的多个组件。它探索了多粒度的轻量级数据重组以增强本地性，研究了计算、通信和I/O任务的协调、本地感知负载平衡和协调的资源分区以利用高性能计算平台。拟议研究的一个关键目标是针对当前异构超级计算机的大规模并行性和可扩展性潜力设计特定领域的优化，使开发的技术可以很容易地转移和应用到专门的学术集群和商业计算环境。推广工作的目标是通过招聘工作坊本科生，并吸引他们到跨学科的研究生课程。课程开发活动强调跨学科，层并行。有关详细信息，请访问项目网站http://sites.psu.edu/XPSGenomics

The proposed research harnesses parallelism to accelerate thepervasive bioinformatics workflow of detecting genetic variations.This workflow determines the genetic variants present in anindividual, given DNA sequencing data. The variant detection workflowis an integral part of current genomic data analysis, and severalstudies have linked genetic variants to diseases. Typical instancesof this workflow currently take several hours to multiple days tocomplete with state-of-the-art software, and current algorithms andsoftware are unable to exploit and benefit from even modest levels ofhardware parallelism. Most prior approaches to parallelization andperformance tuning of genomic data analysis pipelines have targetedcomputation, I/O, or network data transfer bottlenecks in isolation,and consequently, are limited in the overall performance improvementthey can achieve. This project targets end-to-end accelerationmethodologies and uses emerging heterogeneous supercomputers toreduce workflow time-to-completion.The project focuses on holistic methodologies to accelerate multiplecomponents within the genetic variant detection workflow. It exploreslightweight data reorganizations at multiple granularities to enhancelocality, investigates compute-, communication-, and I/O taskcotuning, locality-aware load-balancing, and coordinated resourcepartitioning to exploit high-performance computing platforms. A keygoal of the proposed research is to design domain-specificoptimizations targeting the massive parallelism and scalabilitypotential of current heterogeneous supercomputers, so that thedeveloped techniques can be easily transferred and applied to dedicated academic cluster and commercial computational environments.Outreach efforts target undergraduate students through recruitingworkshops and attract them to interdisciplinary graduate programs.Curriculum development activities emphasize cross-layer parallelism.For further information, see project web site at http://sites.psu.edu/XPSGenomics