Tuning big data analysis infrastructure for HIV research

摘要： 艾滋病毒/艾滋病研究领域的大数据分析状况严重匮乏。测序成本的下降刺激了新的软件工具和分析框架的发展。这些努力中的大部分是由真正广泛(且资金充足)的合作项目推动的，例如1000基因组、ENCODE、modENCODE、GTEx、人类微生物组、癌症基因组图谱等。虽然这些项目强化了NGS数据分析和处理的许多方面，以及数据表示的既定标准(例如BAM、VCF、CRAM格式)，但它们面临着一系列与艾滋病毒研究人员所面临的明显不同的挑战，例如，几乎没有突变的长稳定基因组(即人类)与具有许多突变的短可变基因组(即艾滋病毒)。因此，针对艾滋病毒的下一代测序工具和应用程序的开发在很大程度上一直是各个实验室的领域，它们独立地为常见问题设计合理的特别但分类的解决方案，导致领域支离破碎，基本上没有公认的标准，现有解决方案与最终用户的需求之间存在差距。目前为NGS分析编写“全栈”定制内部解决方案的做法是不可扩展和不可维护的，在很大程度上无法利用NGS数据分析其他领域的发展，并阻碍了这一变革性技术在艾滋病毒研究中的采用。该提案的具体目标是解决与艾滋病毒/艾滋病有关的NGS分析的实际问题，方法是将经过验证的和新开发的工具和模块组合成“数据到答案”系列工作流程，并创建一种公开可用和可获得的解决方案，适用于需要对NGS数据进行常规和定制分析的大部分艾滋病毒/艾滋病研究人员。

Abstract: The state of big data analytics in the field of HIV/AIDS research is critically lacking. Decreasing cost of sequencing stimulated the development of novel software tools and analysis frameworks. The bulk of these efforts has been driven by truly expansive (and well-funded) collaborative projects such as the 1000 genomes, ENCODE, modENCODE, GTEx, the Human Microbiome, the Cancer Genome Atlas, and others. While these projects hardened many aspects of NGS data analysis and manipulation, as well as established standards for data representation (e.g. BAM, VCF, CRAM formats) they were facing a set of challenges that is markedly distinct from those faced by HIV researchers, e.g. long stable genomes with few mutations (i.e., human) versus short variable genomes with many mutations (i.e., HIV). Consequently, the development of HIV-specific tools and applications for next generation sequencing (NGS) has largely been the domain of individual labs, independently designing sensible ad hoc, yet disaggregated, solutions to common problems, resulting in a fragmented field largely without accepted standards and gaps between available solutions and the needs of end users. The current practice of writing “full-stack” custom in-house solutions for NGS analyses is not scalable, not maintainable, largely fails to leverage the developments from other domains of NGS data analysis, and hampers the adoption of this transformative technology in HIV research. The specific aims of this proposal address practical aspects of HIV/AIDS-related NGS analysis by assembling proven and newly developed tools and modules into “data to answer” series of workflows, and creating a publicly available and accessible turnkey solution suitable for a large proportion of HIV/AIDS researchers needing to perform routine and bespoke analyses of NGS data..