New algorithms and tools for large-scale genomic analyses

项目总结 对大型、复杂数据集的探索和解释对于基因组学中的发现至关重要。然而， 研究人员现在面临着一个根本性的限制；由于现代技术的发展，前所未有的实验成为可能 DNA测序技术，现有的用于比较的“基因组运算”算法和数据格式 和解剖得到的数据集不能跟上数据集大小和不可阻挡的增长的步伐 复杂性。基因组算术(GA)代表了一套强大且广泛使用的技术，使人们能够 探索基因组特征集合之间的关系(例如，基因、序列比对、芯片序列峰或 任何可以用染色体坐标描述的东西)。GA用于广泛的分析 包括：交叉/重叠特征的检测(例如，序列比对和外显子)，描述 数据集之间的要素覆盖，以及要素数据集的合并、减去和互补。镓 功能被所有基因组浏览器和数据可视化工具以及分析软件使用，例如 关贸总协定和SamTools。我们的BEDTOOLS软件已成为基因组学研究的主要内容，并用于 广泛的基因组分析。然而，持续的支持和发展也揭示了关键 现有功能的局限性和妨碍分析灵活性的关键局限性。我们认为 需要在基因组算术算法、数据格式和用户友好软件方面进行创新，以：(1) 使研究人员能够使用简单、灵活的工具进行大规模分析；(2)改进分析工具，以 跟上现代数据集的规模；(3)可视化和量化基因组之间的关系 数据集。 因此，这项提案的总体目标是为基因组学社区提供创新的 新的算法和软件，跟上现代基因组学实验的步伐，促进未来 发现。具体目标是：(1)开发一套完善的基因组算法和 使用BEDTOOLS提供可伸缩分析的编程接口。(2)创造新的算法和基因组 支持大规模数据集比较的区间草图方法。(3)实现规模化 可视化和统计分析基于我们在设计可扩展新数据方面的最新进展 格式。这些创新将产生可扩展的新算法、数据结构和格式 让世界各地数以千计的基因组学研究人员受益。

PROJECT SUMMARY The exploration and interpretation of large, complex datasets is vital to discovery in genomics. However, researchers now confront a fundamental limitation; unprecedented experiments are possible thanks to modern DNA sequencing technologies, yet existing “genome arithmetic” algorithms and data formats for comparing and dissecting the resulting datasets are incapable of keeping pace with inexorable growth in dataset size and complexity. Genome arithmetic (GA) represents a powerful and widely used set of techniques that allow one to explore relationships among sets of genome features (e.g., a gene, sequence alignment, ChIP-seq peak, or anything that can be described with chromosome coordinates). GA is used for a broad spectrum of analyses including: the detection of intersecting/overlapping features (e.g., sequence alignments and exons), describing feature coverage among datasets, and the merging, subtraction, and complementation of feature datasets. GA functionality is used by all genome browsers and data visualization tools, and by analysis software such as GATK and SAMTOOLS. Our BEDTOOLS software has become a staple of genomics research and is used in a broad range of genomic analyses. However, continuous support and development have also revealed key limitations with its current functionality and crucial limitations that hinder analytical flexibility. We argue that innovations in genome arithmetic algorithms, data formats and user-friendly software are needed to: (1) empower researchers to conduct large-scale analyses with simple, flexible tools; (2) improve analysis tools to keep pace with the scale of modern datasets; (3) visualize and quantify relationships among genome datasets. Therefore, the overall objective of this proposal is to provide the genomics community with innovative new algorithms and software that keep pace with modern genomics experiments and facilitate future discoveries. The Specific Aims are to: (1) Develop a refined suite of genome arithmetic algorithms and programming interface for scalable analysis with BEDTOOLS. (2) Create new algorithms and genome interval sketching approaches to enable large-scale dataset comparisons. (3) Enable large-scale visualization and statistical analyses grounded in our recent advances in devising scalable new data formats. These innovations will yield with scalable new algorithms, data structures and formats that will empower thousands of genomics researchers around the world.