New algorithms and tools for large-scale genomic analyses

项目摘要 大型复杂数据集的探索和解释对于基因组学的发现至关重要。然而，在这方面， 研究人员现在面临着一个根本性的限制;由于现代技术的发展，前所未有的实验成为可能。 DNA测序技术，但现有的“基因组算术”算法和数据格式的比较 和解剖结果数据集无法跟上数据集大小的不可阻挡的增长， 复杂性基因组算法（GA）代表了一组功能强大且广泛使用的技术， 探索基因组特征组之间的关系（例如，基因、序列比对、ChIP-seq峰，或 任何可以用染色体坐标描述的东西）。遗传算法用于广泛的分析 包括：相交/重叠特征的检测（例如，序列比对和外显子），描述 数据集之间的特征覆盖，以及特征数据集的合并，减法和互补。GA 所有基因组浏览器和数据可视化工具以及分析软件（如 GATK和SAMTOOLS。我们的BEDTOOLS软件已成为基因组学研究的主要工具， 广泛的基因组分析。然而，持续的支持和发展也揭示了关键的 其当前功能的局限性以及阻碍分析灵活性的关键局限性。我们认为 需要在基因组算术算法、数据格式和用户友好型软件方面进行创新，以便：（1） 使研究人员能够使用简单、灵活的工具进行大规模分析;（2）改进分析工具， 与现代数据集的规模保持同步;（3）可视化和量化基因组之间的关系 数据集。 因此，本提案的总体目标是为基因组学界提供创新的 新的算法和软件，跟上现代基因组学实验的步伐，并促进未来 发现。具体目标是：（1）开发一套完善的基因组算术算法， 可编程接口，用于BEDTOOLS的可扩展分析。(2)创建新的算法和基因组 间隔素描方法，使大规模的数据集比较。(3)实现大规模 可视化和统计分析基于我们在设计可扩展的新数据方面的最新进展 格式.这些创新将产生可扩展的新算法，数据结构和格式， 赋予全世界成千上万的基因组学研究人员力量。

PROJECT SUMMARY The exploration and interpretation of large, complex datasets is vital to discovery in genomics. However, researchers now confront a fundamental limitation; unprecedented experiments are possible thanks to modern DNA sequencing technologies, yet existing “genome arithmetic” algorithms and data formats for comparing and dissecting the resulting datasets are incapable of keeping pace with inexorable growth in dataset size and complexity. Genome arithmetic (GA) represents a powerful and widely used set of techniques that allow one to explore relationships among sets of genome features (e.g., a gene, sequence alignment, ChIP-seq peak, or anything that can be described with chromosome coordinates). GA is used for a broad spectrum of analyses including: the detection of intersecting/overlapping features (e.g., sequence alignments and exons), describing feature coverage among datasets, and the merging, subtraction, and complementation of feature datasets. GA functionality is used by all genome browsers and data visualization tools, and by analysis software such as GATK and SAMTOOLS. Our BEDTOOLS software has become a staple of genomics research and is used in a broad range of genomic analyses. However, continuous support and development have also revealed key limitations with its current functionality and crucial limitations that hinder analytical flexibility. We argue that innovations in genome arithmetic algorithms, data formats and user-friendly software are needed to: (1) empower researchers to conduct large-scale analyses with simple, flexible tools; (2) improve analysis tools to keep pace with the scale of modern datasets; (3) visualize and quantify relationships among genome datasets. Therefore, the overall objective of this proposal is to provide the genomics community with innovative new algorithms and software that keep pace with modern genomics experiments and facilitate future discoveries. The Specific Aims are to: (1) Develop a refined suite of genome arithmetic algorithms and programming interface for scalable analysis with BEDTOOLS. (2) Create new algorithms and genome interval sketching approaches to enable large-scale dataset comparisons. (3) Enable large-scale visualization and statistical analyses grounded in our recent advances in devising scalable new data formats. These innovations will yield with scalable new algorithms, data structures and formats that will empower thousands of genomics researchers around the world.