A modular data analysis ecosystem using portable encapsulated projects

使用便携式封装项目的模块化数据分析生态系统

• 批准号: 10468680
• 负责人: Nathan Sheffield
• 金额: $ 39.38万
• 依托单位: UNIVERSITY OF VIRGINIA
• 依托单位国家: 美国
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-08-01 至 2023-07-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/10468680
• 关键词: Adopted; Bioinformatics; Biomedical Research; Complex; Data; Data Analyses; Data Collection; Data Set; Ecosystem; Encapsulated; Environment; Goals; Human; Individual; Inherited; Institution; Knowledge; Link; Manuals; Process; Provider; Pythons; Research; Research Project Grants; Running; Series; Standardization; Structure; Surface; System; Techniques; base; bioinformatics tool; cluster computing; computerized data processing; data management; data sharing; data tools; innovation; insight; interest; large datasets; next generation; novel; novel strategies; portability; tool; tool development

Project summary Overview As the amount of available data increases, it becomes more challenging to process it. Data processing is simple on the surface: it is a mapping from data to analysis. Unfortunately, too often, this requires a unique structure for each combination of dataset and analysis. This makes it difficult to do things like run several different analyses on one dataset, or plug several different datasets to one analysis, because each connection structure must be defined manually. To alleviate this challenge of linking data to tools, this proposal develops the concept of Portable Encapsulated Projects (PEP) and a series of tools that read and process such projects. Essentially, the PEP format aims to standardize the description of data collections, enabling both data providers and data users to communicate through the common interface of a standard format. Practically, this means individuals who describe their projects using this format will immediately inherit both greater portability for analysis as well as greater access to external complementary data. This link operates around a simple, standard, extensible definition of a project. Accompanying this, this proposal develops Python and R packages to provide a modular framework with a low barrier to entry that makes it easy to build robust pipelines and other tools centered around the PEP format. This system presents a new approach to organizing data-intensive biomedical research projects. Significance and innovation This proposal sits at the interface of data management and bioinformatics tool development. While significant effort is already dedicated to each of these individually, there has been less focus at the level of connecting the two. This proposal will build a standardized interface between data and tools in bioinformatics, providing practical advances in formats and tools to facilitate this interaction. This effort approaches computational projects in a novel way, and builds both concepts and tools that can revolutionize bioinformatics research. The goal is not to develop new tools, but to make existing tools more easily applied to existing data. In computational research, a huge amount of effort is spent in data cleanup: preparing data for analysis. By facilitating the connection from data to tools, this will encourage re-analysis of existing data with novel analysis techniques, leading to new discovery. It will also make it easier to analyze new data in tandem with existing data, increasing the value of both. It will contribute to reusability, larger-scale analysis, portable computing environments, and data sharing. There is increasing interest in data sharing and accessibility across scientific domains, and this proposal will facilitate this. Early versions are already adopted for both local compute and cluster computing at four different research institutions, and as the project matures, it will unite various research environments around a common data description. This will make it easier to share data and tools across users, research groups, and institutions. 1

项目摘要 概述 随着可用数据的数量增加，对其进行处理变得更加挑战。数据处理很简单 表面：这是从数据到分析的映射。不幸的是，这常常需要每个组合的独特结构 数据集和分析。这很难做一些事情，例如在一个数据集上进行多个不同的分析，或插入几个 一个分析的不同数据集，因为必须手动定义每个连接结构。 为了减轻将数据链接到工具的挑战，该建议开发了便携式封装项目的概念 （PEP）以及一系列阅读和处理此类项目的工具。本质上，PEP格式旨在标准化 数据收集的描述，使数据提供商和数据用户能够通过公共接口进行通信 标准格式。实际上，这意味着使用这种格式描述其项目的个人将立即继承 分析的更大便携性以及更大的访问外部完成数据。此链接围绕一个 项目的简单，标准，可扩展的定义。 随之而来的是，该提案开发了Python和R软件包，以提供一个模块化框架，并具有低障碍 进入围绕PEP格式的稳健管道和其他工具变得容易的条目。该系统提出了 组织数据密集型生物医学研究项目的新方法。 有力和创新 该建议位于数据管理和生物信息学工具开发的界面。虽然显着的努力是 已经专门针对每个单独的，关注两者的关注程度较小。这个建议 将在生物信息学中的数据和工具之间建立标准化的接口，从而提供格式和工具的实际进步 促进这种相互作用。这项工作以新颖的方式处理计算项目，并构建概念和工具 这可以彻底改变生物信息学研究。目标不是开发新工具，而是使现有工具更容易 应用于现有数据。 在计算研究中，在数据清理中花费了大量精力：准备数据进行分析。通过促进 从数据到工具的连接，这将鼓励通过新颖的分析技术重新分析现有数据，从而导致新 发现。这也将使与现有数据同时分析新数据变得更加容易，从而增加两者的价值。会 有助于可重复性，大规模分析，便携式计算环境和数据共享。 对整个科学领域的数据共享和可访问性的兴趣越来越大，该提案将有助于这一点。 在四个不同研究机构的本地计算和集群计算已经采用了早期版本，并且 随着项目的成熟，它将围绕共同数据描述统一各种研究环境。这将成为 更容易在用户，研究小组和机构之间共享数据和工具。 1

项目成果

Identity and compatibility of reference genome resources.

• DOI: 10.1093/nargab/lqab036
• 发表时间: 2021-06
• 期刊: NAR genomics and bioinformatics
• 影响因子: 4.6
• 作者: Stolarczyk M;Xue B;Sheffield NC
• 通讯作者: Sheffield NC

GenomicDistributions: fast analysis of genomic intervals with Bioconductor.

• DOI: 10.1186/s12864-022-08467-y
• 发表时间: 2022-04-12
• 期刊: BMC genomics
• 影响因子: 4.4
• 作者: Kupkova K;Mosquera JV;Smith JP;Stolarczyk M;Danehy TL;Lawson JT;Xue B;Stubbs JT 4th;LeRoy N;Sheffield NC
• 通讯作者: Sheffield NC

Linking big biomedical datasets to modular analysis with Portable Encapsulated Projects.

• DOI: 10.1093/gigascience/giab077
• 发表时间: 2021-12-06
• 期刊: GigaScience
• 影响因子: 9.2
• 作者: Sheffield NC;Stolarczyk M;Reuter VP;Rendeiro AF
• 通讯作者: Rendeiro AF

GEOfetch: a command-line tool for downloading data and standardized metadata from GEO and SRA.

• DOI: 10.1093/bioinformatics/btad069
• 发表时间: 2023-03-01
• 期刊: Bioinformatics (Oxford, England)

Refget: standardized access to reference sequences.

• DOI: 10.1093/bioinformatics/btab524
• 发表时间: 2021-12-22
• 期刊: Bioinformatics (Oxford, England)
• 作者: Yates AD;Adams J;Chaturvedi S;Davies RM;Laird M;Leinonen R;Nag R;Sheffield NC;Hofmann O;Keane TM
• 通讯作者: Keane TM