A modular data analysis ecosystem using portable encapsulated projects

使用便携式封装项目的模块化数据分析生态系统

• 批准号: 10224819
• 负责人: Nathan Sheffield
• 金额: $ 39.38万
• 依托单位: UNIVERSITY OF VIRGINIA
• 依托单位国家: 美国
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-08-01 至 2023-07-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/10224819
• 关键词: Adopted; Bioinformatics; Biomedical Research; Complex; Data; Data Analyses; Data Collection; Data Set; Ecosystem; Encapsulated; Environment; Goals; Human; Individual; Inherited; Institution; Knowledge; Link; Manuals; Process; Provider; Pythons; Research; Research Project Grants; Running; Series; Standardization; Structure; Surface; System; Techniques; base; bioinformatics tool; cluster computing; computerized data processing; data management; data sharing; data tools; innovation; insight; interest; large datasets; next generation; novel; novel strategies; portability; tool; tool development

Project summary Overview As the amount of available data increases, it becomes more challenging to process it. Data processing is simple on the surface: it is a mapping from data to analysis. Unfortunately, too often, this requires a unique structure for each combination of dataset and analysis. This makes it difficult to do things like run several different analyses on one dataset, or plug several different datasets to one analysis, because each connection structure must be defined manually. To alleviate this challenge of linking data to tools, this proposal develops the concept of Portable Encapsulated Projects (PEP) and a series of tools that read and process such projects. Essentially, the PEP format aims to standardize the description of data collections, enabling both data providers and data users to communicate through the common interface of a standard format. Practically, this means individuals who describe their projects using this format will immediately inherit both greater portability for analysis as well as greater access to external complementary data. This link operates around a simple, standard, extensible definition of a project. Accompanying this, this proposal develops Python and R packages to provide a modular framework with a low barrier to entry that makes it easy to build robust pipelines and other tools centered around the PEP format. This system presents a new approach to organizing data-intensive biomedical research projects. Significance and innovation This proposal sits at the interface of data management and bioinformatics tool development. While significant effort is already dedicated to each of these individually, there has been less focus at the level of connecting the two. This proposal will build a standardized interface between data and tools in bioinformatics, providing practical advances in formats and tools to facilitate this interaction. This effort approaches computational projects in a novel way, and builds both concepts and tools that can revolutionize bioinformatics research. The goal is not to develop new tools, but to make existing tools more easily applied to existing data. In computational research, a huge amount of effort is spent in data cleanup: preparing data for analysis. By facilitating the connection from data to tools, this will encourage re-analysis of existing data with novel analysis techniques, leading to new discovery. It will also make it easier to analyze new data in tandem with existing data, increasing the value of both. It will contribute to reusability, larger-scale analysis, portable computing environments, and data sharing. There is increasing interest in data sharing and accessibility across scientific domains, and this proposal will facilitate this. Early versions are already adopted for both local compute and cluster computing at four different research institutions, and as the project matures, it will unite various research environments around a common data description. This will make it easier to share data and tools across users, research groups, and institutions. 1

项目概要 概述 随着可用数据量的增加，处理数据变得更具挑战性。数据处理很简单 表面：它是从数据到分析的映射。不幸的是，这常常需要每个组合都有独特的结构 数据集和分析。这使得很难对一个数据集运行多个不同的分析，或者插入多个数据集。 不同的数据集进行一次分析，因为每个连接结构都必须手动定义。 为了缓解将数据链接到工具的这一挑战，该提案提出了可移植封装项目的概念 （PEP）以及一系列读取和处理此类项目的工具。从本质上讲，PEP 格式旨在标准化 数据收集的描述，使数据提供者和数据使用者能够通过通用接口进行通信 的标准格式。实际上，这意味着使用这种格式描述其项目的个人将立即继承 既具有更大的分析便携性，也能更方便地访问外部补充数据。该链接围绕一个 项目的简单、标准、可扩展的定义。 与此同时，该提案开发了 Python 和 R 包，以提供一个低门槛的模块化框架。 可以轻松构建以 PEP 格式为中心的强大管道和其他工具的条目。该系统提出了一个 组织数据密集型生物医学研究项目的新方法。 意义与创新 该提案位于数据管理和生物信息学工具开发的接口处。虽然付出了巨大的努力 已经分别致力于其中的每一个，在连接两者的层面上的关注较少。这个提议 将在生物信息学数据和工具之间建立标准化接口，提供格式和工具方面的实用进步 以促进这种互动。这项工作以一种新颖的方式处理计算项目，并构建了概念和工具 这可以彻底改变生物信息学研究。目标不是开发新工具，而是使现有工具更容易使用 应用于现有数据。 在计算研究中，大量的精力花费在数据清理上：准备用于分析的数据。通过促进 从数据到工具的连接，这将鼓励使用新颖的分析技术重新分析现有数据，从而产生新的 发现。它还将使新数据与现有数据一起分析变得更加容易，从而增加两者的价值。它将 有助于可重用性、大规模分析、便携式计算环境和数据共享。 人们对跨科学领域的数据共享和可访问性越来越感兴趣，该提案将促进这一点。 早期版本已被四个不同的研究机构采用用于本地计算和集群计算，并且 随着该项目的成熟，它将围绕一个共同的数据描述将各种研究环境联合起来。这将使 更容易在用户、研究小组和机构之间共享数据和工具。 1