BIGDATA: Mid-Scale: DA: ESCE: Collaborative Research: Scalable Statistical Computing for Emerging Omics Data Streams

生物信息学数据集既庞大又复杂。编组和管理必要的资源(例如，硬件、计算机和程序员的时间)需要相当高的技能。有效的分析和理解涉及复杂的统计理解。应用程序领域和可用的数据类型快速变化，需要灵活而熟悉的编程环境。合作涉及不同规模和不同专业知识的不同研究小组。该项目开发和传播新的有效办法，以解决目前和正在出现的大量数据统计分析和解释方面的问题。该项目结合了两个非常广泛使用和互补的生物信息学项目--BioConductor和Galaxy的优势。该项目由三个部分组成。第一，提供可伸缩访问，开发适用于可伸缩分析的R编程范例。将开发R/BioConductor软件，通过变换核迭代数据，有效地将大量数据简化为统计描述。BioConductor将部署在可访问的基于云的环境中使用，并将整合到Galaxy部署方案中。第二个组成部分是通过开发用于分析大型生物信息学数据的高性能统计方法，为大型基因组数据提供统计方法。这将最初的技术成果应用于基因组学中统计分析的具体要求。应用领域包括：超大原始数据的质量评估和标准化；下游询问的数据简化和不确定性测量计算；以及新生物学发现的发现、报告和审计。开发需要新的计算方法，避免所有数据在内存中的计算模型(在当前的算法实现中很普遍)，并将单一算法重新表达为可并发执行的独立组件。这强调可扩展和可组合的元素，以产生更丰富的统计基因组学工具包。其目的是利用R？S的优势作为快速开发统计方法的语言，并强调在BioConductor项目中已被证明具有优势的领域。第三部分涉及决策制定。这一相位提供了R/生物导体工作流程到银河的整合。我们将在Galaxy工作流程中部署AIM 2的关键成果。流分析的新实时反馈将引入Galaxy，并由BioConductor利用。该项目包括非常重要的能力建设。BioConductor项目成功地征集、测试和传播了600多个R包，用于统计分析和理解高通量基因组数据。所有包都包含大量文档，包括描述意图、功能和互操作性的小插曲。一揽子计划反映了来自广泛科学界的贡献，并使国家和国际研究生、研究生和商业研究活动在统计、生物信息学和计算领域得以开展。该项目通过解决对大型和复杂生物信息数据的统计分析的内存和性能限制，进一步加强了BioConductor的能力建设影响。Galaxy为数据密集型生物医学研究提供了对计算资源的广泛访问。该项目通过提供对大型生物信息数据的可扩展处理，并使广泛的生物信息社区能够进行探索性分析，加强了银河系统的能力建设影响。BioConductor和Galaxy的结合提供了显著的协同作用，促进了在R开发的统计和生物信息学研究通过Galaxy迅速转化为广泛使用。

Bioinformatic data sets are large and complicated. Marshalling and managing necessary resources (e.g., hardware; computer and programmer time) requires significant skill. Effective analysis and comprehension involves sophisticated statistical understanding. Domains of application and available data types change rapidly, requiring flexible and familiar programming environments. Collaborations involve diverse research groups of heterogeneous size and expertise. This project develops and disseminates new and efficient approaches to solving present and emerging problems in statistical analysis and interpretation of very large data. The project combines the strengths of two very widely used and complementary bioinformatics projects, Bioconductor and Galaxy.The project has three components. The first, providing scalable access, develops R programming paradigms appropriate for scalable analysis. R/Bioconductor software will be developed for efficient reduction of large data to statistical descriptions by iterating data through transformation kernels. Bioconductor will be deployed for use in an accessible cloud-based environment, and will be integrated into the Galaxy deployment scheme. The second component is to provide statistical methods for big genomic data bydeveloping high performance statistical methodologies for analysis of large bioinformatics data. This applies the initial technical achievements to specific requirements of statistical analysis in genomics. Domains of application include: quality assessment and normalization of very large raw data; data reduction and uncertainty measure calculation for downstream interrogation; and discovery, reporting and auditing of novel biological findings. Developments require novel computational approaches that avoid all-data-in-memory computational models (prevalent in current algorithm implementations), and that re-express monolithic algorithms as concurrently executable independent components. This emphasizes extensible and composable elements to yield a richer toolkit for statistical genomics. The aim leverages R?s strength as a language for rapid development of statistical methodologies, and emphasizes areas of proven strength in the Bioconductor project. The third component addresses decision making. This aspect provides integration of R / Bioconductor work flows into Galaxy. We will deploy key results from Aim 2 as Galaxy work flows. New real-time feedback for streaming analytics will be introduced to Galaxy, and leveraged by Bioconductor.The project includes very significant capacity building. The Bioconductor project successfully solicits, tests, and disseminates over 600 R packages for the statistical analysis and comprehension of high-throughput genomic data. All packages include extensive documentation, including vignettes describing intent, function, and interoperability. Packages reflect contributions from a broad scientific community, and enable national and international graduate, post-graduate, and commercial research activities in statistical, bioinformatic, and computational domains. This project furthers the capacity building impact of Bioconductor by addressing memory and performance limitations to statistical analysis of large and complicated bioinformatic data. Galaxy enables broad access to computational resources for data intensive biomedical research. This project enhances the capacity building impacts of Galaxy by providing scalable processing of big bioinformatic data, and enabling exploratory analysis by a broad bioinformatic community. The coupling of Bioconductor and Galaxy provides significant synergy, facilitating rapid translation of statistical and bioinformatic research developed in R to broad use through Galaxy.