Evaluation and optimization of NWB neurophysiology software and data in the cloud

NWB 神经生理学软件和云数据的评估和优化

• 批准号: 10827688
• 负责人: Benjamin K Dichter
• 金额: $ 22.87万
• 依托单位: UNIVERSITY OF CALIF-LAWRENC BERKELEY LAB
• 依托单位国家: 美国
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-09-01 至 2024-08-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/10827688
• 关键词: Address; Administrative Supplement; Adopted; Adoption; Archives; BRAIN initiative; Cloud Computing; Code; Communication; Communities; Computer Systems; Computer software; Consultations; Data; Data Analyses; Data Engineering; Data Set; Development; Education; Electrophysiology (science); Elements; Ensure; Environment; Evaluation; Expedite Dissemination; Funding Opportunities; Goals; Grant; Infrastructure; Laboratories; Libraries; Methods; Neurosciences; Output; Parents; Performance; Process; Publications; Publishing; Pythons; Readiness; Reading; Registries; Reproducibility; Research; Research Personnel; Resources; Running; Source; Standardization; System; Technology; Testing; Visualization; Work; Writing; base; cloud based; cloud storage; computerized data processing; cost; cost effective; data access; data exchange; data format; data management; data sharing; data standards; data tools; design; extracellular; hackathon; improved; innovative technologies; neurophysiology; new technology; parallel processing; parallelization; parent grant; parent project; rapid growth; response; software development; tool; usability; web services

PROJECT SUMMARY - Evaluation and optimization of NWB software and data in the cloud. This proposal is for an administrative supplement for the U24 grant “Advancing standardization of neurophysiology data through dissemination of NWB.” The parent project centers around providing support for the usage of Neurodata Without Borders (NWB), a data standard for neurophysiology data that allows neuroscience researchers to package and publish their data in a form that is readily available and reusable by others. Through the parent project, the team is taking several approaches to engage with the user community and lower the barriers to entry for adopting NWB, including hosted hackathons, one-on-one consultations, and tutorials. The team also ensures the continued quality of the NWB codebase through bug tracking, test coverage, and continued engagement with scientific software developers to assist with the integration of new tools for analysis, visualization, search, and publication of NWB datasets. Through our engagements with the community, we have identified the need to optimize NWB software and data for usage in the cloud as a key obstacle to adoption of NWB that we anticipate will become significant in the coming years. As neurophysiology data volumes continue to grow at a rapid pace, researchers are increasingly seeking to leverage the parallel processing capabilities of cloud infrastructure for converting data to NWB and analysis of NWB data. However, the current NWB conversion tools are not yet equipped for cloud integration and the NWB data layout is not optimized for cloud-based reading and analysis. To address these gaps, we will focus on two key aims. First, we will evaluate and optimize strategies for using cloud resources to enable researchers to perform efficient, cost-effective cloud-based conversion of data to NWB. Specifically, we will package our NeuroConv conversion software into containers that contain all of the necessary elements for NeuroConv to be run on any cloud computing environment, and we will develop tools for integrating existing cloud resources, e.g., for input and output of conversion data from/to cloud storage. Second, we will evaluate and optimize reading of NWB data from cloud storage to enhance cloud-based analysis. Specifically, we will integrate the Kerchunk software package designed to read data efficiently from the cloud with the PyNWB software for reading NWB data and we will evaluate the performance of different data layout strategies and optimize the storage of NWB data to enhance the efficiency for cloud-based access and analysis. Successful completion of the proposed work will create the necessary infrastructure and guidance for neuroscience researchers to take full advantage of cloud computing for conversion of data to NWB and analysis of data in NWB. This will enable researchers to convert and analyze their neurophysiology data faster and with fewer resources, which promises to improve data sharing and expedite scientific discovery.

项目摘要-评估和优化NWB软件和云中的数据。 该提案是U24赠款“推进标准化”的行政补充。 神经生理学数据通过NWB的传播。父项目的中心是为以下方面提供支持： 使用神经数据无国界（NWB），神经生理学数据的数据标准，允许 神经科学研究人员将他们的数据打包并以一种易于获得和可重复使用的形式发布， 他人通过父项目，团队正在采取多种方法与用户社区进行互动 降低采用NWB的准入门槛，包括主办黑客马拉松，一对一咨询， 教程.该团队还通过错误跟踪、测试和分析来确保NWB代码库的持续质量。 覆盖面，并继续与科学软件开发人员合作，以协助整合新的 用于分析、可视化、搜索和发布NWB数据集的工具。 通过与社区的合作，我们确定了优化NWB软件的必要性， 在云中使用的数据是采用NWB的关键障碍，我们预计这将在 未来的岁月。随着神经生理学数据量继续快速增长，研究人员正在 越来越多地寻求利用云基础设施的并行处理能力来转换数据 分析NWB数据。然而，目前的NWB转换工具还没有配备云 NWB数据布局未针对基于云的阅读和分析进行优化。解决这些 我们将重点关注两个关键目标。首先，我们将评估和优化使用云资源的策略， 使研究人员能够执行高效、经济高效的基于云的数据到NWB的转换。我们特别 将我们的NeuroConv转换软件打包到包含所有必要元素的容器中， NeuroConv将在任何云计算环境中运行，我们将开发工具， 云资源，例如，用于从/向云存储输入和输出转换数据。第二，我们将评估 并优化从云存储阅读NWB数据，以增强基于云的分析。具体来说，我们将 将Kerchunk软件包与PyNWB集成，该软件包旨在有效地从云中读取数据 用于阅读NWB数据的软件，我们将评估不同数据布局策略的性能，并 优化NWB数据的存储，以提高基于云的访问和分析的效率。 成功完成拟议的工作将为以下方面建立必要的基础设施和提供指导： 神经科学研究人员充分利用云计算将数据转换为NWB， 分析了NWB中的数据。这将使研究人员能够更快地转换和分析他们的神经生理学数据 而且资源更少，这有望改善数据共享，加快科学发现。