Evaluation and optimization of NWB neurophysiology software and data in the cloud

点击翻译按钮获取中文摘要

PROJECT SUMMARY - Evaluation and optimization of NWB software and data in the cloud. This proposal is for an administrative supplement for the U24 grant “Advancing standardization of neurophysiology data through dissemination of NWB.” The parent project centers around providing support for the usage of Neurodata Without Borders (NWB), a data standard for neurophysiology data that allows neuroscience researchers to package and publish their data in a form that is readily available and reusable by others. Through the parent project, the team is taking several approaches to engage with the user community and lower the barriers to entry for adopting NWB, including hosted hackathons, one-on-one consultations, and tutorials. The team also ensures the continued quality of the NWB codebase through bug tracking, test coverage, and continued engagement with scientific software developers to assist with the integration of new tools for analysis, visualization, search, and publication of NWB datasets. Through our engagements with the community, we have identified the need to optimize NWB software and data for usage in the cloud as a key obstacle to adoption of NWB that we anticipate will become significant in the coming years. As neurophysiology data volumes continue to grow at a rapid pace, researchers are increasingly seeking to leverage the parallel processing capabilities of cloud infrastructure for converting data to NWB and analysis of NWB data. However, the current NWB conversion tools are not yet equipped for cloud integration and the NWB data layout is not optimized for cloud-based reading and analysis. To address these gaps, we will focus on two key aims. First, we will evaluate and optimize strategies for using cloud resources to enable researchers to perform efficient, cost-effective cloud-based conversion of data to NWB. Specifically, we will package our NeuroConv conversion software into containers that contain all of the necessary elements for NeuroConv to be run on any cloud computing environment, and we will develop tools for integrating existing cloud resources, e.g., for input and output of conversion data from/to cloud storage. Second, we will evaluate and optimize reading of NWB data from cloud storage to enhance cloud-based analysis. Specifically, we will integrate the Kerchunk software package designed to read data efficiently from the cloud with the PyNWB software for reading NWB data and we will evaluate the performance of different data layout strategies and optimize the storage of NWB data to enhance the efficiency for cloud-based access and analysis. Successful completion of the proposed work will create the necessary infrastructure and guidance for neuroscience researchers to take full advantage of cloud computing for conversion of data to NWB and analysis of data in NWB. This will enable researchers to convert and analyze their neurophysiology data faster and with fewer resources, which promises to improve data sharing and expedite scientific discovery.