DANDI: Distributed Archives for Neurophysiology Data Integration

神经科学数据包含了来自令人难以置信的物种多样性的信息，这些信息是由过量的 设备，并封装了科学思考和决策的结果。这些生成的数据大部分 仍然局限在实验室内，更广泛的科学界无法接触到。这项研究 根据大脑倡议授予的项目正在生成多样化的数据集合，这些数据可以转变为 加快发现步伐。这些数据集很大，大小从GB到PB不等，它们代表 多样的数据类型和分类的元数据。整合而不是进一步孤立这些众多的努力 需要以对神经科学有意义的方式来归档、保存、共享和处理数据 研究人员。任何技术解决方案都应该减少存储和计算的冗余，允许 接近数据的计算，并在适当情况下提供对研究人员或公民的简单但受保护的访问 科学家。考虑到这些计划的规模以及样本大小和数据类型的范围，任何解决方案都应该 还考虑到社区中广泛的个人技术专长，因此允许轻松 参与和摄取档案，同时支持对科学家的教育和培训 使用这些技术。为了解决这些问题，我们提出了​DANDI：分布式档案 神经生理学数据集成。​我们利用我们团队在信息学、标准方面的丰富经验 开发、软件工程、社区建设，并利用强大的开源软件堆栈 创建此存档。该档案将通过使用​无国界神经数据来降低神经科学家的门槛 (NWB；http://nwb.org​​)标准作为一致的数据格式，通过提供与其他 标准，并提供强大的工具和方便的Web界面来与档案进行交互。丹迪 将：1)​提供用于版本化神经生理学数据存储的云平台，用于 协作、归档和保存。2)​提供了简单易用的神经生理学数据提交工具 和档案中的访问；以及3)通过用于数据摄取的标准化应用促进采用NWB， 可视化和处理。​我们将与当地研究人员、更广泛的神经生理学社区合作， 并与联邦和其他资助者一起决定丹迪将存储多长时间和哪些数据。 档案馆还将使用最先进的数据分发技术来增加冗余和故障 容忍性，并允许跨云和本地计算资源进行分布式计算。因此， 努力将显著减少实验室和云之间的障碍，促进协作和数据 交换。总体而言，我们的目标是利用我们的集体专业知识来创建和支持基于NWB的 与当前研究人员工作流程无缝集成并增强的神经生理学档案，降低了 为科学研究和合作设置障碍，并保存信息以供广泛重复使用。

Neuroscientific data contain information from an incredible diversity of species, are generated by a plethora of devices, and encapsulate the results of scientific thinking and decision making. Most of this generated data remains confined within laboratories and is not accessible to the broader scientific community. The research projects awarded under the Brain Initiative are generating a diverse collection of data that can transform and accelerate the pace of discovery. These datasets are large--ranging in size from GBs to PBs-- and represent diverse data types and assorted metadata. To integrate, rather than further isolate, these numerous efforts there is a need to archive, preserve, share, and process data in a way that is meaningful to neuroscience researchers. Any technological solution should reduce redundancy of storage and computation, allow computing near data, and provide easy, but protected when appropriate, access to researchers or citizen scientists. Given the scale of these initiatives and the range of sample sizes and data types, any solution should also consider the broad range of individual technical expertise in the community and therefore allow easy engagement with and ingestion into an archive, while supporting education and training of the scientists in using these technologies. To solve these problems, we propose ​DANDI: Distributed Archives for Neurophysiology Data Integration.​We leverage our team’s extensive experience in informatics, standards development, software engineering, community building, and leverage a robust open-source software stack to create this archive. The archive will lower barriers for neuroscientists by using the ​Neurodata Without Borders (NWB; ​http://nwb.org​) standard as a consistent data format, by providing interoperability with other standards, and by providing robust tools and convenient Web interfaces to interact with the archive. DANDI will: 1) ​provide a cloud platform for versioned neurophysiology data storage for the purposes of collaboration, archiving, and preservation. 2) ​provide easy to use tools for neurophysiology data submission and access in the archive; and 3) facilitate adoption of NWB via standardized applications for data ingestion, visualization and processing. ​We will work with local investigators, the broader neurophysiology community, and with federal and other funders to determine how long and which pieces of data will be stored in DANDI. The archive will also use state of the art data distribution technologies to increase redundancy and fault tolerance, and allow distributed computing across cloud and local computing resources. Consequently the effort will significantly reduce the barrier between laboratories and the cloud, fostering collaboration and data exchange. Overall, we aim to leverage our collective expertise to create and support an NWB-based neurophysiology archive that seamlessly integrates with and enhances current researcher workflows, lowers barriers for scientific inquiry and collaboration, and preserves information for wide reuse.