CC*DNI DIBBs: Multi-Institutional Open Storage Research InfraStructure (MI-OSiRIS)

每个科学领域都以各种形式生成和使用数据：程序、仪器输出、论文、笔记、应用程序、模拟、视频和音频记录等。科学家面临的持续和不断发展的挑战是如何存储、访问、转换、管理和管理有效开展研究所需的各种数据，并透明地与校园内或其他机构的其他研究人员共享这些数据。MI-Osiris项目正在通过将基于对象的软件定义的存储技术与受监控的托管网络基础设施相结合来解决这一挑战，从而为科学家提供一个分布式存储系统，使他们能够直接从任何参与机构的资源中访问他们的数据。此外，MI-Osiris利用每个机构现有的认证基础设施，允许科学家在所有参与机构中提供对他们的数据的受控访问。通过记录和发布设计、代码和操作经验，MI-Osiris项目成为支持数据密集型多机构科学协作的可复制模型。MI-Osiris通过在每个参与机构部署对象存储服务器，通过管理的高速网络连接它们，并根据每个科学研究领域的特定需求分发数据，实现了一个基于Cave的PB级分布式数据系统。Cave是一个开源存储平台，支持多种数据访问方法(传统文件、原生对象和数据块)，并允许在每个研究域的基础上配置访问、复制、分发和完整性。MI-Osiris构建在低成本的商用硬件上，可以为每个节点提供每秒千兆字节的I/O带宽。该系统通过战略性地部署已用网络发现、监控和管理平台(网络管理抽象层)增强的PerSONAR实例，来监控和管理其合作机构、科学用户和Cave存储组件之间的网络路径。Globus Online服务器提供对MI-Osiris外部数据的访问。此外，MI-Osiris利用Cave的软件定义的存储方面来自动化一些数据生命周期管理任务。

Every field of science generates and utilizes data in various forms: programs, instrument outputs, papers, notes, applications, simulations, video and audio recordings, etc. The continuing and evolving challenge for scientists is how to store, access, transform, manage and curate the variety of data required to effectively conduct their research, and transparently share it with other researchers across campus or at other institutions. The MI-OSiRIS project is addressing that challenge by combining an object-based software-defined storage technology with a monitored, managed network infrastructure to give scientists a distributed storage system which allows them to directly access their data from resources at any of the participating institutions. Furthermore, MI-OSiRIS utilizes each institution's existing authentication infrastructure to allow scientists to provide controlled access to their data across all participating institutions. By documenting and publishing designs, code, and operational experiences, the MI-OSiRIS project serves as a replicable model for supporting data-intensive, multi-institutional science collaborations.MI-OSiRIS implements a Ceph-based petabyte-scale distributed data system by deploying object storage servers at each participating institution, connecting them via a managed high speed network, and distributing data based on the specific requirements of each science research domain. Ceph, an open source storage platform, supports multiple data access methods (traditional file, native object, and block), and allows configuration of access, replication, distribution, and integrity on a per-research-domain basis. MI-OSiRIS is built on low-cost, commodity hardware and can deliver gigabytes per second of I/O bandwidth per node. The system monitors and manages the network paths between its partner institutions, science users and Ceph storage components by strategically deploying perfSONAR instances which have been augmented with a network discovery, monitoring, and management platform (Network Management Abstraction Layer). Globus Online servers provide access to data from outside MI-OSiRIS. In addition, MI-OSiRIS leverages Ceph's software defined storage aspects to automate some data-lifecycle management tasks.