CC*DNI DIBBs: Multi-Institutional Open Storage Research InfraStructure (MI-OSiRIS)

每个科学领域都以各种形式生成和利用数据：程序，仪器输出，论文，笔记，应用程序，模拟，视频和音频记录等科学家面临的持续和不断发展的挑战是如何存储，访问，转换，管理和策划有效进行研究所需的各种数据，并透明地与校园或其他机构的其他研究人员共享。MI-OSiRIS项目通过将基于对象的软件定义存储技术与受监控、受管理的网络基础设施相结合，为科学家提供分布式存储系统，使他们能够直接从任何参与机构的资源中访问数据，从而解决了这一挑战。 此外，MI-OSiRIS利用每个机构现有的认证基础设施，允许科学家在所有参与机构中提供对其数据的受控访问。 通过记录和发布设计、代码和操作经验，MI-OSiRIS项目成为支持数据密集型、多机构科学合作的可复制模型。MI-OSiRIS通过在每个参与机构部署对象存储服务器，通过托管高速网络连接它们，根据各科研领域的具体需求进行数据分发。Ceph是一个开源存储平台，支持多种数据访问方法（传统的文件、本机对象和块），并允许在每个研究域的基础上配置访问、复制、分发和完整性。MI-OSiRIS构建在低成本的商用硬件上，可以为每个节点提供每秒千兆字节的I/O带宽。该系统通过战略性地部署已通过网络发现，监控和管理平台（网络管理抽象层）增强的perfSONAR实例来监控和管理其合作机构，科学用户和Ceph存储组件之间的网络路径。 Globus Online服务器提供对MI-OSiRIS外部数据的访问。 此外，MI-OSiRIS还利用Ceph的软件定义存储功能来自动执行某些数据生命周期管理任务。

Every field of science generates and utilizes data in various forms: programs, instrument outputs, papers, notes, applications, simulations, video and audio recordings, etc. The continuing and evolving challenge for scientists is how to store, access, transform, manage and curate the variety of data required to effectively conduct their research, and transparently share it with other researchers across campus or at other institutions. The MI-OSiRIS project is addressing that challenge by combining an object-based software-defined storage technology with a monitored, managed network infrastructure to give scientists a distributed storage system which allows them to directly access their data from resources at any of the participating institutions. Furthermore, MI-OSiRIS utilizes each institution's existing authentication infrastructure to allow scientists to provide controlled access to their data across all participating institutions. By documenting and publishing designs, code, and operational experiences, the MI-OSiRIS project serves as a replicable model for supporting data-intensive, multi-institutional science collaborations.MI-OSiRIS implements a Ceph-based petabyte-scale distributed data system by deploying object storage servers at each participating institution, connecting them via a managed high speed network, and distributing data based on the specific requirements of each science research domain. Ceph, an open source storage platform, supports multiple data access methods (traditional file, native object, and block), and allows configuration of access, replication, distribution, and integrity on a per-research-domain basis. MI-OSiRIS is built on low-cost, commodity hardware and can deliver gigabytes per second of I/O bandwidth per node. The system monitors and manages the network paths between its partner institutions, science users and Ceph storage components by strategically deploying perfSONAR instances which have been augmented with a network discovery, monitoring, and management platform (Network Management Abstraction Layer). Globus Online servers provide access to data from outside MI-OSiRIS. In addition, MI-OSiRIS leverages Ceph's software defined storage aspects to automate some data-lifecycle management tasks.