SDCI Data Improvement: Data Grids for Community Driven Applications

OCI 0721400里根摩尔加州大学圣地亚哥分校SDCI数据改进：社区驱动应用程序的数据网格通过创建共享数据集合，促进了学术机构之间的合作研究。分布式数据管理的事实标准是由圣地亚哥超级计算机中心（SDSC）的数据密集型计算环境小组开发的存储资源代理数据网格（SRB）。SRB广泛用于国家和国际项目，支持用于共享数据的数据网格，用于发布和发现数据的数字图书馆，用于保存数据的永久档案，以及用于联合传感器数据的实时系统。SRB是通用的网络基础设施（软件中间件），用于生产管理跨科学领域（如天文学，生物学，生态学，高能物理学，医学，海洋学，地震学和人文科学）的超过2 PB的数据。SRB通过抽象数据共享、发现和访问所需的主要对象（文件、流、元数据、资源、用户），提供关键的网络基础设施。根据开发SRB的经验和来自强大用户社区的反馈，我们发现还需要抽象数据管理策略。为了自动执行管理策略，我们正在开发下一代分布式数据管理技术，称为集成的面向规则的数据系统（iRODS）。该系统将基于规则的数据管理与服务器端工作流技术相集成，以执行微服务（执行明确定义的任务的软件模块）。将管理策略描述为控制微服务执行的规则的能力，首次使定义特定于用户和社区范围的策略，自动执行管理策略以及验证关于存储库管理的断言成为可能。iRODS背后的概念为开发可定制和可验证的中间件系统提供了一个平台。基于规则的系统结合的思想和技术，从守卫逻辑程序（前向链接和回溯），数据库（触发器和ECA规则），和抽象的工作流系统。过程和声明语义控制模块化软件，不仅定制，而且程序验证。微服务的声明式链接提供的模块化支持远程操作执行的基于规则的验证。管理虚拟化的创新应用支持数据网格、数字图书馆、持久存储库和档案以及教育环境的策略执行自动化，这些环境有望为21世纪世纪的许多科学突破做出贡献。新方法使支持分布式数据管理的网络基础设施的所有组件都能够有计划地发展。这项建议有三个目标：维持对工代机构数据管理软件的支持;开发和支持下一代基于规则的数据网络基础设施，并开发过渡框架，以迁移到新的数据管理基础设施。其目的是开发可演进的解决方案，将国家网络基础设施的全部力量带给科学家和工程师社区。SRB软件支持跨学科数据共享以及特定学科的协作。iRODS软件中数据管理政策的定制和抽象功能将有助于小型和大型社区使用统一的网络基础设施共享数据。

OCI 0721400Reagan MooreUniversity of California at San DiegoSDCI Data Improvement: Data Grids for Community Driven Applications Collaborative research between academic institutions is facilitated through the creation of shared data collections. The de facto standard for distributed data management is the Storage Resource Broker data grid (SRB) developed by the Data Intensive Computing Environment group at the San Diego Supercomputer Center (SDSC). The SRB is used extensively in national and international projects in support of data grids for sharing data, digital libraries for publishing and discovering data, persistent archives for preserving data, and real-time systems for federating sensor data. The SRB is generic cyberinfrastructure (software middleware) that is used in production managing over 2 PBs of data across scientific domains such as astronomy, biology, ecology, high energy physics, medicine, oceanography, seismology, and the humanities. Intellectual merit The SRB provides critical cyberinfrastructure by abstracting the main objects (files, streams, metadata, resources, users) needed for data sharing, discovery and access. Based on the experience in developing the SRB and feedback from a robust user community, we have found that abstraction of data management policies is also needed. To automate execution of management policies, we are developing the next generation of distributed data management technology called the integrated Rule Oriented Data System (iRODS). The system integrates rule-based data management with server-side workflow technology for executing micro-services (software modules that perform well-defined tasks). The ability to characterize management policies as rules controlling the execution of micro-services makes it possible for the first time to define both user-specific and community-wide policies, automate the execution of the management policies, and validate assertions about repository management. The concepts behind iRODS provide a platform for developing customizable and verifiable middleware systems. The rule based system combines ideas and techniques from guarded logic programs (forward-chaining and backtracking), databases (triggers and ECA rules), and abstract workflow systems. Procedural and declarative semantics control modular software for not only customization but also program verification. The modularity provided by declarative chaining of micro-services enables rule-based validation of remote operation execution. The innovative application of management virtualization supports automation of policy execution for data grids, digital libraries, persistent repositories and archives, as well as educational environments that are expected to contribute to many of the scientific breakthroughs of the 21st century. The new approach enables planned evolution of all components of cyberinfrastructure that support distributed data management. This proposal has three aims: to maintain support for the SRB data management software; to develop and support the next generation of rule-based data cyber-infrastructure and to develop a transition framework to migrate to the new data management infrastructure. The aim is to develop evolvable solutions to bring the full power of a national cyberinfrastructure to communities of scientists and engineers. Broader Impact The SRB software enables inter-disciplinary data sharing as well as discipline specific collaborations. The customization and abstraction features of data management policies in the iRODS software will help both small and large communities share data using uniform cyberinfrastucture.