Collaborative Research: The CrystalGrid Framework

CrystalGrid框架(CGF)项目将研究X射线结晶学领域整个数据空间中数据的获取、传输和管理，解决管理数据表示、格式、数据容器、管理域和不同仪器和设备中的广泛异构性的方法。直到最近，各个实验室只是强制实施本地格式和程序的同质性，而不是存储依赖于实验室的元数据。然而，随着结晶学家开始在实验室之间交叉以实现他们的研究目标，以及随着输出数据流的数量和大小的增加，留给每个研究的时间越来越少，这种特别系统是有限的。必须明确本地工作流程，必须正式描述程序，并以开放、可共享的方式表示数据的历史和集合。为每个实验创建和管理完整的、可访问的记录是至关重要的，以及整个领域的数据获取和管理的异质性。为满足这一需要，该项目将开发一个网络服务接口以及数据和元数据系统框架，涵盖整个结晶学领域。项目参与者和合作者将利用现有的项目，例如互惠网络和公共仪器中间件体系结构，这些项目解决了问题领域中更狭隘的问题。CGF还将利用具有重叠兴趣领域的合作项目，例如总部位于英国的Comb-e-Chem项目。CGF项目涉及处理结晶学应用空间中的数据、程序和仪器的异构性这一经典问题，以及将该领域的整个数据收集、传输和管理需求集成到一个无缝的首尾系统中的另一个经典问题。挑战是创建一个在多个方面管理异构性的虚拟化系统，并仅使用开放、可扩展和可互操作的标准和方法提供垂直集成。虽然该项目构成了对相关计算机科学问题的研究，但执行研究的计划集中在生产一种产品(CGF)，该产品将立即用于解决X射线结晶学领域新出现的技术问题。在结晶学中，具体目标之一是使人们能够获得否则可能永远看不到的结构结果，因此CGF将帮助增加科学知识体系，并提高联邦对全国范围内大量x射线衍射仪和相关仪器的投资回报。尽管该项目专门针对全球数百个结晶学实验室，但其中创建的软件和方法旨在为任何从单个实验室实践转向共享的全球协作系统的科学重复使用。在高能物理和天文学等科学领域，科学家们长期以来一直共享单一、独特的大型仪器，不得不创建共享的数据管理和仪器元数据。CGF可能在其他科学学科中有用，这些学科仍然使用广泛分布的基于实验室的仪器，现在需要在数据网格中进行链接。

The CrystalGrid Framework (CGF) project will research the acquisition, transport, and curation of data over the entire data space of the field of X-ray crystallography, addressing methods for managing wide heterogeneity in data representations, formats, data containers, administrative domains and diverse instruments and equipment. Until recently, individual labs have simply imposed local homogeneity of format and procedure, and not stored lab-dependent metadata. This ad hoc system is limited, however, as crystallographers begin to cross between labs to accomplish their research objectives, and as increasing numbers and sizes of output data streams leave less time for each investigation. Local workflow must be made explicit, procedures must be formally described, and the history and assemblages of data expressed in an open, shareable way. Creation and management of complete, accessible records for each experiment is critical, as well as heterogeneity in data acquisition and management across the field. To meet that need, this project will develop a framework of web service interfaces and data and metadata systems addressing the whole spectrum of crystallography. Project participants and collaborators will leverage existing projects, such as Reciprocal Net and Common Instrument Middleware Architecture, that address narrower issues in the problem domain. The CGF will also draw on collaborating projects with overlapping areas of interest, such as the UK-based Comb-e-Chem project. The resulting framework will be a useful environment for crystallographic investigations and an extensible platform on which new web-based applications can be built.The CGF project involves the classic problem of dealing with heterogeneity in data, procedures, and instruments in the crystallography application space, and another classic problem in integrating the entire data collection, transport, and curation requirements of the domain into a seamless beginning to end system. The challenge is to create a virtualization system that manages heterogeneity in more than a single aspect and to provide vertical integration using only open, extensible, and interoperable standards and methodologies.While the project constitutes research into pertinent computer science problems, the plan for performing the research is centered on producing a product (the CGF) that will immediately be useful in addressing emerging technical problems in the field of X-ray crystallography. Within crystallography, one of the specific goals is to make structural results accessible that might otherwise never be seen, and so the CGF will help increase the body of scientific knowledge and improve the return on federal investment in the large numbers of x-ray diffractometers and associated instruments nationwide. Although the project targets specifically a few hundreds of crystallography labs worldwide, the software and methods created in it are intended to be reusable for any science moving from individual lab practices to a shared, global collaboratory system. In sciences such as high-energy physics and astronomy, the scientists have long shared single, unique, large instruments and had to create shared data management and instrument metadata. CGF is likely to be useful in other scientific disciplines which still use widely-distributed lab-based instruments that now need to be linked in data grids.