Integrated resource for reproducibility in macromolecular crystallography

 描述（由申请人提供）：我们提出开发一系列数据整理工具，以存储、解析、操作、验证、策划、分析和传播大分子衍射图像以及所有相关的相关元数据。所提出的系统将有几个好处，通过（1）创建一种手段，以改善现有的结构，作为技术处理衍射图像的进步，（2）检测错误（以及潜在的欺诈），以确保结构质量和可再现性，（3）防止结构基因组学和其他已经关闭或即将关闭的项目收集的数据丢失，（4）为漫衍射效应的分析提供数据;（5）为新的衍射分析算法和硬件建立“训练集”。生物学家，生物信息学家， 软件和硬件开发人员都将成为这些工具的受益者。拟议的研究是专为语义而不是句法分析的衍射图像，并有几个具体的目标。首先，我们将开发用于自动提取和管理衍射图像和相关元数据的工具，以及在结构确定方法改进时生成再处理所需的所有数据的描述。其次，我们将建立一个基于网络的系统，用于组织、搜索、分析和数据挖掘衍射图像的适当子集和机器可读格式的相关元数据。这将包括用于编程访问的全面API、将多个实例链接到分布式联合的能力，以及最先进的压缩和传输技术。第三，我们将开发工具来自动验证，预处理和评分衍射图像，并检测潜在的问题和错误。这些工具将利用新的和现有的程序进行图像和数据分析，包含识别可能错误的统计数据，并提供统计数据以将错误与特定元数据相关联。第四，我们将创建一个机制来发现衍射数据，这些数据还没有用目前可用的方法产生X射线结构。五是 将建立一个试点资源，包括所有开发的工具，并收集测试数据集， 开发用于验证和错误检测的新工具。我们将与多个合作伙伴密切合作。最重要的是RCSB蛋白质数据库（PDB），它将帮助我们确保衍射元数据的准确性和完整性。其他合作伙伴将包括扩散X射线散射社区，探测器供应商，同步加速器光束线管理人员，IUCr衍射数据沉积工作组（DDDWG）的成员和晶体学社区。与RCS B PD B一起，我们将与这些社区组织研讨会，以便（a）改进元数据提取和（B）更好地定义衍射图像的子集。通过解决当前常见的，不可逆的和不必要的原始衍射数据在数据处理过程中的损失，我们的项目有助于确保大分子晶体学学科能够不断自我完善。

 DESCRIPTION (provided by applicant): We propose the development of a collection of data wrangling tools to store, parse, manipulate, validate, curate, analyze, and disseminate macromolecular diffraction images together with all associated relevant metadata. The proposed system will have several benefits, by (1) creating a means to improve existing structures as technology for processing diffraction image advances, (2) detecting errors (and potentially, fraud) in existing structures to ensure structure quality and reproducibility, (3) preventing the loss of data collected by structural genomics and other programs that have closed or will close, (4) providing data for analysis of diffuse diffraction effects, and (5) buildng a "training set" for new diffraction analysis algorithms and hardware. Biologists, bioinformaticians, and software and hardware developers will all be beneficiaries of these tools. The proposed research is designed for semantic rather than syntactic analysis of diffraction images, and has several specific goals. First, we will develop tools for automatically extracting and curating diffraction images and associated metadata, as well as producing descriptions of all data needed for reprocessing when methods for structure determination improve. Second, we will create a web-based system for organizing, searching, analyzing, and data mining of appropriate subsets of diffraction images and associated metadata in machine- readable formats. This will include a comprehensive API for programmatic access, the ability to link multiple instances into a distributed federation, and state-of-the-art compression and transfer technologies. Third, we will develop tools to automatically validate, preprocess, and score diffraction images, and to detect potential issues and errors. These tools will make use of new and existing programs for image and data analysis, contain heuristics to identify possible errors, and provide statistics to correlate errors with specific metadata. Fourth, we will create a mechanism to discover diffraction data that have not yielded X-ray structures with currently available methods. Fifth, we will set up a pilot resource incorporating all the developed tools, and collect a test data set for the development of new tools for validation and error detection. We will work closely with multiple collaborators. Most important is the RCSB Protein Data Bank (PDB), who will help us ensure the accuracy and completeness of the diffraction metadata. Other partners will include the diffuse X-ray scattering community, detector vendors, synchrotron beamline managers, members of the IUCr Diffraction Data Deposition Working Group (DDDWG) and the crystallographic community in general. Together with the RCSB PDB, we will organize workshop(s) with these communities in order to (a) improve metadata extraction and (b) better define subsets of diffraction images. By addressing the currently common, irreversible and unnecessary loss of raw diffraction data during the data reduction process, our project helps ensure that the discipline of macromolecular crystallography is capable of continuous self-improvement.