Integrated resource for reproducibility in macromolecular crystallography

 描述(由申请人提供)：我们建议开发一套数据处理工具，用于存储、解析、操作、验证、整理、分析和传播大分子衍射图像以及所有相关的元数据。所提议的系统将具有几个好处，通过(1)创建一种随着处理衍射图像的技术进步而改进现有结构的手段，(2)检测现有结构中的错误(以及潜在的欺诈)以确保结构质量和重现性，(3)防止由结构基因组学和其他已关闭或将关闭的程序收集的数据的丢失，(4)为漫射衍射效应的分析提供数据，以及(5)为新的衍射分析算法和硬件建立“训练集”。生物学家，生物信息学家， 而软件和硬件开发人员都将成为这些工具的受益者。这项研究旨在对衍射图像进行语义分析，而不是句法分析，并有几个具体的目标。首先，我们将开发工具，用于自动提取和精选衍射图像和相关元数据，并在结构确定方法改进时产生对重新处理所需的所有数据的描述。其次，我们将创建一个基于Web的系统，用于以机器可读格式组织、搜索、分析和数据挖掘适当的衍射图和相关元数据子集。这将包括用于编程访问的全面API、将多个实例链接到分布式联邦的能力，以及最先进的压缩和传输技术。第三，我们将开发工具来自动验证、预处理和评分衍射图像，并检测潜在的问题和错误。这些工具将利用新的和现有的程序进行图像和数据分析，包含启发式方法来确定可能的错误，并提供统计数据以将错误与特定的元数据相关联。第四，我们将创建一种机制，用目前可用的方法发现尚未产生X射线结构的衍射数据。第五，我们 我将建立一个包含所有已开发工具的试点资源，并收集测试数据集 开发新的验证和错误检测工具。我们将与多个合作者密切合作。最重要的是RCSB蛋白质数据库(PDB)，它将帮助我们确保衍射元数据的准确性和完整性。其他合作伙伴将包括漫射X射线散射界、探测器供应商、同步加速器光束线管理者、IUCr衍射数据沉积工作组成员和整个结晶学社区。我们将与RCSB PDB一起组织与这些社区的研讨会(S)，以便(A)改进元数据提取和(B)更好地定义衍射图像的子集。通过解决数据简化过程中常见的、不可逆的和不必要的原始衍射数据丢失问题，我们的项目有助于确保高分子结晶学学科能够不断自我完善。

 DESCRIPTION (provided by applicant): We propose the development of a collection of data wrangling tools to store, parse, manipulate, validate, curate, analyze, and disseminate macromolecular diffraction images together with all associated relevant metadata. The proposed system will have several benefits, by (1) creating a means to improve existing structures as technology for processing diffraction image advances, (2) detecting errors (and potentially, fraud) in existing structures to ensure structure quality and reproducibility, (3) preventing the loss of data collected by structural genomics and other programs that have closed or will close, (4) providing data for analysis of diffuse diffraction effects, and (5) buildng a "training set" for new diffraction analysis algorithms and hardware. Biologists, bioinformaticians, and software and hardware developers will all be beneficiaries of these tools. The proposed research is designed for semantic rather than syntactic analysis of diffraction images, and has several specific goals. First, we will develop tools for automatically extracting and curating diffraction images and associated metadata, as well as producing descriptions of all data needed for reprocessing when methods for structure determination improve. Second, we will create a web-based system for organizing, searching, analyzing, and data mining of appropriate subsets of diffraction images and associated metadata in machine- readable formats. This will include a comprehensive API for programmatic access, the ability to link multiple instances into a distributed federation, and state-of-the-art compression and transfer technologies. Third, we will develop tools to automatically validate, preprocess, and score diffraction images, and to detect potential issues and errors. These tools will make use of new and existing programs for image and data analysis, contain heuristics to identify possible errors, and provide statistics to correlate errors with specific metadata. Fourth, we will create a mechanism to discover diffraction data that have not yielded X-ray structures with currently available methods. Fifth, we will set up a pilot resource incorporating all the developed tools, and collect a test data set for the development of new tools for validation and error detection. We will work closely with multiple collaborators. Most important is the RCSB Protein Data Bank (PDB), who will help us ensure the accuracy and completeness of the diffraction metadata. Other partners will include the diffuse X-ray scattering community, detector vendors, synchrotron beamline managers, members of the IUCr Diffraction Data Deposition Working Group (DDDWG) and the crystallographic community in general. Together with the RCSB PDB, we will organize workshop(s) with these communities in order to (a) improve metadata extraction and (b) better define subsets of diffraction images. By addressing the currently common, irreversible and unnecessary loss of raw diffraction data during the data reduction process, our project helps ensure that the discipline of macromolecular crystallography is capable of continuous self-improvement.