Prospective analysis to determine model accuracy performance and boundaries in the post-AlphaFold2 environment

蛋白质结构的实验测定通常提供原子精度模型，但本质上是时间- 消耗巨大，成本高昂，而且并非总是可行。计算建模提供了另一种选择。深度应用最新进展 结构建模的学习方法现在允许为大规模的建筑物生成高度精确的结构模型。 大多数单一蛋白质。蛋白质复合物建模的建模精度方面的进展也有望在 近期这些发展极大地扩展了模型在生物学和医学中的应用。 然而，即使是高精度的模型也不等同于实验结果。而 实验结构有既定的标准和程序来确定准确性，模型充其量是注释 准确的预测。越来越多地使用模型强调了需要类似的知识，规范， 模型的程序。结构预测的批判性评估（CASP，由家长资助），已经是 重点是评估建模技术的性能和估计模型的方法的性能 准确性，为此，目前正在获取大量额外的数据，作为CASP 15实验的一部分。而是一 在足够的条件范围内详细评估准确性和准确性预测特性需要更多 比通过传统的CASP程序获得的信息更广泛。近百万款车型的发布 从DeepMind获得的AlphaFold 2方法创造了执行此类分析的机会。 我们将使用这些数据对模型准确性和准确性预测性能进行前瞻性分析。 具体来说，我们将比较结构释放的PDB与相应的先前存放的模型。我们将 评估这些实验和计算结构之间的一致性，包括整体和单个氨基酸 酸水平，以及预测和实际精度之间的一致性。严格地说，我们将把这些因素作为一个 相关变量的功能，特别是实验信息的质量和类型，环境影响 例如晶体堆积、功能相关结构特征、稀有结构特征、蛋白质界面区域、序列 长度和用于生成模型的可用氨基酸序列比对的深度。我们还将构建工具 分析和可视化这些数据之间的关系和相互依存关系，以促进和扩大我们的 了解模型精度性能。这项工作的总体目标是提供一个细粒度的景观， 使用当前建模方法获得的准确度，以及每种情况下准确度的置信限。虽然 目前的分析仅限于AlphaFold 2模型，最重要的变量的识别应该允许我们应用 在常规CASP实验中获得的较小数据集中代表的方法范围的程序，以及 深度学习方法在结构生物学中的新应用，例如蛋白质复合物的建模。 我们已经拥有的基础设施和专业知识，包括广泛的准确性分析工具，以及 我们在精度、精度估算方法和数据库管理方面的丰富经验，使我们成为 合适的团队来完成这项任务。拟议的研究福尔斯完全属于当前R 01的科学范围。

Experimental determination of protein structure often provides atomic accuracy models, but is inherently time- consuming, costly, and not always possible. Computational modeling offers an alternative. Recent application of deep learning methods to structure modeling now allows the generation of highly accurate models of structure for the great majority of single proteins. Progress in modeling accuracy for the modeling of protein complexes is also expected in the near term. These developments have dramatically expanded the utility of models in biology and medicine. However, having even high accuracy models at one’s disposal is not the same as an experimental result. While experimental structures come with established norms and procedures to ascertain accuracy, models are at best annotated with predictions of accuracy. The increased use of models underscores the need for similar knowledge, norms, and procedures for models. The Critical Assessment of Structure Prediction (CASP, funded by the parent grant), is already focused on assessing performance of modeling techniques and the performance of methods for estimating model accuracy, and to this end, extensive additional data are currently being acquired as part of the CASP15 experiment. But a detailed assessment of accuracy and accuracy prediction properties under an adequate range of conditions requires more extensive information than will be available through traditional CASP procedures. The release of close to a million models obtained with the AlphaFold2 method from DeepMind has created an opportunity to perform such an analysis. We will use these data to perform a prospective analysis of model accuracy and accuracy prediction performance. Specifically, we will compare structures released by the PDB with the corresponding previously deposited models. We will assess the agreement between these experimental and computed structures, both overall and at the individual amino acid level, and the agreement between the predicted and actual accuracy. Critically, we will examine these factors as a function of the relevant variables, particularly the quality and type of the experimental information, environmental effects such as crystal packing, function related structural features, rare structural features, protein interface regions, sequence length, and the depth of the available amino acid sequence alignment used to generate the model. We will also build tools to analyze and visualize the relationships and interdependencies between these data, to facilitate and expand our understanding of model accuracy performance. The overall goal of this work is to provide a fine-grained landscape of accuracy obtained with current modeling methods, and confidence limits on accuracy in each situation. Although the present analysis is restricted to AlphaFold2 models, identification of the most important variables should allow us to apply the procedures to the range of methods represented in the smaller data sets obtained in regular CASP experiments, and to new applications of deep learning methods in structural biology, for example the modeling of protein complexes. The infrastructure and expertise we already have in place, including the wide range of accuracy analysis tools, and our extensive experience with accuracy, accuracy estimation methods, and database management, make us the appropriate group to address this task. The proposed research falls well within the scientific scope of the current R01.