Assessment of predictions in the model quality assessment category
Assessment of predictions in the model quality assessment category
复制标题
DOI：
10.1002/prot.21669
复制
发表时间：
2007-01-01
影响因子：
2.9
通讯作者：
Tramontano, Anna
中科院分区：
文献类型：
--
作者：
Cozzetto, Domenico;Kryshtafovych, Andriy;Tramontano, Anna
本文介绍了我们对CASP7中提交给模型质量评估（QA）类别的预测的评估。在这个新引入的类别中，预测器被要求为蛋白质结构模型提供质量估计。QA类别使用自动生成的模型，这些模型传统上分发给CASP参与者作为预测的输入。要求预测者提供这些单个模型的质量指数（QM1）以及每个残差的预期正确性指数（QM2）。我们计算了模型的观测质量和预测质量之间的相关性，以及参与组获得的单个残差之间的相关性，并评估了差异的统计学显著性。我们还将结果与通过“朴素预测器”获得的结果进行了比较，该预测器分配了与模型与已知结构的最相似蛋白质的结构有多接近相关的质量分数。评估模型总体质量的方法的目的可以是双重的：从一组合理的选择中选择最佳（或最佳之一）模型，或者为单个模型分配非相对质量值。这两种战略的应用是不同的，但同样重要。我们对质量保证类别的评估表明，确实存在有效解决第一项任务的方法，而就第二个方面而言，还有改进的余地。尽管提交残差水平准确性预测的群体数量有限，但我们的数据表明，通过依赖于对同一目标的不同模型进行比较的方法，可以实现该任务中相当高的准确性。
The article presents our evaluation of the predictions submitted to the model quality assessment (QA) category in CASP7. In this newly introduced category, predictors were asked to provide quality estimates for protein structure models. The QA category uses the automatically produced models that are traditionally distributed to CASP participants as input for predictions. Predictors were asked to provide an index of the quality of these individual models (QM1) as well as an index for the expected correctness of each of their residues (QM2). We computed the correlation between the observed and predicted quality of the models and of the individual residues achieved by the participating groups and evaluated the statistical significance of the differences. We also compared the results with those obtained by a "naive predictor" that assigns a quality score related to how close the model is to the structure of the most similar protein of known structure. The aims of a method for assessing the overall quality of a model can be twofold: selecting the best (or one of the best) model(s) among a set of plausible choices, or assigning a nonrelative quality value to an individual model. The applications of the two strategies are different, albeit equally important. Our assessment of the QA category demonstrates that methods for addressing the first task effectively do exist, while there is room for improvement as far as the second aspect is concerned. Notwithstanding the limited number of groups submitting predictions for residue-level accuracy, our data demonstrate that a respectable accuracy in this task can be achieved by methods relying on the comparison of different models for the same target.