LiveBench-1: Continuous benchmarking of protein structure prediction servers
LiveBench-1: Continuous benchmarking of protein structure prediction servers
复制标题
DOI：
10.1110/ps.40501
复制
发表时间：
2001-02-01
期刊：
影响因子：
8
通讯作者：
Rychlewski, L
中科院分区：
文献类型：
--
作者：
Bujnicki, JM;Elofsson, A;Rychlewski, L
我们提出了一种新颖的连续方法，旨在大规模评估可用折叠识别服务器的性能。研究了六种流行的服务器：PDB-Blast、FFAS、T98-lib、Gen-THREADER、3D-PSSM 和 INBGU。该评估是使用 1999 年 10 月至 2000 年 4 月发布的大量选定蛋白质结构作为预测目标进行的。如果目标序列与结构数据库中先前可用的任何蛋白质没有显着相似性，则选择目标。总体而言，服务器能够为一半的目标生成结构相似的模型，但仅为三分之一的目标生成非常准确的序列结构比对。我们进一步将目标分为两类：简单目标和困难目标。我们发现，如果服务器的折叠库中存在结构相似的折叠，则所有服务器都能够找到绝大多数简单目标的正确答案。然而，在硬目标中（PSI-BLAST 等标准方法失败），最敏感的折叠识别服务器只能为 40% 的情况生成类似的模型，其中一半具有非常准确的序列结构比对。在硬目标中，更新库的存在似乎对排名不太重要。考虑所有服务器的结果的“理想组合共识”预测将使正确分配的百分比增加 50%。每个服务器都有许多分配正确的情况，而所有其他服务器的分配都是错误的。这强调了在困难的预测任务中考虑多个服务器的好处。 LiveBench计划（http://BioInfo.PL/LiveBench）仍在继续，诚挚邀请所有感兴趣的开发者加入。
We present a novel, continuous approach aimed at the large-scale assessment of the performance of available fold-recognition servers. Six popular servers were investigated: PDB-Blast, FFAS, T98-lib, Gen-THREADER, 3D-PSSM, and INBGU. The assessment was conducted using as prediction targets a large number of selected protein structures released from October 1999 to April 2000. A target was selected if its sequence showed no significant similarity to any of the proteins previously available in the structural database. Overall, the servers were able to produce structurally similar models for one-half of the targets, but significantly accurate sequence-structure alignments were produced for only one-third of the targets. We further classified the targets into two sets: easy and hard. We found that all servers were able to find the correct answer for the vast majority of the easy targets if a structurally similar fold was present in the server's fold libraries. However, among the hard targets--where standard methods such as PSI-BLAST fail--the most sensitive fold-recognition servers were able to produce similar models for only 40% of the cases, half of which had a significantly accurate sequence-structure alignment. Among the hard targets, the presence of updated libraries appeared to be less critical for the ranking. An "ideally combined consensus" prediction, where the results of all servers are considered, would increase the percentage of correct assignments by 50%. Each server had a number of cases with a correct assignment, where the assignments of all the other servers were wrong. This emphasizes the benefits of considering more than one server in difficult prediction tasks. The LiveBench program (http://BioInfo.PL/LiveBench) is being continued, and all interested developers are cordially invited to join.