The Statistical Evaluation Of Retrieval Efficiency

随着生物信息量的增长，数据库检索的效率已成为生物企业的核心问题。特别是，人们可以改变检索方法，并且必须能够评估该改变是否是改进。我们正在开发基于统计自举的方法，以赋予数据库检索改进的统计意义。这些方法基于描述自举下接收器工作特性曲线[n]的行为的数学中心极限定理。特别是，我们的方法已经被应用于确定PSI-BLAST计划的哪些变化实际上构成了改进。此外，我们正在调查检索中相关性的“同调性”，即假设在对相关性进行等级平均之后，按相关性的降序平均检索记录。等序假设影响检索效率的评估，初步结果表明，尽管它被广泛采用，但假设可能是错误的。

With the growth of biological information, the efficiency of database retrieval has become central to the biological enterprise. In particular, one can change a retrieval method and must be able to evaluate whether the change is an improvement or not. We are developing methods based on the statistical bootstrap to assign statistical significance to improvements in database retrieval. The methods are based on mathematical central limit theorems describing the behavior of the receiver operating characteristic curve [n] under bootstrapping. In particular, our methodology has already been applied to determine which changes to the PSI-BLAST program actually constitute improvements. In addition, we are investigating "isotonicity" of relevance in retrieval, the assumption that after rankwise averaging of relevance, records are retrieved on average in decreasing order of relevance. The isotonic assumption affects the evaluation of retrieval efficiency, and preliminary results indicate that despite its widespread adoption, the assumption can be wrong.