Automatic Bayesian Methods In Text Retrieval

该项目目前的工作重点是开发一个改进的贝叶斯分类模型，并开发新的方法来主动学习贝叶斯模型。 第一章 在关于朴素贝叶斯机器学习方法的文献中，长期以来一直使用两种模型。一种是多元伯努利模型（MBM），另一种是多项模型（MM）。 MBM方法只计算某个特征是否存在，而MM方法计算某个特征在记录中出现的次数。在文本分类领域，已经对这两种方法进行了大量的比较，MM方法通常获胜。我们认为，这是因为MBM模型没有得到适当的优化。事实上，我们已经发现，在文本分类领域的本地词频几乎没有贡献的MM模型的性能。为了支持这一论点，我们开发了一种简化形式的MM模型，它忽略了本地词频（但仍然比MBM更接近MM），我们发现它的性能基本上与MM模型相同。事实上，我们没有找到一个优势，在文本分类使用MM或在其他几个模型，包括SVM的本地术语频率。忽略局部词频的优点是，当将朴素贝叶斯方法应用于非常大的数据库（如PubMed）时，它大大简化了数据存储和计算。这项工作需要进一步研究的一个方面是，当记录很长时会发生什么，本地频率会大得多。我们没有最终的答案，但我们对TREC基因组学数据（160，000份全文文档）的初步研究表明，在某些模型中使用局部术语频率有一个小的优势，但在任何情况下，这个优势都不会超过盈亏平衡点的3%。 （二） 我们已经开发了基于术语的主动学习方法，它提供了一种不同的主动学习方法，并表明它们在许多情况下比简单的不确定性采样或误差减少采样更有效。 第三章 PubMed数据库提出了一个独特的挑战，因为它的规模非常大，超过1900万条记录。由于这种规模，很少有机器学习方法可以在合理的周转时间内应用。一种可以有效应用的方法是朴素贝叶斯，但是当要区分的不同类表现出明显的大小差异时，它的性能很差。但这种不平衡对于希望在PubMed研究的问题来说是常见的。 在这种情况下，我们已经发现，可以通过主动学习启发的方法来选择比整个集合小得多的训练集合。结果产生了几乎200%的性能改进的朴素贝叶斯分类文件的MeSH长期分配。结果明显优于KNN方法，并且还有一个额外的优点，即以这种方式定义的最佳训练集可以用作更复杂的机器学习方法的训练集，其结果甚至比从朴素贝叶斯获得的结果更好。 4)我们使用基于两个泊松分布的概率计算来计算与文档相关的文档，一个用于文档中对文档内容更重要的术语，一个用于更外围的术语。这些被组合成一个概率估计的重要性，在一个文件中的一个术语的基础上，其在文件中的相对频率。该概率估计与术语的全局IDF权重相结合，以说明该术语在计算两个文档之间的相似性时的重要性。从开发这种方法的时候起，我们就知道它效果很好。在过去的几年里，TREC基因组学跟踪的数据已经可以使用，这使我们能够通过将其与Robertson及其同事开发的bm25公式的结果进行比较来测试这种方法。我们发现我们的概率方法有一个很小但在统计上很重要的优势。

Current work on the project is focusing on developing an improved Bayesian classification model and developing new approaches to active learning with a Bayesian model. 1) In the literature on the Naive Bayes machine learning method there have long been two models that have been used. One is the multivariate Bernoulli model (MBM) and the other the multinomial model (MM). The MBM method only counts the presence or absence of a feature, while the MM method counts the number of times a feature appears in a record. A number of comparisons of the two approaches have been made in the area of text categorization and the MM approach has usually won out. It is our belief that the reason for this is that the MBM model has not been properly optimized. In fact we have found that in the area of text categorization the local term frequency contributes virtually nothing to the performance of the MM model. In support of this contention we have developed a simplified form of the MM model which ignores local term frequency (but is still much closer to the MM than to the MBM) and we find that it performs essentially the same as the MM model. In fact we do not find an advantage for the use of local term frequency in text categorization using MM or in several other models including the SVM. The advantage to ignoring local term frequency is that it greatly simplifies the data storage and the calculations when applying the Naive Bayes approach to a very large database such as PubMed. One aspect of this work which calls for further investigation is what happens when the records are long and the local frequencies can then be much larger. We do not have the final answer, but our initial work with the TREC genomics data (160,000 full text documents) suggests that there is a small advantage in the use of local term frequencies in some models, but the advantage is not in any case over about a 3% improvement in break even point. 2) We have developed term based active learning methods which provide a different approach to active learning and have shown that they are in many cases more effective then simple uncertainty sampling or error reduction sampling. 3) The PubMed database presents a unique challenge because of its very large size of over 19 million records. Because of this size few machine learning methods can be applied with a reasonable turn-around time. One method that can be applied efficiently is Naive Bayes, but it performs poorly when the different classes to be distinguished exhibit a marked size discrepancy. But such an imbalance is common for the problems one wishes to study in PubMed. In such a situation we have discovered that a training set much smaller than the whole set can be selected by an active learning inspired method. The result yields an almost 200% improvement in the performance of Naive Bayes in classifying documents for MeSH term assignment. The results are significantly better than a KNN method and there is the added advantage that the optimal training sets defined in this way can be used as the training sets for more sophisticated machine learning methods with even better results than those obtained from Naive Bayes. 4) We compute the documents related to a document using a probability calculation based on two Poisson distributions, one for the terms in a document that are more central to the documents content and one for the terms that are more peripheral. These are combined into a probability estimate of the importance of a term in a document based on its relative frequency in the document. This probability estimate is combined with the global IDF weight of a term to account for that terms importance in computing the similarity between two documents. We have known from the time this approach was developed that it worked well. In the last several years data has become available in the TREC genomics track that has allowed us to test this approach by comparing it with theresults of the bm25 formula developed by Robertson and colleagues. We find a small but statistically significant advantage for our probabilistic approach.