Automatic Bayesian Methods In Text Retrieval

该项目目前的工作重点是开发改进的贝叶斯分类模型，并开发使用贝叶斯模型进行主动学习的新方法。 1)在关于朴素贝叶斯机器学习方法的文献中，长期以来一直有两个模型被使用。一种是多元伯努利模型(MBM)，另一种是多项式模型(MM)。MBM方法只统计特征的存在或不存在，而MM方法计算特征在记录中出现的次数。在文本分类领域，人们对这两种方法进行了大量的比较，其中MM方法通常是胜出的。我们认为，造成这种情况的原因是MBM模式没有得到适当的优化。事实上，我们已经发现，在文本分类领域，局部词频对MM模型的性能几乎没有贡献。为了支持这一论点，我们开发了MM模型的简化形式，该模型忽略了局部项频率(但仍然比MBM更接近MM)，并且我们发现它的性能基本上与MM模型相同。事实上，我们没有发现在使用MM或包括支持向量机在内的其他几种模型的文本分类中使用局部词频的优势。忽略局部词频的优点是，在将朴素贝叶斯方法应用于像PubMed这样的大型数据库时，它极大地简化了数据存储和计算。这项工作的一个方面需要进一步研究，那就是当记录很长时会发生什么，而本地频率可能会更大。我们没有最终的答案，但我们使用TREC基因组学数据(160,000个全文文档)的初步工作表明，在某些模型中使用本地术语频率有一个小的优势，但在任何情况下，该优势都不会超过盈亏平衡点约3%的改善。 2)我们开发了基于项的主动学习方法，它们提供了一种不同的主动学习方法，并表明在许多情况下，它们比简单的不确定性采样或错误减少采样更有效。 3)PubMed数据库是一个独特的挑战，因为它的记录非常大，超过1900万条。由于这种规模，很少有机器学习方法可以在合理的周转时间内应用。一种可以有效应用的方法是朴素贝叶斯，但当要区分的不同类别表现出明显的大小差异时，它的性能很差。但对于人们希望在PubMed研究的问题来说，这种不平衡是很常见的。在这种情况下，我们发现可以通过一种主动的学习启发方法来选择比整个训练集小得多的训练集。结果表明，朴素贝叶斯在网格术语分配的文档分类中的性能提高了近200%。这种方法的结果明显优于KNN方法，而且这样定义的最优训练集可以用作更复杂的机器学习方法的训练集，其结果甚至比用朴素贝叶斯方法得到的结果更好。 4)我们使用基于两个泊松分布的概率计算来计算与文档相关的文档，一个用于文档中对文档内容更重要的术语，另一个用于较边缘的术语。这些被组合成基于术语在文档中的相对频率的术语在文档中的重要性的概率估计。该概率估计与术语的全局IDF权重相结合，以说明该术语在计算两个文档之间的相似度时的重要性。从开发这种方法的时候起，我们就知道它工作得很好。在过去的几年里，TREC基因组学轨道上的数据已经变得可用，这使得我们能够通过将其与Robertson和他的同事开发的BM25公式的结果进行比较来测试这种方法。我们发现我们的概率方法有一个很小但在统计上显著的优势。

Current work on the project is focusing on developing an improved Bayesian classification model and developing new approaches to active learning with a Bayesian model. 1) In the literature on the Naive Bayes machine learning method there have long been two models that have been used. One is the multivariate Bernoulli model (MBM) and the other the multinomial model (MM). The MBM method only counts the presence or absence of a feature, while the MM method counts the number of times a feature appears in a record. A number of comparisons of the two approaches have been made in the area of text categorization and the MM approach has usually won out. It is our belief that the reason for this is that the MBM model has not been properly optimized. In fact we have found that in the area of text categorization the local term frequency contributes virtually nothing to the performance of the MM model. In support of this contention we have developed a simplified form of the MM model which ignores local term frequency (but is still much closer to the MM than to the MBM) and we find that it performs essentially the same as the MM model. In fact we do not find an advantage for the use of local term frequency in text categorization using MM or in several other models including the SVM. The advantage to ignoring local term frequency is that it greatly simplifies the data storage and the calculations when applying the Naive Bayes approach to a very large database such as PubMed. One aspect of this work which calls for further investigation is what happens when the records are long and the local frequencies can then be much larger. We do not have the final answer, but our initial work with the TREC genomics data (160,000 full text documents) suggests that there is a small advantage in the use of local term frequencies in some models, but the advantage is not in any case over about a 3% improvement in break even point. 2) We have developed term based active learning methods which provide a different approach to active learning and have shown that they are in many cases more effective then simple uncertainty sampling or error reduction sampling. 3) The PubMed database presents a unique challenge because of its very large size of over 19 million records. Because of this size few machine learning methods can be applied with a reasonable turn-around time. One method that can be applied efficiently is Naive Bayes, but it performs poorly when the different classes to be distinguished exhibit a marked size discrepancy. But such an imbalance is common for the problems one wishes to study in PubMed. In such a situation we have discovered that a training set much smaller than the whole set can be selected by an active learning inspired method. The result yields an almost 200% improvement in the performance of Naive Bayes in classifying documents for MeSH term assignment. The results are significantly better than a KNN method and there is the added advantage that the optimal training sets defined in this way can be used as the training sets for more sophisticated machine learning methods with even better results than those obtained from Naive Bayes. 4) We compute the documents related to a document using a probability calculation based on two Poisson distributions, one for the terms in a document that are more central to the documents content and one for the terms that are more peripheral. These are combined into a probability estimate of the importance of a term in a document based on its relative frequency in the document. This probability estimate is combined with the global IDF weight of a term to account for that terms importance in computing the similarity between two documents. We have known from the time this approach was developed that it worked well. In the last several years data has become available in the TREC genomics track that has allowed us to test this approach by comparing it with theresults of the bm25 formula developed by Robertson and colleagues. We find a small but statistically significant advantage for our probabilistic approach.