AUTOMATIC BAYESIAN METHODS IN TEXT RETRIEVAL

一种自动化的方法，应用贝叶斯方法在文本检索， 开发了这是搜索词的相关性加权的一种形式， 从两个方面偏离了通常的方法，它们相互补充。 首先，通常的方法涉及相关权重的分配， 基于文档的单个查询中的搜索项， 那些与查询无关的。这通常涉及一个小的 相关文件的数量，因此统计样本， 很难用它来做任何全球性的重要推论， 所涉及的条款的价值。我们修改通常的方法， 一个词在所有查询中的平均重要性， 发生。我们研究了查询集是文档集的情况 从而全局术语相关性权重是定义良好的概念。 第二，通常的方法仅限于当人 判断文档与查询的相关性。这限制了 将该方法用于某些测试集，其中相关关系 已知的或相关反馈情况。我们的做法是， 相关性关系由高得分对的查询和 使用矢量余弦法对文档进行检索。因为后者 是一种自动方法，我们能够生成所需的统计数据， 自动的方式。虽然后一种方法无疑会有更多的 相关性判断的误差比人类涉及的样本量更大 全球加权有助于抵消这一问题。局部权重为 以特别的方式引入，并且发现结果检索是 比矢量余弦检索稍有上级。 这是刚刚描述的模型的两个问题。首先， 以自然贝叶斯方式结合局部项加权， 它不提供对文档长度的校正。我们目前正在 研究可能的方法来纠正这些缺陷， 贝叶斯框架

An automatic approach to applying Bayesian methods in text retrieval has been developed. This is a form of relevance weighting of search terms but departs from the usual approach in two ways which complement each other. First, the usual approach involves the assignment of relevance weights to the search terms in a single query based on the documents that are and those that are not relevant to the query. This involves generally a small number of relevant documents and hence a statistical sample that is difficult to use in making any globally significant inferences about the value of the terms involved. We modify the usual approach by taking the average of the importance of a term over all the queries in which it occurs. We study the case when the set of queries is the set of documents so that the global term relevance weight is a well defined concept. Second, the usual approach is limited to the case when one has human judgments of the relevance of documents to queries. This has limited the use of the method to certain test sets where the relevance relation is known or to relevance feedback situations. Our approach is to replace the relevance relation by the relation of high scoring pairs of query and document using the vector cosine method of retrieval. Because the latter is an automatic method we are able to generate the required statistics in an automatic manner. While this latter approach will undoubtedly have more error than human relevance judgments the larger sample size involved in global weighting helps to offset this problem. Local weighting is introduced in an ad hoc manner and the resultant retrieval is found to be somewhat superior to vector cosine retrieval. Their are two problems with the model just described. First it does not incorporate local term weighting in a natural Bayesian manner and second it does not provide a correction for document length. We are currently investigating possible approaches to correcting these deficits within the Bayesian framework.