Theme Recognition In Document Collections.

许多不同的方法已被调查的目的，聚类集的文件，希望提高检索。不幸的是，这些通常不能提供改进的检索能力。部分问题显然是，一份文件往往涉及一个以上的主题，因此不可能将文件明确归类为明确的类别，而排除其他类别。为了克服这个问题，我们已经开发出的方法，旨在确定一组文件中的主题。主题不必涵盖任何文件的全部。它只需要存在于文档的某个子集中即可识别。其中一些文件可能参与若干主题的界定。寻找主题的方法是基于EM算法，需要一个迭代过程，收敛到主题。该方法已被实施和测试，并发现是成功的。最近，该方法已得到改进，我们已经应用它来检测主题的数据库中的52K文件的主题艾滋病和超过200K文件处理的主题淋巴细胞。最近，我们完成了将其应用于所有MEDLINE，以定义107，863个主题。目前正在对这些方法进行测试，以了解如何使用它们来增强MEDLINE和其他数据库的检索。我们在这个项目上的最新工作是开发一种方法来产生一个相互关联的术语网络，因为它们彼此相关。这是为了取代个别主题。我们正在试验这个网络，作为增强检索的一种方式。

Many different methods have been investigated for the purpose of clustering sets of documents with the hope of improving retrieval. Unfortunately these have generally failed to provide improved retrieval capability. Part of the problem is clearly the fact that a given document often involves more than one subject so that it is not possible to make a clean categorization of the documents into definite categories to the exclusion of others. In order to overcome this problem we have developed methods that are designed to identify a theme among a set of documents. The theme need not encompass the whole of any document. It only needs to exist in some subset of the documents in order to be identifiable. Some of these same documents may participate in the definition of several themes. The method of finding themes is based on the EM algorithm and requires an iterative procedure which converges to themes. The method has been implemented and tested and found to be successful. Recently the method has been improved and we have applied it to detect themes in a database of 52k documents on the subject of AIDS and over 200k documents dealing with the subject of lymphocytes. Recently we completed applying it to all of MEDLINE to define 107,863 themes. These are now being tested to see how they may be used to enhance retrieval in MEDLINE and possibly other databases. Our most recent work on this project is to develop an approach to produce a network of terms that are linked to each other as they are relevant to each other. This serves to take the place of individual themes. We are experimenting with this network as a way to enhance retrieval.