A DOCUMENT PROCESSING SYSTEM

实现了一个C语言程序系统 为查找密切相关的文档而开发的 在梅德莱恩。该系统有许多独特的特点：1)它是 高度模块化，因此系统中的更改相对简单 去表演。2)该系统目前操作的是 ASN1格式，但更改了系统的界面部分 将允许它应用于任何大型数据库，这些数据库包括 离散的文本记录。3)系统设计具有一定程度的可操作性 安全，防止因操作系统崩溃或 停电。4)系统处理的所有数据都存储在 永久形式，如倒置文件结构等。这些结构是 可更新，以便可以不断向系统添加新数据 当它可用时。5)将文档与每个文档进行比较 其他使用贝叶斯形式的分析和关于以下内容的统计 术语的相关性权重是基于从先前的 文档比较。这些统计数据会随着每个新的 处理周期。关于这个系统的最新工作涉及到 开发了一种对短文档进行分数校正的方法。 短文档往往对其他短文档人为地打高分 文档，这会导致不正确的检索。已经有了一个公式 制定了纠正这种反常的方法，而且它已经 实施。

A system of C language programs has been developed for the purpose of finding the closely related documents in Medline. The system has a number of unique features: 1) It is highly modular so that alterations in the system are relatively simple to perform. 2) The system currently operates on Medline data in the ASN1 format but a change in the interface portion of the system would allow it to be applied to any large database consisting of discrete textual records. 3) The system is designed with a degree of security against loss of data due to operating system crashes or power outages. 4) All data processed by the system is stored in permanent form as inverted file structures, etc. These structures are updatable so that new data may be continually added to the system as it becomes available. 5) Documents are compared with each other using a Bayesian form of analysis and the statistics on which the relevance weighting of terms is based are derived from previous document comparisons. These statistics are updated with each new cycle of processing. The latest work on this system has involved developing a method to correct the scores of short documents. Short documents tend to score artificially high against other short documents and this leads to incorrect retrieval. A formula has been worked out to correct for this anomally and it has been implemented.