EVALUATING KNOWLEDGE-BASED EXPERT SYSTEM FOR INDEXING

这项研究将制定详细的书面实验设计 基于人工智能的基于知识的专家系统评价 智能(AI)技术，用于在 国家医学图书馆(NLM)。1956年，公法83-941赋予NLM 国会授权帮助传播和交流 对医学和公共卫生的进步很重要的信息， 将索引确定为执行它的一项功能。NLM目前 每年为大约40万份文档编制索引，为其享誉世界的公开- 可用的MEDLINE数据库。作为KBES，这项研究将使用一个独特的， 称为MedIndExTM(医学索引专家)的高级原型 在NLM开发的。这项研究开发的设计将基于 为提高索引器的性能而开发系统的研究目标。 评估的目的应该是回答一般性研究问题 是，KBES是否影响索引器的性能；具体的研究问题 关联系统功能(知识内容/组织和用户 接口)与质量和效率的绩效衡量标准。基座 在这些目标和问题上，研究将制定任务和措施 在评价中使用，特别是结构化索引任务，以 模拟典型的索引问题。由该公司开发的设计 研究将是早期进行评估的基础 在生产中采用KBES的可行性评估 环境。由于以前没有这样的系统，所以本设计 将建立一种评估KBES以编制索引的方法。 这项研究的信息将通过传统机制报告 (会议记录、期刊、机构报告)。

This study will develop a detailed, written experimental design for evaluating knowledge-based expert systems (KBESs), based on artificial intelligence (AI) techniques, for indexing biomedical information at the National Library of Medicine (NLM). In 1956, Public Law 83-941 gave NLM a congressional mandate to aid the dissemination and exchange of information important to the progress of medicine and public health, identifying indexing as a function to carry it out. NLM currently indexes about 400,000 documents a year for its world-renowned, publicly- available MEDLINE database. As the KBES, this study will use a unique, advanced prototype known as MedIndExTM (Medical Indexing Expert) developed at NLM. The design developed by this study will be based on the research goal of developing systems to improve indexers' performance. The general research question the evaluation should be designed to answer is, does a KBES affect indexers' performance; specific research questions correlate system features (knowledge content/organization and user interface) with performance measures of quality and efficiency. Based on these goals and questions, the study will develop tasks and measures to be used in the evaluation, in particular structured indexing tasks to simulate representative indexing problems. The design developed by this study will be the basis for an evaluation to be conducted as an early assessment of the feasibility of adopting KBESs in a production environment. Since such systems have not existed previously, this design will establish a methodology for evaluating KBESs for indexing. Information from this study will be reported via traditional mechanisms (proceedings, journals, agency reports).