Annotation, development and evaluation for clinical information extraction

描述（由申请人提供）：准确的临床研究、积极的决策支持和广泛覆盖的监测所需的大部分临床信息被锁定在电子病历（EMR）的文本文件中。在翻译科学中利用这些信息的唯一可行方法是使用自然语言处理（NLP）提取和编码信息。在过去的二十年里，几个研究小组已经开发了用于临床笔记的NLP工具，但阻碍临床NLP进展的主要瓶颈是缺乏用于训练和评估NLP应用的标准、注释数据集。如果没有这些标准，单个NLP应用程序将无法在标准注释上训练不同的算法，无法共享和集成NLP模块，也无法比较性能。我们建议开发标准和基础设施，使技术能够从文本医疗记录中提取科学信息，我们建议将这项研究作为涉及美国NLP专家的合作努力来实现这一目标，我们将解决三个具体目标：目标1：扩展现有标准并开发新的共识标准，以一种可互操作、可扩展和可用的方式注释临床文本。目标2：应用现有的方法和工具，并在必要时开发新的方法和工具，以高效和准确的方式手动注释一组公开可用的临床文本。目标3：开发一个公开可用的工具包，用于自动注释临床文本，并使用多维和灵活的评估指标执行共享评估来评估工具包。

DESCRIPTION (provided by applicant): Much of the clinical information required for accurate clinical research, active decision support, and broad-coverage surveillance is locked in text files in an electronic medical record (EMR). The only feasible way to leverage this information for translational science is to extract and encode the information using natural language processing (NLP). Over the last two decades, several research groups have developed NLP tools for clinical notes, but a major bottleneck preventing progress in clinical NLP is the lack of standard, annotated data sets for training and evaluating NLP applications. Without these standards, individual NLP applications abound without the ability to train different algorithms on standard annotations, share and integrate NLP modules, or compare performance. We propose to develop standards and infrastructure that can enable technology to extract scientific information from textual medical records, and we propose the research as a collaborative effort involving NLP experts across the U.S. To accomplish this goal, we will address three specific aims: Aim 1: Extend existing standards and develop new consensus standards for annotating clinical text in a way that is interoperable, extensible, and usable. Aim 2: Apply existing methods and tools, and develop new methods and tools where necessary for manually annotating a set of publicly available clinical texts in a way that is efficient and accurate. Aim 3: Develop a publicly available toolkit for automatically annotating clinical text and perform a shared evaluation to evaluate the toolkit, using evaluation metrics that are multidimensional and flexible.