Semi-Automating Data Extraction for Systematic Reviews

 描述(由申请人提供)：循证医学(EBM)希望通过所有可用的相关证据为患者护理提供信息。系统评估是循证医学的基石，对现代医疗至关重要，为从国家卫生政策到床边决策的方方面面提供信息。但是，进行系统的评审是非常费力的(因此也是昂贵的)：制作一份评审需要数千个工时。此外，生物医学文献库的指数级扩张给评审者带来了前所未有的负担，从而使成本成倍增加。研究人员不再能跟上主要文献，这阻碍了循证护理的实践。 这项工作的长期目标是开发优化循证医学实践的计算工具和方法。因此，拟议的工作建立在我们以前成功开发计算方法以减少循证医学工作量的基础上。更具体地说，我们的目标是开发工具，通过新的机器学习方法，半自动化数据提取的繁琐任务-从生物医学文章的免费文本中识别和提取感兴趣的信息(例如，试验样本大小、干预和结果)。将这项任务半自动化将大大减少审查员的工作量，从而使循证医学在信息过载的时代得以实施。 以前从描述临床试验的文章中自动提取数据的努力显示出了希望，但缺乏现实世界使用所需的准确性和范围。这些方法由于缺乏大量带注释的临床试验语料库，以及构建模型以自动提取合成所需的所有变量的困难而受到阻碍。我们描述了克服这些障碍的方法创新。首先，为了训练我们的机器学习模型，我们建议利用包含临床试验结构化信息的大型现有数据库，而不是通常收集昂贵的手动注释的方法。实际上，这意味着我们将能够利用一个非常大的“伪注释”数据集，该数据集比以前的工作中使用的数据集大一个数量级，从而大大提高了模型的性能。我们广泛的前期工作证明了这种方法的前景和可行性。其次，我们提出了适合于循证医学的文章分类和数据抽取任务的机器学习模型。这些模型将被特别地设计成执行多个相关的感兴趣的数据元素的提取，同时将文章分类为对循证医学有用的临床显著类别。 我们将严格评估开发的方法，以评估它们的实用价值，特别是将自动提取的精度与训练有素的系统评审员所达到的精度进行比较。为了使这些方法对最终用户(系统评价者)有用，我们将开发和评估开源软件和工具，包括一个基于Web的提取工具，该工具集成了我们的机器学习模型，可以自动从上传的文章(PDF)中提取信息。我们将进行用户研究，以评估该工具在实践中的实用性和可用性。

 DESCRIPTION (provided by applicant): Evidence-based medicine (EBM) looks to inform patient care with the totality of available relevant evidence. Systematic reviews are the cornerstone of EBM and are critical to modern healthcare, informing everything from national health policy to bedside decision-making. But conducting systematic reviews is extremely laborious (and hence expensive): producing a single review requires thousands of person-hours. Moreover, the exponential expansion of the biomedical literature base has imposed an unprecedented burden on reviewers, thus multiplying costs. Researchers can no longer keep up with the primary literature, and this hinders the practice of evidence-based care. The long term aim of this work is to develop computational tools and methods that optimize the practice of EBM. The proposed work thus builds upon our previous successful efforts developing computational approaches that reduce the workload in EBM. More specifically, we aim to develop tools that semi-automate the laborious task of data extraction - identifying and extracting the information of interest (e.g., trial sample size, interventions and outcomes) from the free-texts of biomedical articles - via novel machine learning methods. Semi-automating this task will drastically reduce reviewer workload, thus enabling the practice of EBM in an age of information overload. Previous efforts to automate data extraction from articles describing clinical trials have shown promise, but lack the accuracy and scope necessary for real-world use. These approaches have been impeded by the absence of a large corpus of annotated clinical trials, and by the difficulty of constructing models to automatically extract all of the variables necessary for synthesis. We describe methodological innovations to overcome these hurdles. First, to train our machine learning models we propose leveraging large existing databases that contain structured information about clinical trials, in lieu of the usual approach of collecting expensive manual annotations. Practically, this means we will be able to exploit a very large `pseudo-annotated' dataset that is an order of magnitude bigger than what has been used in previous efforts, thus substantially improving model performance. Our extensive preliminary work demonstrates the promise and feasibility of this approach. Second, we propose novel machine learning models appropriate for the tasks of article categorization and data extraction for EBM. These models will specifically be designed to perform extraction of multiple, correlated data elements of interest while simultaneously classifying articles into clinically salient categories useful for EBM. We will rigorously evaluate the developed methods to assess their practical utility, specifically y comparing automated extraction accuracy to that achieved by trained systematic reviewers. And to make these methods useful to end-users (systematic reviewers), we will develop and evaluate open-source software and tools, including a web-based extraction tool that integrates our machine learning models to automatically extract information from uploaded articles (PDFs). We will conduct a user study to evaluate the utility and usability of this tool in practice.