Text Mining Pipeline to Accelerate Systematic Reviews in Evidence-Based Medicine

我们假设一套灵活的、可配置的自动化信息学工具可以显著减少 在保持甚至改进其质量的同时产生系统审查所需的努力。为了测试这一点 假设，我们提出： 目标1.扩展我们对RCT自动标记的研究，以包括其他研究类型和 提供公共资源。 A)将创建机器学习模型，自动将概率估计分配给三种类型的 被系统评价者广泛检查的观察性研究。 B)RCT和其他标记器将针对新发表的PubMed文章进行前瞻性评估。 C)所有PubMed文章将自动标记为随机对照试验、队列研究、病例对照研究和横断面研究 并在链接到公共查询接口的公共数据集中进行注释。用户还将收到文章的标签 按需从非PubMed数据源。 目标2.评估我们的工具在被系统评价者使用时的性能和可用性 在田间条件下。 A)这些工具将进行定制和整合，以便利实地评价。 B)三阶段评估：1.对Metta和RCT标签者的绩效进行回溯性评估。2.实时 “影子”。3.前瞻性对照研究。 目的3.确定在发表的系统性综述后出现的其他临床试验文章 已完成，这与审查主题相关。 目的4.确定与特定ClinicalTrials.gov注册试验相关的出版物。 目标5.开发和评估新的机器学习方法和工具，以促进快速 新的系统性审查主题的证据范围。 A)将制定方法，根据文章与拟议的新系统的相关性对文章进行排名 回顾主题。 B)将创建一个范围确定工具，显示按预测相关性排名的文章，并标有研究设计 属性、样本大小和偏倚估计的Cochrane风险。 拟议的研究将推进撰写系统审查过程中早期步骤的自动化， 从而加强循证医学和将最佳做法纳入临床护理。

We hypothesize that a flexible, configurable suite of automated informatics tools can reduce significantly the effort needed to generate systematic reviews while maintaining or even improving their quality. To test this hypothesis, we propose: Aim 1. To extend our research on automated RCT tagging to include additional study types and provide public resources. A) Machine learning models will be created that automatically assign probability estimates to three types of observational studies that are widely examined by systematic reviewers. B) The RCT and other taggers will be evaluated prospectively for newly published PubMed articles. C) All PubMed articles will be automatically tagged for RCT, cohort, case-control and cross-sectional studies and annotated in a public dataset linked to a public query interface. Users will also receive tags for articles from non-PubMed data sources on demand. Aim 2. To evaluate the performance and usability of our tools when used by systematic reviewers under field conditions. A) The tools will be customized and integrated to facilitate field evaluation. B) A three-stage evaluation: 1. Retrospective evaluation of Metta and RCT Tagger performance. 2. Real-time “shadowing”. 3. Prospective controlled study. Aim 3. To identify additional clinical trial articles, appearing after a published systematic review was completed, that are relevant to the review topic. Aim 4. To identify publications related to specific ClinicalTrials.gov registered trials. Aim 5. To develop and evaluate new machine learning methods and tools that will facilitate rapid evidence scoping for new systematic review topics. A) Methods will be developed for ranking articles with respect to their relevance to a proposed new systematic review topic. B) A scoping tool will be created that displays articles ranked by predicted relevance, tagged with study design attributes, sample sizes, and Cochrane risk of bias estimates. The proposed studies will advance the automation of early steps in the process of writing systematic reviews, and thereby enhance evidence-based medicine and the incorporation of best practices into clinical care.