SWIFT-Review: a text-mining workbench for systematic review.
SWIFT-Review: a text-mining workbench for systematic review.
复制标题
DOI：
10.1186/s13643-016-0263-z
复制
发表时间：
2016-05-23
影响因子：
3.7
通讯作者：
Thayer K
中科院分区：
文献类型：
--
作者：
Howard BE;Phillips J;Miller K;Tandon A;Mav D;Shah MR;Holmgren S;Pelch KE;Walker V;Rooney AA;Macleod M;Shah RR;Thayer K
人们越来越有兴趣使用机器学习方法来对研究进行优先排序，并在进行系统评价时减少筛选文献的人类负担。此外，在系统审查的问题形成阶段确定可解决的问题可能是具有挑战性的，特别是对于拥有大量文献基础的主题。在这里，我们评估SWIFT-Review优先级排序算法的性能，以确定与给定研究问题相关的研究。我们还探索了在问题形成过程中使用SWIFT-Review来识别、分类和可视化大型文献语料库中数据丰富/数据贫乏的研究领域。使用了20个案例研究，包括15个公共数据集，代表了一系列的复杂性和大小，以评估SWIFT-Review的优先排序性能。对于每项研究，人工注释的包含和排除的标题和摘要的种子集被用于机器训练。然后，使用考虑词频和潜在Dirichlet分配(LDA)主题建模的算法对其余参考文献进行相关性排名。这一排名是根据(1)为确定95%的已知相关研究而筛选的研究数量和(2)“过度抽样节省的工作量”(WSS)绩效指标进行评估的。为了评估SWIFT-Review在问题制定中的使用情况，PubMed对171种被认为是EDC的化学物质的文献搜索结果被上传到SWIFT-Review(264,588项研究)，并根据证据流和健康结果进行分类。调查了搜索结果的模式，并使用各种互动图形将其可视化。与使用相同数据集的其他工具的报告表现相比，SWIFT-Review排名程序在15个公共数据集中的11个上获得了最高分数。总体而言，这些结果表明，使用机器学习对文档进行分类以进行筛选，平均有可能节省使用无序文档列表时通常需要的筛选工作的50%以上。此外，SWIFT-Review的标记和注释功能在确定范围和制定问题的活动中也很有用。文本挖掘和机器学习软件，如Swift-Review，可以成为减少人工筛选负担和帮助解决问题的宝贵工具。本文的在线版本(doi：10.1186/s13643-0160263-z)包含补充材料，授权用户可以使用。
There is growing interest in using machine learning approaches to priority rank studies and reduce human burden in screening literature when conducting systematic reviews. In addition, identifying addressable questions during the problem formulation phase of systematic review can be challenging, especially for topics having a large literature base. Here, we assess the performance of the SWIFT-Review priority ranking algorithm for identifying studies relevant to a given research question. We also explore the use of SWIFT-Review during problem formulation to identify, categorize, and visualize research areas that are data rich/data poor within a large literature corpus. Twenty case studies, including 15 public data sets, representing a range of complexity and size, were used to assess the priority ranking performance of SWIFT-Review. For each study, seed sets of manually annotated included and excluded titles and abstracts were used for machine training. The remaining references were then ranked for relevance using an algorithm that considers term frequency and latent Dirichlet allocation (LDA) topic modeling. This ranking was evaluated with respect to (1) the number of studies screened in order to identify 95 % of known relevant studies and (2) the “Work Saved over Sampling” (WSS) performance metric. To assess SWIFT-Review for use in problem formulation, PubMed literature search results for 171 chemicals implicated as EDCs were uploaded into SWIFT-Review (264,588 studies) and categorized based on evidence stream and health outcome. Patterns of search results were surveyed and visualized using a variety of interactive graphics. Compared with the reported performance of other tools using the same datasets, the SWIFT-Review ranking procedure obtained the highest scores on 11 out of 15 of the public datasets. Overall, these results suggest that using machine learning to triage documents for screening has the potential to save, on average, more than 50 % of the screening effort ordinarily required when using un-ordered document lists. In addition, the tagging and annotation capabilities of SWIFT-Review can be useful during the activities of scoping and problem formulation. Text-mining and machine learning software such as SWIFT-Review can be valuable tools to reduce the human screening burden and assist in problem formulation. The online version of this article (doi:10.1186/s13643-016-0263-z) contains supplementary material, which is available to authorized users.