SWIFT-Active Screener: Accelerated document screening through active learning and integrated recall estimation.
SWIFT-Active Screener: Accelerated document screening through active learning and integrated recall estimation.
复制标题
DOI：
10.1016/j.envint.2020.105623
复制
发表时间：
2020-05
影响因子：
11.8
通讯作者：
Shah RR
中科院分区：
文献类型：
--
作者：
Howard BE;Phillips J;Tandon A;Maharana A;Elmore R;Mav D;Sedykh A;Thayer K;Merrick BA;Walker V;Rooney A;Shah RR
在系统综述的筛选阶段，研究人员使用详细的纳入/排除标准来决定一组候选文章中的每一篇文章是否与正在考虑的研究问题相关。一次典型的审查可能需要筛选数千或数万篇文章，并可能利用数百个人小时的劳动力。在这里，我们介绍SWIFT-Active Screener，这是一款基于Web的协作式系统审查软件应用程序，旨在减少审查流程这一资源密集型阶段所需的总体筛选负担。为了确定审查文章的优先顺序，SWIFT-Active Screener使用主动学习，这是一种机器学习，在筛选过程中结合了用户反馈。同时，使用负二项模型估计未筛选文档列表中剩余的相关文章数量。使用一项包含26个不同的系统回顾数据集的模拟，这些数据集之前经过评价者的筛选，我们评估了文档优先排序和召回估计方法。平均而言，95%的相关文章是在只筛选了总参考文献清单的40%之后确定的。在包含5,000条或更多参考文献的5个文档集中，平均只筛选了34%的可用参考文献即可实现95%的召回。此外，我们提议的召回估计器对筛选过程中确定的相关文件的百分比提供了有用的、保守的估计。与传统筛查相比，Swift-Active筛选器可以显著节省时间，而且对于较大的项目规模，节省的时间会更多。此外，在筛选过程中集成显式召回估计解决了所有机器学习系统在文档筛选中面临的一个重要挑战：何时停止筛选优先参考列表。该软件目前以多用户、协作、在线网络应用程序的形式提供。
In the screening phase of systematic review, researchers use detailed inclusion/exclusion criteria to decide whether each article in a set of candidate articles is relevant to the research question under consideration. A typical review may require screening thousands or tens of thousands of articles in and can utilize hundreds of person-hours of labor. Here we introduce SWIFT-Active Screener, a web-based, collaborative systematic review software application, designed to reduce the overall screening burden required during this resource-intensive phase of the review process. To prioritize articles for review, SWIFT-Active Screener uses active learning, a type of machine learning that incorporates user feedback during screening. Meanwhile, a negative binomial model is employed to estimate the number of relevant articles remaining in the unscreened document list. Using a simulation involving 26 diverse systematic review datasets that were previously screened by reviewers, we evaluated both the document prioritization and recall estimation methods. On average, 95% of the relevant articles were identified after screening only 40% of the total reference list. In the 5 document sets with 5,000 or more references, 95% recall was achieved after screening only 34% of the available references, on average. Furthermore, the recall estimator we have proposed provides a useful, conservative estimate of the percentage of relevant documents identified during the screening process. SWIFT-Active Screener can result in significant time savings compared to traditional screening and the savings are increased for larger project sizes. Moreover, the integration of explicit recall estimation during screening solves an important challenge faced by all machine learning systems for document screening: when to stop screening a prioritized reference list. The software is currently available in the form of a multi-user, collaborative, online web application.
登录
查看更多内容
影响因子：
3
作者：
Wallace BC;Trikalinos TA;Lau J;Brodley C;Schmid CH
通讯作者：
Schmid CH
影响因子：
8.8
作者：
Wallace, Byron C.;Small, Kevin;Trikalinos, Thomas A.
通讯作者：
Trikalinos, Thomas A.
DOI：
10.1093/jamia/ocv044
发表时间：
2016-01-01
影响因子：
6.4
作者：
Marshall, Iain J.;Kuiper, Joel;Wallace, Byron C.
通讯作者：
Wallace, Byron C.
DOI：
10.1186/1748-5908-5-56
发表时间：
2010-07-19
期刊：
Implementation science : IS
影响因子：
--
作者：
Ganann R;Ciliska D;Thomas H
通讯作者：
Thomas H
GapsCreener：使用支持向量机技术在PubMed筛选人类遗传关联文献的自动工具。
影响因子：
3
作者：
Yu W;Clyne M;Dolan SM;Yesupriya A;Wulf A;Liu T;Khoury MJ;Gwinn M
通讯作者：
Gwinn M