A human judgment approach to epidemiological forecasting.
A human judgment approach to epidemiological forecasting.
复制标题
DOI：
10.1371/journal.pcbi.1005248
复制
发表时间：
2017-03
影响因子：
4.3
通讯作者：
Rosenfeld R
中科院分区：
文献类型：
--
作者：
Farrow DC;Brooks LC;Hyun S;Tibshirani RJ;Burke DS;Rosenfeld R
尽管在过去的世纪中技术和医学取得了重大进步，但传染病给社会带来了相当大的负担。提前预警有助于缓解即将发生或正在发生的流行病并为之做好准备。从历史上看，由于许多原因，这种能力一直滞后，特别是包括系统目前状况的不确定性以及对推动流行病轨迹的过程的理解。目前，我们有机会获得数据，模型和计算资源，使流行病学预测系统的发展。事实上，美国政府最近发起的几项挑战为这些技术的发展营造了一个开放和协作的环境。这些挑战的主要焦点是发展流行病学预测的统计和计算方法，但在这里，我们考虑一个基于集体人类判断的严肃的替代方案。我们创建了基于网络的“Epicast”预测系统，该系统收集并汇总了人类参与者实时做出的流行病预测，并通过这些预测提出了两个问题：人类判断的准确性如何，以及这些预测与他们更多的计算，数据驱动的替代品相比如何？为了解决前者，我们通过各种指标评估人类能够准确预测流感和基孔肯雅热的轨迹。对于后者，我们表明，对2014 - 2015年和2015 - 2016年美国流感季节的实时综合人类预测通常比几个统计系统做出的相同预测更准确，特别是对于短期目标。我们的结论是，有价值的预测能力，在集体人类的判断，我们讨论了这种方法的优点和缺点。尽管有先进和广泛的医疗保健，在美国每年有大量的死亡可归因于流感等传染病。如果有足够的预警，许多这类案件是很容易预防的。这是流行病学预测的主要目标，这是一个相对较新的领域，试图预测疾病暴发的时间和地点。为了响应对这一努力的日益增长的兴趣，已经为各种疾病开发了许多预测框架。我们要问的是，基于人类集体判断的方法是否可以用来产生合理的预测，以及这种预测与纯粹数据驱动系统产生的预测相比如何。为了回答这个问题，我们在2014 - 2015年和2015 - 2016年美国流感季节以及2014 - 2015年基孔肯雅病毒入侵中美洲期间，从一组专家和非专家志愿者那里实时收集了简单的预测，我们报告了基于这些预测的几种准确性措施。通过将这些预测与已发表的数据驱动方法的预测进行比较，我们对任务的难度建立了一种直觉，并了解到集体人类判断中存在真实的价值。
Infectious diseases impose considerable burden on society, despite significant advances in technology and medicine over the past century. Advanced warning can be helpful in mitigating and preparing for an impending or ongoing epidemic. Historically, such a capability has lagged for many reasons, including in particular the uncertainty in the current state of the system and in the understanding of the processes that drive epidemic trajectories. Presently we have access to data, models, and computational resources that enable the development of epidemiological forecasting systems. Indeed, several recent challenges hosted by the U.S. government have fostered an open and collaborative environment for the development of these technologies. The primary focus of these challenges has been to develop statistical and computational methods for epidemiological forecasting, but here we consider a serious alternative based on collective human judgment. We created the web-based “Epicast” forecasting system which collects and aggregates epidemic predictions made in real-time by human participants, and with these forecasts we ask two questions: how accurate is human judgment, and how do these forecasts compare to their more computational, data-driven alternatives? To address the former, we assess by a variety of metrics how accurately humans are able to predict influenza and chikungunya trajectories. As for the latter, we show that real-time, combined human predictions of the 2014–2015 and 2015–2016 U.S. flu seasons are often more accurate than the same predictions made by several statistical systems, especially for short-term targets. We conclude that there is valuable predictive power in collective human judgment, and we discuss the benefits and drawbacks of this approach. Despite advanced and widely accessible health care, a large number of annual deaths in the United States are attributable to infectious diseases like influenza. Many of these cases could be easily prevented if sufficiently advanced warning was available. This is the main goal of epidemiological forecasting, a relatively new field that attempts to predict when and where disease outbreaks will occur. In response to growing interest in this endeavor, many forecasting frameworks have been developed for a variety of diseases. We ask whether an approach based on collective human judgment can be used to produce reasonable forecasts and how such forecasts compare with forecasts produced by purely data-driven systems. To answer this, we collected simple predictions in real-time from a set of expert and non-expert volunteers during the 2014–2015 and 2015–2016 U.S. flu seasons and during the 2014–2015 chikungunya invasion of Central America, and we report several measures of accuracy based on these predictions. By comparing these predictions with published forecasts of data-driven methods, we build an intuition for the difficulty of the task and learn that there is real value in collective human judgment.