Individual model forecasts can be misleading, but together they are useful
Individual model forecasts can be misleading, but together they are useful
复制标题
DOI：
10.1007/s10654-020-00667-8
复制
发表时间：
2020-08-11
影响因子：
13.6
通讯作者：
Johansson, Michael A.
中科院分区：
文献类型：
--
作者：
Buckee, Caroline O.;Johansson, Michael A.
迄今为止，媒体和政府广泛使用模型预测来应对 COVID-19 是该流行病的一个突出且有争议的特征。在本期中，Chin 等人。比较了美国疫情爆发初期的四个备受瞩目的模型的准确性，这些模型旨在对纽约的死亡人数和重症监护病房 (ICU) 床位利用率进行定量预测 [1]。他们发现，所有四种模型虽然方法不同，但不仅未能准确预测死亡人数和 ICU 使用率，而且未能正确描述不确定性，特别是在疫情的关键早期阶段。虽然克服这些方法上的挑战是关键，但 Chin 等人。还呼吁系统性进步，包括提高数据质量、在政策使用前实时评估预测以及开发多模型方法。作者揭示了“真实情况”数据的巨大差异；用于建立和评估预测模型的流行病学监测数据。再加上 SARS-COV-2 基本流行病学参数的不确定性以及模型框架的局限性，此类模型有可能产生不准确的预测也就不足为奇了。提高数据质量当然有助于改进模型预测，但在生成关键数据的监视系统是新的、不完善的且快速变化的时刻，通常需要进行预测。这些不确定性需要整合到预测本身中。此外，与模型相关的额外不确定性来源——参数不确定性和结构不确定性——也需要注意，并且通常只进行表面处理。总而言之，这些挑战可能会导致一些人首先质疑预测在政策制定中的使用。
The broad use by media and governments of model forecasts to inform the COVID-19 response has been a prominent and controversial feature of the pandemic so far. In this issue, Chin et al. compare the accuracy of four high profile models that, early during the outbreak in the US, aimed to make quantitative predictions about deaths and Intensive Care Unit (ICU) bed utilization in New York [1]. They find that all four models, though different in approach, failed not only to accurately predict the number of deaths and ICU utilization but also to describe uncertainty appropriately, particularly during the critical early phase of the epidemic. While overcoming these methodological challenges is key, Chin et al. also call for systemic advances including improving data quality, evaluating forecasts in real-time before policy use, and developing multi-model approaches. The authors reveal substantial variability in “ground truth” data; epidemiological surveillance data used for both building and evaluating forecasting models. Coupled with uncertainty about basic epidemiological parameters of SARS-COV-2 as well as limitations in model frameworks, it is not surprising that such models have the potential to generate inaccurate forecasts. Improved data quality can certainly help improve model predictions, but forecasts are often needed in moments where surveillance systems that generate key data are new, imperfect, and rapidly changing.These uncertainties need to be integrated into the forecast itself. Moreover, the additional sources of uncertainty associated with the model—parameter uncertainty and structural uncertainty—also need attention, and are often dealt with superficially. Taken together, these challenges may lead some to question the use of forecasts for policy making in the first place.