Recalibrating probabilistic forecasts of epidemics.
Recalibrating probabilistic forecasts of epidemics.
复制标题
DOI：
10.1371/journal.pcbi.1010771
复制
发表时间：
2022-12
影响因子：
4.3
通讯作者：
中科院分区：
文献类型：
--
作者：
分布预测对于各种各样的应用都很重要，包括预测流行病。通常，预测是错误的，或者在为未来事件分配不确定性时不可靠。我们提出了一种重新校准的方法，可以应用于黑箱预测回顾性的预测和观察，以及扩展，使这种方法更有效地重新校准流行病预测。该方法保证在样本内训练和测量时提高校准和日志评分性能。我们还证明了一个重新校准的预报员的预期日志分数的增加等于PIT分布的熵。我们将这种重新校准方法应用于FluSight网络中的27个流感预报员，并表明重新校准可靠地提高了预测精度和校准。这种方法在Github上可用，有效，健壮，易于用作后处理工具来改进流行病预测。传染病的流行每年在全世界造成数百万人死亡，可靠的流行病预测可以使公共卫生官员做出反应，减轻流行病的影响。然而，由于流行病预测是一项艰巨的任务，许多流行病预测没有校准。校准是任何预测所需的属性，我们提供了一种重新校准预测的后处理方法。我们证明了这种方法的有效性，在提高准确性和校准各种各样的流感预测。我们还显示了校准和预报员的预期得分之间的定量关系。我们的重新校准方法是一种工具，任何预测者都可以使用，无论模型选择如何，都可以提高预测的准确性和可靠性。这项工作提供了一个桥梁之间的预测理论，很少涉及的应用领域是新的或有很少的数据，和一些最近的应用流行病预测，预测校准很少进行系统的分析。
Distributional forecasts are important for a wide variety of applications, including forecasting epidemics. Often, forecasts are miscalibrated, or unreliable in assigning uncertainty to future events. We present a recalibration method that can be applied to a black-box forecaster given retrospective forecasts and observations, as well as an extension to make this method more effective in recalibrating epidemic forecasts. This method is guaranteed to improve calibration and log score performance when trained and measured in-sample. We also prove that the increase in expected log score of a recalibrated forecaster is equal to the entropy of the PIT distribution. We apply this recalibration method to the 27 influenza forecasters in the FluSight Network and show that recalibration reliably improves forecast accuracy and calibration. This method, available on Github, is effective, robust, and easy to use as a post-processing tool to improve epidemic forecasts. Epidemics of infectious disease cause millions of deaths worldwide each year, and reliable epidemic forecasts can allow public health officials to respond to mitigate the effects of epidemics. However, because epidemic forecasting is a difficult task, many epidemic forecasts are not calibrated. Calibration is a desired property of any forecast, and we provide a post-processing method that recalibrates forecasts. We demonstrate the effectiveness of this method in improving accuracy and calibration on a wide variety of influenza forecasters. We also show a quantitative relationship between calibration and a forecaster’s expected score. Our recalibration method is a tool that any forecaster can use, regardless of model choice, to improve forecast accuracy and reliability. This work provides a bridge between forecasting theory, which rarely deals with applications in domains that are new or have little data, and some recent applications of epidemic forecasting, where forecast calibration is rarely analyzed systematically.
登录
查看更多内容
影响因子：
1.1
作者：
Gneiting, Tilmann;Ranjan, R.
通讯作者：
Ranjan, R.
影响因子：
3.2
作者：
Wilks, Daniel S.;Hamill, Thomas M.
通讯作者：
Hamill, Thomas M.
DOI：
10.2307/2981683
发表时间：
1984-01-01
影响因子：
2
作者：
DAWID, AP
通讯作者：
DAWID, AP
影响因子：
3.2
作者：
Gneiting, T;Raftery, AE;Goldman, T
通讯作者：
Goldman, T
DOI：
10.1111/j.1467-9868.2007.00587.x
发表时间：
2007-01-01
影响因子：
5.8
作者：
Gneiting, Tilmann;Balabdaoui, Fadoua;Raftery, Adrian E.
通讯作者：
Raftery, Adrian E.