On the selection of forecasting accuracy measures
On the selection of forecasting accuracy measures
复制标题
DOI：
10.1080/01605682.2021.1892464
复制
发表时间：
2021-02-20
影响因子：
3.6
通讯作者：
Assimakopoulos, Vassilios
中科院分区：
文献类型：
--
作者：
Koutsandreas, Diamantis;Spiliotis, Evangelos;Assimakopoulos, Vassilios
围绕着选择最合适的误差度量来评估预测方法的性能，存在着很多争议。虽然统计人员主张使用具有良好统计特性的计量，但从业人员更喜欢易于沟通和理解的计量。此外，研究人员认为，参数化模型的损失函数应该与如何进行事后绩效测量相一致。在本文中，我们要问：这重要吗？如果我们选择一种方法而不是另一种方法，预测方法的相对排名是否会发生重大变化？样本内损失函数和样本外性能度量的不匹配是否会降低预测模型的性能？针对平均排名点预测精度，我们回顾了学术界和实践中最常用的措施，并进行了大规模的实证研究，以了解措施之间的选择的重要性。我们的研究结果表明，不同的误差测量之间只有很小的差异，特别是在每个测量类别（百分比，相对，或缩放）。
A lot of controversy exists around the choice of the most appropriate error measure for assessing the performance of forecasting methods. While statisticians argue for the use of measures with good statistical properties, practitioners prefer measures that are easy to communicate and understand. Moreover, researchers argue that the loss-function for parameterizing a model should be aligned with how the post-performance measurement is made. In this paper we ask: Does it matter? Will the relative ranking of the forecasting methods change significantly if we choose one measure over another? Will a mismatch of the in-sample loss-function and the out-of-sample performance measure decrease the performance of the forecasting models? Focusing on the average ranked point forecast accuracy, we review the most commonly-used measures in both the academia and practice and perform a large-scale empirical study to understand the importance of the choice between measures. Our results suggest that there are only small discrepancies between the different error measures, especially within each measure category (percentage, relative, or scaled).