Novel evaluation metrics for sparse spatio-temporal point process hotspot predictions - a crime case study
Novel evaluation metrics for sparse spatio-temporal point process hotspot predictions - a crime case study
复制标题
DOI：
10.1080/13658816.2016.1159684
复制
发表时间：
2016-01-01
影响因子：
5.7
通讯作者：
Cheng, Tao
中科院分区：
文献类型：
--
作者：
Adepeju, Monsuru;Rosser, Gabriel;Cheng, Tao
许多物理和社会学过程被表示为时间和空间上的离散事件。这些时空点过程通常是稀疏的，这意味着它们不能用传统的回归模型进行聚合和处理。基于点过程框架的模型可以用于预测目的。评估这些模型的预测性能提出了一个独特的挑战，因为同样的稀疏性阻止了使用流行的措施，如均方根误差。统计可能性是一个有效的替代方案，但这并不能衡量绝对的性能，因此从业者和研究人员很难解释。出于这种限制，我们开发了一个实用的工具包的时空点过程预测的评价指标。这些指标基于热点的概念，热点代表了高密度的区域。除了测量预测准确性外，我们的评估工具包还考虑了预测性能的更广泛方面，包括预测热点的空间和时间分布特征以及不同预测方法的互补性比较。我们展示了我们的评估指标的应用，使用犯罪预测的案例研究，比较四种不同的预测方法，使用犯罪数据从两个不同的位置和多种犯罪类型。结果突出了预测准确性和预测热点的时空分散之间的相互作用。新的评估框架可以应用于在各种场景中比较多种预测方法，从而对基于点过程的预测的预测性能产生有价值的新见解。
Many physical and sociological processes are represented as discrete events in time and space. These spatio-temporal point processes are often sparse, meaning that they cannot be aggregated and treated with conventional regression models. Models based on the point process framework may be employed instead for prediction purposes. Evaluating the predictive performance of these models poses a unique challenge, as the same sparseness prevents the use of popular measures such as the root mean squared error. Statistical likelihood is a valid alternative, but this does not measure absolute performance and is therefore difficult for practitioners and researchers to interpret. Motivated by this limitation, we develop a practical toolkit of evaluation metrics for spatio-temporal point process predictions. The metrics are based around the concept of hotspots, which represent areas of high point density. In addition to measuring predictive accuracy, our evaluation toolkit considers broader aspects of predictive performance, including a characterisation of the spatial and temporal distributions of predicted hotspots and a comparison of the complementarity of different prediction methods. We demonstrate the application of our evaluation metrics using a case study of crime prediction, comparing four varied prediction methods using crime data from two different locations and multiple crime types. The results highlight a previously unseen interplay between predictive accuracy and spatio-temporal dispersion of predicted hotspots. The new evaluation framework may be applied to compare multiple prediction methods in a variety of scenarios, yielding valuable new insight into the predictive performance of point process-based prediction.