Comparison Metrics for Large Scale Political Event Data Sets
Comparison Metrics for Large Scale Political Event Data Sets
复制标题
大规模政治事件数据集的比较指标
DOI：
--
复制
发表时间：
2015
期刊：
影响因子：
--
通讯作者：
Parus Analytics
中科院分区：
文献类型：
--
作者：
Philip A. Schrodt;Parus Analytics
本文解决了围绕使用全自动方法生成的政治事件数据来预测政治冲突的三个一般问题。我首先研究机器编码数据和人类编码数据的数据生成过程之间的差异，我认为当代努力的主要差异不在于编码的精度，而在于使用多个源的效果。虽然使用多个源在人类编码中几乎没有任何缺点，但它很有可能在自动编码中引入噪声。然后，我根据按二元事件频率加权的 CAMEO“五类”中每周事件计数之间的相关性，提出一个用于比较事件数据源的指标，并用两个示例进行说明： • 新 ICEWS 公共数据集与仅基于 BBC 世界广播摘要的未发布数据集的比较。 • 35 年KEDS 路透社和法新社黎凡特系列的TABARI 浅层解析器和PETRCH 完整解析器的比较。在 ICEWS/BBC 比较的情况下，该指标不仅在显示整体收敛方面有用（典型的加权相关性在 0.45 范围内，考虑到两个数据集之间的差异，该相关性高得惊人），而且显示了跨时间和地区的变化。就 TABARI/KEDS 而言，该指标显示了具有大量报告的系列的高度收敛性，并且还表明，PETRARCH 编码减少了重大冲突事件的数量（大概主要是通过消除误报）在大多数二元组中减少了大约 2 倍。在这两项测试中，该指标都擅长识别异常二元组，即 ICEWS 中的亚洲和 TABARI 编码黎凡特系列中的巴勒斯坦。本文最后列出了一系列问题的优先顺序，进一步的研究和开发可能会取得成效。
This paper addresses three general issues surrounding the use of political event data generated by fully automated methods in forecasting political conflict. I first look at the differences between the data generation process for machine and human coded data, where I believe the major difference in contemporary efforts is found not in the precision of the coding, but rather the effects of using multiple sources. While the use of multiple sources has virtually no downside in human coding, it has great potential to introduce noise in automated coding. I then propose a metric for comparing event data sources based on the correlations between weekly event counts in the CAMEO “pentaclasses” weighted by the frequency of dyadic events, and illustrate this with two examples: • A comparison of the new ICEWS public data set with an unpublished data set based only on the BBC Summary of World Broadcasts. • A comparison of the TABARI shallow parser and PETRARCH full parser for the 35-year KEDS Reuters and Agence France Presse Levant series. In the case of the ICEWS/BBC comparison, the metric appears useful not only in showing the overall convergence—typical weighted correlations are in the range of 0.45, surprisingly high given the differences between the two data sets—and showing variations across time and regions. In the case of TABARI/KEDS, the metric shows high convergence for the series with a large number of reports, and also shows that the PETRARCH coding reduces the number of material conflict events—presumably mostly by eliminating false positives—by around a factor of 2 in most dyads. In both tests, the metric is good at identifying anomalous dyads, Asia in the case of ICEWS and Palestine in the case of the TABARI-coded Levant series. The paper concludes with a prioritized list of issues where further research and development is likely to prove productive.