When Correlation Is Not Enough: Validating Populism Scores from Supervised Machine-Learning Models
When Correlation Is Not Enough: Validating Populism Scores from Supervised Machine-Learning Models
复制标题
当相关性不够时：验证监督机器学习模型的民粹主义得分
DOI：
10.1017/pan.2022.32
复制
发表时间：
2022
影响因子：
5.4
通讯作者：
Robert A. Huber
中科院分区：
文献类型：
--
作者：
M. Jankowski;Robert A. Huber
尽管民粹主义政党在世界许多地方取得了持续的成功，但我们缺乏有关政党民粹主义水平的全面信息。Di Cocco和Monechi（DCM）最近对政治分析的贡献表明，可以通过使用监督机器学习预测政党选举宣言中的民粹主义得分来弥补这一研究差距。在本文中，我们提供了一个详细的讨论所建议的方法。基于最近关于机器学习模型验证的争论，我们认为DCM论文中提供的有效性检查是不够的。我们进行了一系列额外的有效性检查，并实证表明，该方法不适合从文本中获得民粹主义分数。我们的结论是，随着时间的推移和国家之间的民粹主义测量仍然是一个巨大的挑战，实证研究。更一般地说，我们的论文说明了对监督机器学习模型进行更全面验证的重要性。
Abstract Despite the ongoing success of populist parties in many parts of the world, we lack comprehensive information about parties’ level of populism over time. A recent contribution to Political Analysis by Di Cocco and Monechi (DCM) suggests that this research gap can be closed by predicting parties’ populism scores from their election manifestos using supervised machine learning. In this paper, we provide a detailed discussion of the suggested approach. Building on recent debates about the validation of machine-learning models, we argue that the validity checks provided in DCM’s paper are insufficient. We conduct a series of additional validity checks and empirically demonstrate that the approach is not suitable for deriving populism scores from texts. We conclude that measuring populism over time and between countries remains an immense challenge for empirical research. More generally, our paper illustrates the importance of more comprehensive validations of supervised machine-learning models.