Analyzing ordinal data with metric models: What could possibly go wrong?
Analyzing ordinal data with metric models: What could possibly go wrong?
复制标题
DOI：
10.1016/j.jesp.2018.08.009
复制
发表时间：
2018-11-01
影响因子：
3.5
通讯作者：
Kruschke, John K.
中科院分区：
文献类型：
--
作者：
Liddell, Torrin M.;Kruschke, John K.
我们调查了《人格与社会心理学杂志》（JPSP）、《心理科学》（PS）和《实验心理学杂志：综合》（JEP：G）中提到“李克特”一词的所有文章，发现100%分析有序数据的文章都使用了度量模型。我们提出了新的证据，分析有序的数据，如果他们是度量可以系统地导致错误。我们展示了假警报（即，检测不存在的效应，I型错误）和未能检测到效应（即，第二类错误（Type II Error）。我们证明了系统的反转效应，其中治疗有序数据作为度量表示相反的顺序的手段比真正的顺序的手段。对于析因设计中的交互作用和回归中的趋势分析，我们也会遇到同样的问题-误报、遗漏和倒置。我们表明，平均在多个有序测量不解决，甚至改善这些问题。一个核心的贡献是一个图形解释如何以及何时发生的虚假陈述。此外，我们指出，没有万无一失的方法来检测这些问题，通过处理的顺序值作为度量，而是我们提倡使用有序概率模型（或类似），因为它们会更好地描述数据。最后，虽然频率论的方法，一些有序概率模型是可用的，我们使用贝叶斯方法，因为它们的灵活性，在指定模型和丰富性和准确性，在提供参数估计。提供了一个R脚本，用于运行比较有序概率单位模型和度量模型的分析。
We surveyed all articles in the Journal of Personality and Social Psychology (JPSP), Psychological Science (PS), and the Journal of Experimental Psychology: General (JEP:G) that mentioned the term "Likert," and found that 100% of the articles that analyzed ordinal data did so using a metric model. We present novel evidence that analyzing ordinal data as if they were metric can systematically lead to errors. We demonstrate false alarms (i.e., detecting an effect where none exists, Type I errors) and failures to detect effects (i.e., loss of power, Type II errors). We demonstrate systematic inversions of effects, for which treating ordinal data as metric indicates the opposite ordering of means than the true ordering of means. We show the same problems - false alarms, misses, and inversions - for interactions in factorial designs and for trend analyses in regression. We demonstrate that averaging across multiple ordinal measurements does not solve or even ameliorate these problems. A central contribution is a graphical explanation of how and when the misrepresentations occur. Moreover, we point out that there is no sure-fire way to detect these problems by treating the ordinal values as metric, and instead we advocate use of ordered-probit models (or similar) because they will better describe the data. Finally, although frequentist approaches to some ordered-probit models are available, we use Bayesian methods because of their flexibility in specifying models and their richness and accuracy in providing parameter estimates. An R script is provided for running an analysis that compares ordered-probit and metric models.