A Unified Statistical Learning Model for Rankings and Scores with Application to Grant Panel Review
A Unified Statistical Learning Model for Rankings and Scores with Application to Grant Panel Review
复制标题
DOI：
--
复制
发表时间：
2022-01
期刊：
影响因子：
--
通讯作者：
Michael Pearce;E. Erosheva
中科院分区：
文献类型：
--
作者：
Michael Pearce;E. Erosheva
排名和分数是两种常见的数据类型，法官用来表达偏好和/或对一组对象质量的看法。存在许多模型来分别研究每种类型的数据，但是没有统一的统计模型在不首先执行数据转换的情况下同时捕获两种数据类型。我们提出了马洛斯-二项式模型来缩小这一差距，它结合了马洛斯' $\phi$排名模型与二项式得分模型，通过共享参数，量化对象的质量，共识排名，和法官之间的共识水平。我们提出了一个有效的树搜索算法来计算模型参数的精确MLE，通过分析和模拟研究模型的统计特性，并将我们的模型应用于真实的数据，从一个实例中的赠款小组审查，收集分数和部分排名。此外，我们还演示了如何使用模型输出来对对象进行置信度排名。所提出的模型被证明是明智的联合收割机信息从分数和排名量化对象的质量和测量的共识与适当的水平的统计不确定性。
Rankings and scores are two common data types used by judges to express preferences and/or perceptions of quality in a collection of objects. Numerous models exist to study data of each type separately, but no unified statistical model captures both data types simultaneously without first performing data conversion. We propose the Mallows-Binomial model to close this gap, which combines a Mallows' $\phi$ ranking model with Binomial score models through shared parameters that quantify object quality, a consensus ranking, and the level of consensus between judges. We propose an efficient tree-search algorithm to calculate the exact MLE of model parameters, study statistical properties of the model both analytically and through simulation, and apply our model to real data from an instance of grant panel review that collected both scores and partial rankings. Furthermore, we demonstrate how model outputs can be used to rank objects with confidence. The proposed model is shown to sensibly combine information from both scores and rankings to quantify object quality and measure consensus with appropriate levels of statistical uncertainty.