Integration of Prediction Scores From Various Automated Essay Scoring Models Using Item Response Theory
Integration of Prediction Scores From Various Automated Essay Scoring Models Using Item Response Theory
复制标题
DOI：
10.1109/tlt.2023.3253215
复制
发表时间：
2023-12-01
影响因子：
3.7
通讯作者：
Ueno，Maomi
中科院分区：
文献类型：
--
作者：
Uto，Masaki;Aomi，Itsuki;Ueno，Maomi
在自动作文评分（AES）中，作文会自动评分，而无需人工评分。在过去的几十年里，已经提出了许多基于各种手动设计的特征或各种深度神经网络（DNN）架构的AES模型。每个AES模型都有其独特的优点和特点。因此，与其使用单一AES模型，还不如适当整合来自各种AES模型的预测，以实现更高的评分准确度。在这篇文章中，我们提出了一种方法，使用项目反应理论整合预测分数从各种AES模型，同时考虑到模型之间的评分行为的特点的差异。结果表明，该方法比单独的AES模型和传统的积分方法具有更高的精度。此外，所提出的方法有助于解释每个AES模型的评分特性和评分集成机制。
In automated essay scoring (AES), essays are automatically graded without human raters. Many AES models based on various manually designed features or various architectures of deep neural networks (DNNs) have been proposed over the past few decades. Each AES model has unique advantages and characteristics. Therefore, rather than using a single-AES model, appropriate integration of predictions from various AES models is expected to achieve higher scoring accuracy. In this article, we propose a method that uses item response theory to integrate prediction scores from various AES models while taking into account differences in the characteristics of scoring behavior among models. It is found that the proposed method achieves higher accuracy than that of individual AES models and conventional score-integration methods. Furthermore, the proposed method facilitates interpreting each AES model's scoring characteristics and score-integration mechanism.