Overwritable automated japanese short-answer scoring and support system
Overwritable automated japanese short-answer scoring and support system
复制标题
可重写的自动日语简答评分和支持系统
DOI：
10.1145/3106426.3106513
复制
发表时间：
2017
期刊：
影响因子：
--
通讯作者：
Kameda Masayuki
中科院分区：
文献类型：
--
作者：
Ishioka Tsunenori;Kameda Masayuki
我们为新的国家中心笔试开发了一种日语简答自动评分和支持机器。我们的方法是基于这样一个事实，即准确识别文本蕴涵和/或同义词几年来几乎是不可能的。该系统根据评估标准或标准自动生成分数，并由人工评分员对其进行修改。该系统确定模型答案与实际书面答案之间的语义相似度以及一定程度的语义同一性和隐含。由于评分结果需要在多个层次上进行分类，我们使用随机森林来有效地利用许多预测器，而不是使用支持向量机。一个实验原型在Linux计算机上作为Web系统运行。我们比较了一个案例的人工分数和自动分数，其中3-6个分配点被放置在8个类别的社会研究测试中作为试探性考试。当不需要高度语义判断时，70%-90%的数据得分之间的差异在1分以内。
We have developed an automated Japanese short-answer scoring and support machine for new National Center written test exams. Our approach is based on the fact that accurate recognizing textual entailment and/or synonymy has been almost impossible for several years. The system generates automated scores on the basis of evaluation criteria or rubrics, and human raters revise them. The system determines semantic similarity between the model answers and the actual written answers as well as a certain degree of semantic identity and implication. Owing to the need for the scoring results to be classified at multiple levels, we use random forests to utilize many predictors effectively rather than use support vector machines. An experimental prototype operates as a web system on a Linux computer. We compared human scores with the automated scores for a case in which 3--6 allotment points were placed in 8 categories of a social studies test as a trial examination. The differences between the scores were within one point for 70--90 percent of the data when high semantic judgment was not needed.