The joint student response analysis and recognizing textual entailment challenge: making sense of student responses in educational applications
The joint student response analysis and recognizing textual entailment challenge: making sense of student responses in educational applications
复制标题
联合学生反应分析和识别文本蕴含挑战：理解教育应用中的学生反应
DOI：
--
复制
发表时间：
2015
影响因子：
2.7
通讯作者：
C. Leacock
中科院分区：
文献类型：
--
作者：
M. Dzikovska;Rodney D. Nielsen;C. Leacock
我们提出了联合学生反应分析（SRA）和第8次识别文本蕴涵挑战的结果。这项挑战的目标是将来自教育自然语言处理和计算语义学社区的研究人员聚集在一起。SRA任务的目标是评估学生对科学领域问题的回答，重点是回答内容的正确性和完整性。9个团队参加了这次挑战，总共提交了18次测试，使用的方法和特征改编自之前对自动简短答案评分的研究，识别文本蕴涵和语义文本相似性。我们对结果进行了扩展分析，重点关注评估指标、应用场景以及参与者使用的方法和特征的影响。我们的结论是，为了能够利用句法依赖特性和外部语义资源来完成这项任务，需要进行额外的研究，这可能是由于现有资源中科学领域的覆盖范围有限。然而，根据应用场景调整特征和模型的三种方法中的每一种都获得了更好的系统性能，值得研究团体进一步研究。
We present the results of the joint student response analysis (SRA) and 8th recognizing textual entailment challenge. The goal of this challenge was to bring together researchers from the educational natural language processing and computational semantics communities. The goal of the SRA task is to assess student responses to questions in the science domain, focusing on correctness and completeness of the response content. Nine teams took part in the challenge, submitting a total of 18 runs using methods and features adapted from previous research on automated short answer grading, recognizing textual entailment and semantic textual similarity. We provide an extended analysis of the results focusing on the impact of evaluation metrics, application scenarios and the methods and features used by the participants. We conclude that additional research is required to be able to leverage syntactic dependency features and external semantic resources for this task, possibly due to limited coverage of scientific domains in existing resources. However, each of three approaches to using features and models adjusted to application scenarios achieved better system performance, meriting further investigation by the research community.