Challenges in the Automatic Analysis of Students' Diagnostic Reasoning
Challenges in the Automatic Analysis of Students' Diagnostic Reasoning
复制标题
自动分析学生诊断推理的挑战
DOI：
10.1609/aaai.v33i01.33016974
复制
发表时间：
2018
期刊：
影响因子：
--
通讯作者：
Iryna Gurevych
中科院分区：
文献类型：
--
作者：
Claudia Schulz;Christian M. Meyer;Michael Sailer;J. Kiesewetter;Elisabeth Bauer;F. Fischer;M. Fischer;Iryna Gurevych
诊断推理是许多职业的关键组成部分。为了提高学生的诊断推理能力，教育心理学家分析并反馈这些学生在诊断时使用的认知活动，特别是假设生成，证据生成，证据评估和得出结论。然而，这种手动分析非常耗时。我们的目标是通过自动化的认知活动识别，使诊断推理分析和反馈的大规模采用。我们创建了第一个语料库，包括诊断推理自我解释的学生从两个域与认知活动注释。基于语料库创建和任务的特点，我们讨论了使用人工智能方法自动识别认知活动的三个挑战：正确识别认知活动跨度，可靠区分相似的认知活动，以及检测重叠的认知活动。我们提出了一个单独的性能指标为每个挑战，从而为未来的研究提供了一个评估框架。事实上，我们对各种最先进的递归神经网络架构的评估表明，目前的技术无法解决其中的一些挑战。
Diagnostic reasoning is a key component of many professions. To improve students’ diagnostic reasoning skills, educational psychologists analyse and give feedback on epistemic activities used by these students while diagnosing, in particular, hypothesis generation, evidence generation, evidence evaluation, and drawing conclusions. However, this manual analysis is highly time-consuming. We aim to enable the large-scale adoption of diagnostic reasoning analysis and feedback by automating the epistemic activity identification. We create the first corpus for this task, comprising diagnostic reasoning selfexplanations of students from two domains annotated with epistemic activities. Based on insights from the corpus creation and the task’s characteristics, we discuss three challenges for the automatic identification of epistemic activities using AI methods: the correct identification of epistemic activity spans, the reliable distinction of similar epistemic activities, and the detection of overlapping epistemic activities. We propose a separate performance metric for each challenge and thus provide an evaluation framework for future research. Indeed, our evaluation of various state-of-the-art recurrent neural network architectures reveals that current techniques fail to address some of these challenges.
影响因子：
4.5
作者：
Demner-Fushman D;Chapman WW;McDonald CJ
通讯作者：
McDonald CJ