On the Automatic Assessment of Computational Thinking Skills: A Comparison with Human Experts
On the Automatic Assessment of Computational Thinking Skills: A Comparison with Human Experts
复制标题
计算思维能力的自动评估：与人类专家的比较
DOI：
10.1145/3027063.3053216
复制
发表时间：
2017
期刊：
影响因子：
--
通讯作者：
G. Robles
中科院分区：
文献类型：
--
作者：
J. Moreno;Marcos Román;C. Harteveld;G. Robles
编程和计算思维技能在世界各地的学校得到推广。然而，仍然缺乏帮助学习者和教育工作者评估这些技能的工具。我们已经实现了一个名为Dr.Scratch的评估工具，它分析Scratch项目，目的是评估计算思维的几个方面的发展水平。要显示其有效性，需要解决的问题之一是将该工具提供的(自动)评价与(人工)专家的(人工)评价进行比较。在本文中，我们将Scratch博士提供的评估与16位计算机科学教育专家对Scratch项目的450多项评估进行了比较。我们的结果表明，自动评估和手动评估之间存在很强的相关性。由于教育工作者对这类工具的使用进行了充分的辩论，我们讨论了它的含义和局限性，并为进一步的研究提供了建议。
Programming and computational thinking skills are promoted in schools worldwide. However, there is still a lack of tools that assist learners and educators in the assessment of these skills. We have implemented an assessment tool, called Dr. Scratch, that analyzes Scratch projects with the aim to assess the level of development of several aspects of computational thinking. One of the issues to address in order to show its validity is to compare the (automatic) evaluations provided by the tool with the (manual) evaluations by (human) experts. In this paper we compare the assessments provided by Dr. Scratch with over 450 evaluations of Scratch projects given by 16 experts in computer science education. Our results show strong correlations between automatic and manual evaluations. As there is an ample debate among educators on the use of this type of tools, we discuss the implications and limitations, and provide recommendations for further research.