Targeting data collection in games based assessment
Targeting data collection in games based assessment
复制标题
基于游戏的评估中的目标数据收集
DOI：
10.1016/j.caeo.2021.100054
复制
发表时间：
2021
影响因子：
3.6
通讯作者：
Walsh C
中科院分区：
文献类型：
--
作者：
Walsh C
教育游戏性能数据有可能允许评估新类型的复杂程序技能。然而，先前的工作已经表明，游戏数据不容易与现有的评估验证范式一致，并且游戏性能分数难以用于熟练度测试。一个新的评估范式，可以科普游戏数据的性质还没有出现。在本文中，我们发现了一系列由游戏中的工程环境引起的数据收集结构问题，并可能通过这些问题得到解决。选择和游戏的迭代性质被发现允许课程专业化。我们发现的证据表明，早期尝试新的游戏是不太可靠的，也许最好放弃，我们提出了一个解决方案，权重分数，以反映重复任务的新奇。我们发现捕捉竞争对手或合作者能力对性能的影响具有挑战性，但提出了机器人解决这个问题的潜力。最后，我们还研究了使用响应时间作为能力的代理。时间的物理测量被证明是困难的，而且使用起来可能不公平，但我们提出了一种可能的速度随机处理方法，可以在一些游戏中使用响应时间来获得一些技能。
Educational game performance data has the potential to allow new types of complex, procedural skills to be assessed. However, prior work has shown that gameplay data do not readily align to existing assessment validation paradigms, and game performance scores are difficult to use for proficiency testing. A new assessment paradigm that can cope with the nature of gameplay data has not emerged. In this paper, we uncovered a range of structural issues in data collection caused by, and potentially solved by, the engineered environments in games. Choice and the iterative nature of games were found to allow curriculum specialisation. We found evidence that early attempts at new games are less reliable and perhaps best discarded, and we propose a solution to weight scores to reflect novelty in repeated tasks. We found capturing the effect of competitor or collaborator ability on performance challenging but propose the potential for bots to resolve this. Finally, we also investigated the use of response time as a proxy for ability. The physical measure of time proved difficult and potentially unfair to use, but we propose a possible stochastic treatment of speed that could allow scoring some skills in some games using response time.