Cognitive assessment in a computer-based coaching environment in higher education: diagnostic assessment of development of knowledge and problem-solving skill in statistics
Cognitive assessment in a computer-based coaching environment in higher education: diagnostic assessment of development of knowledge and problem-solving skill in statistics
复制标题
高等教育中基于计算机的辅导环境中的认知评估：统计知识发展和解决问题技能的诊断评估
DOI：
--
复制
发表时间：
2007
期刊：
影响因子：
--
通讯作者：
Zhidong Zhang
中科院分区：
文献类型：
--
作者：
Zhidong Zhang
在统计学学习领域(ANOVA)中，使用贝叶斯网络和循证设计(ECD)来探索诊断认知评估(DCA)。评估环境模拟在基于网络的统计学习环境中发生的问题解决活动。评估模型由评估结构和证据模型组成。评估结构对应于认知域模型中知识和程序技能的组成部分，并在评估模型中表示为解释性变量。解释变量代表了学生评估问题表现的具体方面。贝叶斯网络被用来连接解释变量和证据变量。这些链接使网络能够将证据信息传播到评估模型中的解释性模型变量。DCA的目的是推断学生已经掌握的知识和技能的认知成分。这些推论是使用贝叶斯网络概率实现的，以基于对学生评估任务表现特征的观察来估计学生已掌握特定知识或技能的可能性。
这项研究的目的是开发一个在特定统计领域实施DCA的贝叶斯评估模型，并根据其实现DCA目标的潜力对其进行评估。本研究将模型开发方法应用于单因素方差分析模型域，以达到研究的目的。结果记录了：(A)特定领域的模型开发过程；(B)贝叶斯评估模型的性质；(C)通过使用该模型成功地更新后验概率，评估网络在跟踪学生掌握进展方面的表现；(D)使用掌握可能性的对数赔率比估计作为衡量“掌握进展”的指标；(E)基于网络的诊断推理的稳健性；以及(F)使用贝叶斯评估模型进行诊断性评估，样本为20名完成评估任务的学生。结果表明，贝叶斯评估网络提供了关于特定认知成分的有效诊断信息，并能够跟踪实现学习目标的发展。
Diagnostic cognitive assessment (DCA) was explored using Bayesian networks and evidence-centred design (ECD) in a statistics learning domain (ANOVA). The assessment environment simulates problem solving activities that occurred in a web-based statistics learning environment. The assessment model is composed of assessment constructs, and evidence models. Assessment constructs correspond to components of knowledge and procedural skill in a cognitive domain model and are represented as explanatory variables in the assessment model. Explanatory variables represent specific aspects of student's performance of assessment problems. Bayesian networks are used to connect the explanatory variables to the evidence variables. These links enable the network to propagate evidential information to explanatory model variables in the assessment model. The purpose of DCA is to infer cognitive components of knowledge and skill that have been mastered by a student. These inferences are realized probabilistically using the Bayesian network to estimate the likelihood that a student has mastered specific components of knowledge or skill based on observations of features of the student's performance of an assessment task.
The objective of this study was to develop a Bayesian assessment model that implements DCA in a specific domain of statistics, and evaluate it in relation to its potential to achieve the objectives of DCA. This study applied a method for model development to the ANOVA score model domain to attain the objectives of the study. The results documented: (a) the process of model development in a specific domain; (b) the properties of the Bayesian assessment model; (c) the performance of the network in tracing students' progress towards mastery by using the model to successfully update the posterior probabilities; (d) the use of estimates of log odds ratios of likelihood of mastery as a measure of "progress toward mastery;" (e) the robustness of diagnostic inferences based on the network; and (f) the use of the Bayesian assessment model for diagnostic assessment with a sample of 20 students who completed the assessment tasks. The results indicated that the Bayesian assessment network provided valid diagnostic information about specific cognitive components, and was able to track development towards achieving mastery of learning goals.