Rigging the deck: Selecting good problems for expert-novice card-sorting experiments
Rigging the deck: Selecting good problems for expert-novice card-sorting experiments
复制标题
准备好甲板：为专家-新手卡片分类实验选择好的问题
DOI：
--
复制
发表时间：
2012
期刊：
影响因子：
--
通讯作者：
G. Kortemeyer
中科院分区：
文献类型：
--
作者：
S. F. Wolf;D. Dougherty;G. Kortemeyer
Chi等人的一项开创性研究牢固地建立了新手按"表面特征"（例如，"倾斜"、"摆"、"抛射运动"等），尽管专家使用"深层结构"（例如，“能量守恒”、“牛顿2”等）。然而，重复这项研究的努力经常失败，因为区分专家和新手的能力对所使用的问题集非常敏感。在以可测量的方式区分专家和新手的问题集中，问题的哪些属性是最重要的？为了回答这个问题，我们研究了已知物理专家和新手使用大量不同问题的分类。这个集合需要很大，这样我们就可以通过考虑使用穷举蒙特卡罗方法的小子集和使用模拟退火的较大子集来确定专家和新手的区分程度。我们发现，只要问题集是精心制作的，由具有特定教学和上下文特征的问题组成，准确分类专家和新手所需的问题数量可以惊人地少。最后，我们发现，不仅你问什么（深层结构）很重要，而且你问它的方式（问题背景）也很重要。
A seminal study by Chi et al. firmly established the paradigm that novices categorize physics problems by ‘‘surface features’’ (e.g., ‘‘incline,’’ ‘‘pendulum,’’ ‘‘projectile motion,’’ etc.), while experts use ‘‘deep structure’’ (e.g., ‘‘energy conservation,’’ ‘‘Newton 2,’’ etc.). Yet, efforts to replicate the study frequently fail, since the ability to distinguish experts from novices turns out to be highly sensitive to the problem set being used. Exactly what properties of problems are most important in problem sets that discriminate experts from novices in a measurable way? To answer this question, we studied the categorizations by known physics experts and novices using a large, diverse set of problems. This set needed to be large so that we could determine how well experts and novices could be discriminated by considering both small subsets using an exhaustive Monte Carlo approach and larger subsets using simulated annealing. We found that the number of questions required to accurately classify experts and novices can be surprisingly small so long as the problem set is carefully crafted to be composed of problems with particular pedagogical and contextual features. Finally, we found that not only was what you ask (deep structure) important, but also how you ask it (problem context).