Selecting Student-Authored Questions for Summative Assessments
Selecting Student-Authored Questions for Summative Assessments
复制标题
选择学生撰写的问题进行总结性评估
DOI：
--
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
G. Denyer
中科院分区：
文献类型：
--
作者：
Alice Huang;D. Hancock;Matthew Clemson;Giselle C. Yeo;Dylan J. Harney;Paul Denny;G. Denyer
为形成性和总结性评估制作高质量的多项选择题（MCQ）是一项耗时的任务，需要很高的技能，创造力和洞察力。向在线考试的过渡，以及随之而来的以前久经考验的MCQ的曝光，加剧了问题制作的挑战，并突出了对创新解决方案的需求。几个小组已经表明，利用学生群体为学习库制作大量与教学大纲一致的MCQ是切实可行的。虽然学生生成的问题非常适合形成性反馈和实践活动，但通常认为它们不适合高风险评估。在这项研究中，我们的目的是证明，培训可以提供给学生在一个可扩展的方式，以产生类似质量的问题，由专家产生的，并确定合适的问题，可以实现最小的学术审查和编辑。生物化学和分子生物学的学生被分配了一系列的活动，旨在指导他们在写作和批评MCQ的艺术。这项培训产生了1000多个MCQ，然后通过专家的学术判断或通过数据驱动的方法来衡量这些问题的潜力，这些问题在低风险测试中进行了客观的测试。通过这两种方法选择的问题，然后部署在一个高风险的学期内评估问题，从两个学术创作的来源：教科书衍生MCQ，和过去的论文问题。来自这四个来源的120个MCQ被部署在超过600名学生的评估中。每个问题都经过严格的性能分析，包括从经典测试理论和更复杂的项目反应理论（IRT）测量标准度量的计算。结果表明，MCQs撰写的学生和选择在低成本的表现，以及由学者撰写的问题，说明了这种策略的潜力，有效地创造了大量的高质量MCQs的终结性评估。
Production of high-quality multiple-choice questions (MCQs) for both formative and summative assessment is a time-consuming task requiring great skill, creativity, and insight. The transition to online examinations, with the concomitant exposure of previously tried-and-tested MCQs, exacerbates the challenges of question production, and highlights the need for innovative solutions. Several groups have shown that it is practical to leverage the student cohort to produce a very large number of syllabus-aligned MCQs for study banks. Although student-generated questions are well suited for formative feedback and practice activities, they are generally not thought to be suitable for high-stakes assessments. In this study, we aimed to demonstrate that training can be provided to students in a scalable fashion to generate questions of similar quality to those produced by experts, and that identification of suitable questions can be achieved with minimal academic review and editing. Biochemistry and Molecular Biology students were assigned a series of activities designed to coach them in the art of writing and critiquing MCQs. This training resulted in the production of over one thousand MCQs that were then gauged for potential by either expert academic judgement, or via a data-driven approach in which the questions were trialled objectively in a low-stakes test. Questions selected by either method were then deployed in a high-stakes in-semester assessment alongside questions from two academically authored sources: textbook-derived MCQs, and past paper questions. A total of 120 MCQs from these four sources were deployed in assessments attempted by over 600 students. Each question was subjected to rigorous performance analysis, including the calculation of standard metrics from classical test theory and more sophisticated Item Response Theory (IRT) measures. The results showed that MCQs authored by students and selected at low cost performed as well as questions authored by academics, illustrating the potential of this strategy for the efficient creation of large numbers of high quality MCQs for summative assessment.
独立于语言的 CS1 知识评估的项目反应理论评估
DOI：
10.1145/3287324.3287370
发表时间：
2019
期刊：
ACM Technical Symposium on Computer Science Education
影响因子：
--
作者：
Xie, Benjamin;Davidson, Matthew J.;Li, Min;Ko, Andrew J.
通讯作者：
Ko, Andrew J.