How Can the Use of Data from Computer-Delivered Assessments Improve the Measurement of Twenty-First Century Skills?
How Can the Use of Data from Computer-Delivered Assessments Improve the Measurement of Twenty-First Century Skills?
复制标题
使用计算机评估数据如何改进 21 世纪技能的衡量？
DOI：
--
复制
发表时间：
2018
期刊：
影响因子：
--
通讯作者：
R. Adams
中科院分区：
文献类型：
--
作者：
Dara Ramalingam;R. Adams
科技不断改变着我们的生活方式。鉴于它在我们生活中的中心地位，它在教育评估中的应用越来越多，并成为近年来教育评估研究的一个重要焦点。虽然计算机交付评估的最初动机是提高评估效率，但本章表明，计算机交付可以通过捕获过程数据来增强有效性和可靠性。当计算机传送评估时，考生与环境的每一次互动都可以被记录为过程数据。过程数据的使用很有希望提供以前无法获得的见解，不仅可以了解学生是否解决了任务，还可以了解他们如何解决任务。此外，正是那些有助于培养21世纪技能的过程，才有可能在教学和学习方面实现直接目标。然而，在没有分析和使用计划的情况下收集大量信息不太可能产生有用的结果。通过对2012年国际学生评估项目（PISA）中收集的数字阅读评估过程数据的项目反应理论分析，本章说明了如何使用与学生导航问题空间的方式相关的过程数据来提高有效性和可靠性。通过拟合数据的备选项目反应模型，表明如果在这些数据与感兴趣领域的能力发展理论之间建立了明确的联系，则可以通过使用过程数据来改进测量。
Technology is continuing to change the way we live. Given its centrality to our lives, it is not surprising that its use in educational assessment has been increasing, and has been an important focus of education assessment research in recent years. While the initial motivation for computer-delivered assessments was gains in assessment efficiency, this chapter demonstrates that computer delivery can enhance validity and reliability through the capture of process data. When an assessment is computer-delivered, every interaction of the test-taker with the environment may be recorded as process data. The use of process data holds much promise for providing previously inaccessible insights into not just whether a student solved a task, but how they did so. Further, it is the processes that contribute to twenty-first century skills that are likely to be amenable to direct targeting in terms of teaching and learning. However, collecting large amounts of information in the absence of a plan for its analysis and use is unlikely to lead to useful outcomes. Through item response theory analysis of process data collected in the Digital Reading Assessment included as part of the 2012 cycle of the Programme for International Student Assessment (PISA), this chapter illustrates how process data that relates to the way a student navigates the problem space can be used to improve validity and reliability. By fitting alternative item response models to the data, it is shown that measurement can be improved by using process data if a clear connection is made between these data and theories of developing competence in the domain of interest.