Construction of Instructionally Sensitive Test Items

在过去的几十年里，对学生成绩和能力的评估在教育研究和政策中得到了广泛的应用。例如，政策制定者依靠学生考试成绩来追究学校对他们的行为的责任(例如，每个学生都成功通过了2015年的法案)。教育研究人员经常使用学生考试成绩作为因变量，以确定促进或损害学生学习的个人、课堂或学校层面的特征。此外，学生的成绩是衡量具体教学质量方面有效性的最常用的标准。也就是说，教育研究者和政策制定者都希望测试能够提供有关学生个人能力或教学效果的有益信息(Pellegrino，2002)。这种解释要求工具能够捕捉课堂教学的效果，即测试和项目是教学敏感的(Popham，2007)。因此，教学敏感度对于使用考试分数作为教学有效性的标准的有效性至关重要。然而，尽管有可靠和连贯的方法来衡量现有测试和测试项目的教学敏感性，但关于如何构建新的教学敏感项目却知之甚少。因此，拟议的项目旨在两个主要目标，(1)调查项目属性，以提高教学敏感性和(2)更密切地检查教学变量联系课堂教学和标准化测试项目的反应。本课题研究的内容领域是中学数学教学中所教授的线性方程。定义了五个项目里程碑，每个里程碑都标志着项目连续阶段的成功完成。这些主要阶段是(I)通过专家访谈和二次分析来确定和说明试题属性，(Ii)根据所确定的试题属性系统地开发测试项目，(Iii)收集学生成绩数据和课堂人工制品，(Iv)分析以回答主要的研究问题，以及(V)为受众发布教育测量和教学研究的结果。能够有目的地构建教学敏感项目将有助于确保测试的教学敏感性，而不需要广泛的预先测试或依赖现有的工具，因此更具经济效益。此外，关于如何构建教学敏感项目的知识将使教育评估的有效性论点得到充分证实。

Over the last decades, assessments of students’ achievement and competencies have found widespread use in educational research and policy. For example, policy-makers rely on student test scores to hold schools accountable for their action (e.g., Every Student Succeeds Act of 2015). Educational researchers regularly use student test scores as dependent variables to identify individual, classroom- or school-level characteristics that foster or impair students’ learning. Moreover, students’ achievement is the most commonly applied criterion for judging the effectiveness of specific teaching quality aspects. That is, both educational researchers and policy-makers expect tests to provide beneficial information on students’ individual abilities or the effectiveness of teaching (Pellegrino, 2002).This interpretation requires that instruments are capable of capturing effects of classroom instruction, that is, that tests and items are instructionally sensitive (Popham, 2007). Thus, instructional sensitivity is essential for the validity of the use of test scores as criteria for teaching effectiveness. Yet despite the availability of sound and coherent ways of measuring instructional sensitivity of existing tests and test items, there is little knowledge on how to construct new instructionally sensitive items. Thus, the proposed project aims at two main objectives, (1) the investigation of item properties that enhance instructional sensitivity and (2) a closer examination of instructional variables that link classroom instruction and responses in standardized test items. The content area to be studied in the project is linear equations as taught in mathematics instruction at secondary level. Five project milestones are defined, each marking the successful completion of successive phases of the project. These main phases are (I) the identification and specification of item properties by means of expert interviews and secondary analyses, (II) the development of test items systematically varying with respect to the identified item properties, (III) the collection of student achievement data as well as classroom artefacts, (IV) analyses to answer the main research questions, and (V) the publication of results for audiences both from educational measurement as well as from instructional research.Being able to purposefully build instructionally sensitive items will help to ensure the instructional sensitivity of tests without extensive pretesting or relying on existing instruments, thus being more economically beneficial. Moreover, knowledge on how to construct instructionally sensitive items will allow adequately substantiating validity arguments of educational assessments.