Construction of Instructionally Sensitive Test Items

在过去的几十年里，对学生成绩和能力的评估在教育研究和政策中得到了广泛的应用。例如，政策制定者依靠学生的考试成绩来要求学校对其行为负责（例如，2015年《每个学生成功法案》）。教育研究人员经常使用学生考试成绩作为因变量，以确定促进或损害学生学习的个人、课堂或学校层面的特征。此外，学生成绩是衡量具体教学质量方面有效性的最常用标准。也就是说，教育研究者和政策制定者都希望考试能提供关于学生个人能力或教学效果的有益信息（Pellegrino, 2002）。这种解释要求工具能够捕捉课堂教学的效果，也就是说，测试和项目具有教学敏感性（Popham, 2007）。因此，教学敏感性对于使用考试成绩作为教学有效性标准的有效性至关重要。然而，尽管有可靠和连贯的方法来测量现有测试和测试项目的教学敏感性，但关于如何构建新的教学敏感性项目的知识很少。因此，拟议的项目旨在实现两个主要目标，(1)调查提高教学敏感性的项目属性，(2)更仔细地检查将课堂教学和标准化测试项目的反应联系起来的教学变量。项目研究的内容领域是中学数学教学中的线性方程。定义了五个项目里程碑，每个里程碑标志着项目连续阶段的成功完成。这些主要阶段是(I)通过专家访谈和二次分析来识别和规范项目属性，（II）根据识别的项目属性系统地开发不同的测试项目，（III）收集学生成绩数据和课堂人工制品，（IV）分析以回答主要研究问题，以及(V)从教育测量和教学研究中向受众发布结果。能够有目的地构建具有教学敏感性的项目将有助于确保测试的教学敏感性，而无需大量的预测试或依赖现有仪器，从而更具经济效益。此外，关于如何构建教学敏感项目的知识将允许充分证实教育评估的有效性论点。

Over the last decades, assessments of students’ achievement and competencies have found widespread use in educational research and policy. For example, policy-makers rely on student test scores to hold schools accountable for their action (e.g., Every Student Succeeds Act of 2015). Educational researchers regularly use student test scores as dependent variables to identify individual, classroom- or school-level characteristics that foster or impair students’ learning. Moreover, students’ achievement is the most commonly applied criterion for judging the effectiveness of specific teaching quality aspects. That is, both educational researchers and policy-makers expect tests to provide beneficial information on students’ individual abilities or the effectiveness of teaching (Pellegrino, 2002).This interpretation requires that instruments are capable of capturing effects of classroom instruction, that is, that tests and items are instructionally sensitive (Popham, 2007). Thus, instructional sensitivity is essential for the validity of the use of test scores as criteria for teaching effectiveness. Yet despite the availability of sound and coherent ways of measuring instructional sensitivity of existing tests and test items, there is little knowledge on how to construct new instructionally sensitive items. Thus, the proposed project aims at two main objectives, (1) the investigation of item properties that enhance instructional sensitivity and (2) a closer examination of instructional variables that link classroom instruction and responses in standardized test items. The content area to be studied in the project is linear equations as taught in mathematics instruction at secondary level. Five project milestones are defined, each marking the successful completion of successive phases of the project. These main phases are (I) the identification and specification of item properties by means of expert interviews and secondary analyses, (II) the development of test items systematically varying with respect to the identified item properties, (III) the collection of student achievement data as well as classroom artefacts, (IV) analyses to answer the main research questions, and (V) the publication of results for audiences both from educational measurement as well as from instructional research.Being able to purposefully build instructionally sensitive items will help to ensure the instructional sensitivity of tests without extensive pretesting or relying on existing instruments, thus being more economically beneficial. Moreover, knowledge on how to construct instructionally sensitive items will allow adequately substantiating validity arguments of educational assessments.