Construction of Instructionally Sensitive Test Items

在过去的几十年里，对学生成绩和能力的评估在教育研究和政策中得到了广泛的应用。例如，政策制定者依靠学生的考试成绩来让学校对他们的行为负责（例如，2015年《每个学生成功法案》（Every Student Succeeds Act）。教育研究人员经常使用学生的考试成绩作为因变量，以确定促进或损害学生学习的个人，课堂或学校水平的特征。此外，学生的成绩是衡量具体教学质量方面成效的最常用标准。也就是说，教育研究者和政策制定者都希望考试能够提供关于学生个人能力或教学效果的有益信息（Pellegrino，2002）。这种解释要求工具能够捕捉课堂教学的效果，也就是说，考试和项目是敏感的（Popham，2007）。因此，教学敏感性是必不可少的有效性使用的测试成绩作为标准的教学效果。然而，尽管现有的测试和测试项目的教学敏感性的声音和连贯的方法的可用性，有很少的知识，如何构建新的教学敏感性项目。因此，拟议的项目旨在两个主要目标，（1）调查项目的属性，提高教学的敏感性和（2）更密切的检查教学变量，链接课堂教学和标准化测试项目的反应。该项目研究的内容领域是中学数学教学中教授的线性方程。确定了五个项目里程碑，每个里程碑标志着项目连续阶段的成功完成。这些主要阶段是（I）通过专家访谈和二次分析的方式识别和规范项目属性，（II）根据所识别的项目属性系统地开发测试项目，（III）收集学生成绩数据以及课堂人工制品，（IV）分析以回答主要研究问题，（V）公布教育测量和教学研究的结果，有目的地构建教学敏感性测试项目有助于确保测试的教学敏感性，而不需要广泛的预测试或依赖现有的工具，从而更有经济效益。此外，知识如何构建敏感的项目将允许充分证实教育评估的有效性参数。

Over the last decades, assessments of students’ achievement and competencies have found widespread use in educational research and policy. For example, policy-makers rely on student test scores to hold schools accountable for their action (e.g., Every Student Succeeds Act of 2015). Educational researchers regularly use student test scores as dependent variables to identify individual, classroom- or school-level characteristics that foster or impair students’ learning. Moreover, students’ achievement is the most commonly applied criterion for judging the effectiveness of specific teaching quality aspects. That is, both educational researchers and policy-makers expect tests to provide beneficial information on students’ individual abilities or the effectiveness of teaching (Pellegrino, 2002).This interpretation requires that instruments are capable of capturing effects of classroom instruction, that is, that tests and items are instructionally sensitive (Popham, 2007). Thus, instructional sensitivity is essential for the validity of the use of test scores as criteria for teaching effectiveness. Yet despite the availability of sound and coherent ways of measuring instructional sensitivity of existing tests and test items, there is little knowledge on how to construct new instructionally sensitive items. Thus, the proposed project aims at two main objectives, (1) the investigation of item properties that enhance instructional sensitivity and (2) a closer examination of instructional variables that link classroom instruction and responses in standardized test items. The content area to be studied in the project is linear equations as taught in mathematics instruction at secondary level. Five project milestones are defined, each marking the successful completion of successive phases of the project. These main phases are (I) the identification and specification of item properties by means of expert interviews and secondary analyses, (II) the development of test items systematically varying with respect to the identified item properties, (III) the collection of student achievement data as well as classroom artefacts, (IV) analyses to answer the main research questions, and (V) the publication of results for audiences both from educational measurement as well as from instructional research.Being able to purposefully build instructionally sensitive items will help to ensure the instructional sensitivity of tests without extensive pretesting or relying on existing instruments, thus being more economically beneficial. Moreover, knowledge on how to construct instructionally sensitive items will allow adequately substantiating validity arguments of educational assessments.