Design and Use of Illustrations in Test Items as a Form of Accommodation for English Language Learners in Science and Mathemcatics Assessment

本项目探讨了在接受英语测试的英语学习者中评估科学知识的有效方法的需求。该项目将调查测试项目中的小插图作为ELLs测试住宿形式的有效性。在带有插图的项目中，不需要插图来理解项目的文本，项目的文本也没有明确提及插图。因为它们可以添加到现有项目的文本中，所以小插图比其他形式的测试调节更不可能改变项目要测量的结构。该项目将调查小插图如何将有限的英语水平对学生在科学测试中的表现的影响降到最低。此外，它将试图确定设计图解测试项目的基本原则。ELL和非ELL中学生将获得三种类型的项目（插图系统设计的小插图项目，插图任意创建的小插图项目，以及没有插图的项目），它们与制定的课程的距离有两个级别（近和远）。各种形式的分析（包括统计分析和口头协议）将确定：(1)英语好学生和非英语好学生在使用小插图来理解项目的方式上是否以及如何存在显著差异，(2)小插图的使用是否会减少由于语言因素导致的英语好学生和非英语好学生之间的测试成绩差异，以及(3)项目的距离水平是否会调节小插图项目的有效性。虽然在测试项目中经常使用插图，但在评估开发文献中关于如何使用插图的指导有限。此外，插图作为一种资源的价值，确保ELL学生理解一个给定的项目是关于什么的，项目要求他们做什么，还没有被系统地调查过。符号学、认知心理学、语言学和社会文化理论汇集在一起，制定了系统的程序，以开发插图作为测试中的视觉支持。该项目将有助于加强英语学生的课堂实践和大规模评估。小插图有可能成为ELLs的低成本，易于实施的测试住宿形式。适当设计和使用小插图作为ELLs测试便利形式的一套原则可以被各州，教师和测试行业使用。

This project addresses the need for effective approaches for assessing science knowledge among English language learners (ELLs) who are tested in English. The project will investigate the effectiveness of vignette illustrations in test items as a form of testing accommodation for ELLs. In a vignette-illustrated item, the illustration is not necessary to understand the text of the item and the text of the item makes no explicit reference to the illustration. Since they can be added to the text of existing items, vignette illustrations are less likely than other forms of testing accommodation to alter the construct that items are intended to measure.The project will investigate how vignette illustrations minimize the impact of limited English proficiency on student performance in science tests. Also, it will attempt to identify basic principles for designing illustrated test items. ELL and non-ELL middle-school students will be given items of three types (vignette-illustrated items whose illustrations are designed systematically, vignette-illustrated items whose illustrations are created arbitrarily, and items without illustrations) at two levels of distance to the enacted curriculum (close and distal). Diverse forms of analysis (including statistical analyses and verbal protocols) will determine: (1) whether and how ELL and non-ELL students differ significantly on the ways in which they use vignettes to make sense of items, (2) whether the use of vignettes reduces test score differences due to language factors between ELL and non-ELL students, and (3) whether the level of distance of the items moderates the effectiveness of vignette-illustrated items.While illustrations are frequently used in test items, there is limited guidance in the assessment development literature on how to approach illustrations. Furthermore, the value of illustrations as a resource for ensuring that ELL students understand what a given item is about and what the item asks them to do has not been systematically investigated. Semiotics, cognitive psychology, linguistics, and social cultural theory are brought together to develop systematic procedures for developing illustrations as visual supports in tests.The project will contribute to enhanced practice in both classroom and large-scale assessment primarily of ELL students. Vignette illustrations have the potential to become a low-cost, easy-to-implement form of testing accommodation for ELLs. The set of principles for the proper design and use of vignette illustrations as a form of testing accommodation for ELLs could be used by states, teachers and the testing industry.