Design and Use of Illustrations in Test Items as a Form of Accommodation for English Language Learners in Science and Mathemcatics Assessment

该项目解决了在接受英语测试的英语学习者(ELL)中评估科学知识的有效方法的需要。该项目将调查Vignette插图在测试项目中作为ELL测试住宿形式的有效性。在插图项目中，不需要插图来理解项目的文本，并且项目的文本没有明确地引用插图。由于Vignette插图可以添加到现有试题的文本中，因此与其他形式的测试设施相比，Vignette插图不太可能改变试题的结构。该项目将调查Vignette插图如何将有限的英语水平对学生在科学考试中表现的影响降至最低。此外，它还将试图确定设计图文并茂的测试项目的基本原则。Ell和非ELL中学生将获得三种类型的项目(插图有系统地设计的插图项目，插图任意创作的插图项目和没有插图的项目)，距离制定的课程有两个水平(近距离和远距离)。不同形式的分析(包括统计分析和口头协议)将决定：(1)英语和非英语学生在使用插图理解试题的方式上是否以及如何存在显著差异；(2)插图的使用是否缩小了由于语言因素而导致的英语和非英语学生之间的考试成绩差异；(3)试题的距离是否影响了插图项目的有效性。虽然插图在测试中经常使用，但评估发展文献中关于如何使用插图的指导很有限。此外，插图作为确保ELL学生理解给定项目是关于什么以及该项目要求他们做什么的资源的价值还没有得到系统的调查。该项目将符号学、认知心理学、语言学和社会文化理论结合在一起，开发系统的程序来开发插图作为测试的视觉支持。该项目将有助于加强课堂实践和主要针对英语学习学生的大规模评估。Vignette插图有可能成为ELL的一种低成本、易于实施的测试住宿形式。将插图作为ELL考试场所的一种形式的适当设计和使用的一套原则可供各州、教师和考试行业使用。

This project addresses the need for effective approaches for assessing science knowledge among English language learners (ELLs) who are tested in English. The project will investigate the effectiveness of vignette illustrations in test items as a form of testing accommodation for ELLs. In a vignette-illustrated item, the illustration is not necessary to understand the text of the item and the text of the item makes no explicit reference to the illustration. Since they can be added to the text of existing items, vignette illustrations are less likely than other forms of testing accommodation to alter the construct that items are intended to measure.The project will investigate how vignette illustrations minimize the impact of limited English proficiency on student performance in science tests. Also, it will attempt to identify basic principles for designing illustrated test items. ELL and non-ELL middle-school students will be given items of three types (vignette-illustrated items whose illustrations are designed systematically, vignette-illustrated items whose illustrations are created arbitrarily, and items without illustrations) at two levels of distance to the enacted curriculum (close and distal). Diverse forms of analysis (including statistical analyses and verbal protocols) will determine: (1) whether and how ELL and non-ELL students differ significantly on the ways in which they use vignettes to make sense of items, (2) whether the use of vignettes reduces test score differences due to language factors between ELL and non-ELL students, and (3) whether the level of distance of the items moderates the effectiveness of vignette-illustrated items.While illustrations are frequently used in test items, there is limited guidance in the assessment development literature on how to approach illustrations. Furthermore, the value of illustrations as a resource for ensuring that ELL students understand what a given item is about and what the item asks them to do has not been systematically investigated. Semiotics, cognitive psychology, linguistics, and social cultural theory are brought together to develop systematic procedures for developing illustrations as visual supports in tests.The project will contribute to enhanced practice in both classroom and large-scale assessment primarily of ELL students. Vignette illustrations have the potential to become a low-cost, easy-to-implement form of testing accommodation for ELLs. The set of principles for the proper design and use of vignette illustrations as a form of testing accommodation for ELLs could be used by states, teachers and the testing industry.