Doctoral Dissertation Research: Examining Sequence of Contextualized Items in Science

学生在科学测试中的表现往往受到个别测试项目背景的影响。 本博士论文的研究项目将系统地描述科学测试项目上下文中使用的序列模式，并获得关于上下文信息的序列模式如何支持或阻碍学生在科学测试中的表现的经验证据。 上下文信息包括补充信息，诸如在测试项目之前或之后的小插曲或所选背景事实。 该项目将通过提供以下内容来解决关于情境化项目的文献中的三个空白：一个用于选择待研究的测试项目特征的一致的基于理论的框架;关于学生成绩如何受到测试项目的特定元素或特征影响的新知识;以及更好地了解学生分组及其表现，以推进关于如何实现一致公平测试的知识。 特别是，这个项目将产生的理论和经验证据，关于如何序列的上下文信息可以归因于英语学习者（ELL）和非ELL组之间的差异表现。 从ELL和非ELL比较的结果将导致重要的改进，在如何开发项目，以确保测试的公平性。 作为博士论文研究改进奖，该项目将帮助有前途的学生建立一个强大的，独立的研究生涯。该项目的设计和数据分析将由三个研究问题构成：（1）学生的表现如何与科学测试项目的上下文信息序列的不同维度相关联？ (2)控制不同层次的语言复杂性在上下文中，是如何联系的上下文序列的科学表现的英语学习者和非英语学习者？ (3)语境序列的不同维度如何影响学生对任务的解释和反应？ 该项目将包括调查的多个方面，包括心理测量学，测试项目的创建，误解和认知诊断评估。 该项目将根据上下文信息序列的维度生成项目开发指南。 从这个过程中产生的项目将唤起学生存储的知识相关的内容和/或正在评估的过程技能。 然后，新开发的科学项目将在不同的八年级学生中进行实地测试。 一系列的心理测量和统计程序将适用于学生的考试成绩。 该活动将包括有声思维和眼动追踪学生认知访谈，并对认知访谈数据进行定量和定性分析。

Student performance on science tests often is substantially influenced by the context of individual test items. This doctoral dissertation research project will systematically describe the sequence patterns used in science test-item contexts and obtain empirical evidence on how the sequence patterns of contextual information supports or hinders student performance on science tests. Contextual information includes supplemental information, such as a vignette or selected background facts that precede or follow a test item. The project will address three gaps in the literature on contextualized items by providing: a consistent theory-based framework for selecting test-item characteristics to be studied; new knowledge about how student performance is influenced by specific elements or features of the test items; and better understanding of subgroups of students and their performance to advance knowledge about how to achieve consistently equitable testing. In particular, this project will produce both theory and empirical evidence regarding how the sequence of contextual information can be attributed to differential performance between English language learners (ELL) and non-ELL groups. Findings from the ELL and non-ELL comparisons will lead to important improvements in how items are developed to ensure test fairness. As a Doctoral Dissertation Research Improvement award, the project will enable a promising student to establish a strong, independent research career.The project's design and data analyses will be framed by three research questions: (1) How is student performance associated with different dimensions of the sequence of contextual information for science test items? (2) Controlling for different levels of linguistic complexity in contexts, how is the contextual sequence linked to science performance of ELLs and non-ELLs? (3) How do different dimensions of sequence of context influence how students interpret and respond to tasks? The project will include multiple facets of inquiry, including psychometrics, test-item creation, misconceptions, and cognitive diagnostic assessment. The project will generate item-development guidelines based on dimensions of sequence of contextual information. Items resulting from this process will evoke students' stored knowledge relevant to the content and/or process skills being assessed. The newly developed science items then will be field tested with a diverse population of eighth graders. A range of psychometric and statistical procedures will be applied to student test scores. This activity will include both think-aloud and eye-tracking student cognitive interviews, with quantitative and qualitative analyses of the cognitive interview data.