Effects and interaction of linguistic and subject-specific requirements in school-related assessment tasks

学校相关评估任务中语言和学科特定要求的影响和相互作用

• 批准号: 417017613
• 负责人: Professor Dr. Timo Ehmke
• 金额: --
• 依托单位: Germanistisches Institut
• 依托单位国家: 德国
• 项目类别: Research Grants
• 财政年份: 2019
• 资助国家: 德国
• 起止时间: 2018-12-31 至 2022-12-31
• 项目状态: 已结题
• 来源: https://gepris.dfg.de/gepris/projekt/417017613?language=en
• 关键词: Effects; interaction; linguistic; subject; specific

In the school subjects of mathematics and science, assessment tasks are frequently used to diagnose and measure students’ subject-specific competences and to provide them with feedback on their learning progress. Because these tasks are often presented in written form, linguistic competences are necessary pre-requisites in order to demonstrate subject-specific cognitive competences. But even though linguistic competences play a vital role for mental model building and the activation of previous knowledge, it is still largely unclear on empirical grounds in how far the linguistic requirements of assessment tasks influence their level of difficulty and how linguistic and subject-specific cognitive item features interact with each other.The aim of this study is therefore to investigate in an experimental design, how subject-specific cognitive requirements (UV1) and linguistic requirements (UV2) of assessment items in the domains of mathematics and physics influence empirical task difficulty. Besides the main effects (UV1, UV2), effects of interaction (UV1*UV2) are measured. The goal is to investigate if, and if yes, which individual learner variables (e.g. subject-specific and linguistic competences) influence empirical task difficulty. The treatment design of the experimental study is as follows: Students solve assessment tasks differing systematically in their subject-specific cognitive and linguistic requirements. The situational context is kept stable, as is the subject-specific domain in which the tasks are situated. Each assessment task is systematically varied on three subject-specific cognitive and three linguistic requirement levels (low, intermediate, high). This results in a 3x3 variation matrix for every task. 15 tasks are created for both subjects each, which in turn are varied within the 3x3 matrix design. The task variations are presented in rotated order in 9 test booklets. Students are randomly assigned to test booklets. Power analyses have identified an optimal sample size of 1350 students. The target group are 9th graders from three federal states of Germany. Methods of IRT are used for multidimensional scaling of the obtained data. This allows us to determine the task parameters for all test tasks and to analyse the effects of the subject-specific cognitive and the linguistics task requirements. Structural Equation Models and Multilevel Analysis which controls for the nested data structure are used for moderator analysis. This analysis allows to investigate which individual student variables influence the empirical task difficulty and in how far domain specific effects can be identified. The results will serve as the basis for practice-oriented recommendations for suitable task design for objective, reliable and valid diagnostic and achievement tests for subject-specific contexts.

在学校的数学和科学科目中，经常使用评估任务来诊断和测量学生的学科能力，并为他们提供关于学习进度的反馈。由于这些任务通常以书面形式呈现，语言能力是必要的先决条件，以证明特定学科的认知能力。但是，尽管语言能力在心理模型的建立和先前知识的激活中起着至关重要的作用，但在经验的基础上，评估任务的语言要求在多大程度上影响了评估任务的难度，以及语言和特定学科的认知项目特征是如何相互作用的，这在很大程度上仍然是不清楚的。数学和物理领域中评估项目的特定学科认知要求（UV1）和语言要求（UV2）如何影响经验任务难度。除了主效应（UV1、UV2）外，还测量了相互作用效应（UV1*UV2）。我们的目标是调查，如果是的话，哪些个体学习者变量（如特定学科和语言能力）影响经验任务的难度。实验研究的处理设计如下：学生解决评估任务系统不同的学科特定的认知和语言的要求。情境背景保持稳定，任务所处的特定主题领域也是如此。每项评估任务都在三个特定学科的认知和三个语言要求水平（低，中，高）上系统地变化。这导致每个任务的3x3变异矩阵。为两个受试者分别创建了15个任务，这些任务在3x3矩阵设计中依次变化。在9个测试小册子中以旋转顺序呈现任务变化。学生们被随机分配到测试小册子。功效分析确定了1350名学生的最佳样本量。目标群体是来自德国三个联邦州的九年级学生。项目反应理论的方法被用来对所获得的数据进行多维标度。这使我们能够确定所有测试任务的任务参数，并分析特定主题的认知和语言学任务要求的影响。结构方程模型和控制嵌套数据结构的多层次分析用于主持人分析。这种分析可以调查哪些学生个体变量影响经验任务的难度，以及在多大程度上可以确定域的具体效果。研究结果将作为面向实践的建议的基础，为针对具体学科的客观、可靠和有效的诊断和成绩测试提出适当的任务设计。