Effects and interaction of linguistic and subject-specific requirements in school-related assessment tasks

在学校的数学和科学学科中，评估任务经常被用来诊断和衡量学生的特定学科能力，并为他们提供学习进度的反馈。由于这些任务通常以书面形式呈现，因此语言能力是展示特定学科认知能力的必要先决条件。但是，尽管语言能力在心智模型的建立和先前知识的激活中起着至关重要的作用，但在评估任务的语言要求对其难度水平的影响程度以及语言和特定主题的认知项目特征如何相互作用方面，从经验的角度来看，仍然很大程度上不清楚。因此，本研究的目的是通过实验设计来探讨数学和物理领域评估项目的学科特定认知要求（UV1）和语言要求（UV2）如何影响经验任务难度。除主要效应（UV1, UV2）外，还测量了相互作用效应（UV1*UV2）。目标是调查是否，如果是，哪些学习者的个体变量（例如，特定学科和语言能力）影响经验任务难度。实验研究的处理设计如下：学生系统地解决不同学科认知和语言要求的评估任务。情境上下文保持稳定，任务所在的特定于主题的领域也是如此。每个评估任务在三个特定学科的认知和三个语言要求水平（低、中、高）上系统地变化。这将为每个任务生成一个3x3的变化矩阵。每个主题都创建了15个任务，这些任务依次在3x3矩阵设计中变化。任务的变化在9个测试手册中以旋转顺序呈现。学生们被随机分配到试卷上。功率分析确定了1350名学生的最佳样本量。目标群体是来自德国三个联邦州的九年级学生。利用IRT方法对得到的数据进行多维尺度变换。这使我们能够确定所有测试任务的任务参数，并分析特定学科认知和语言学任务要求的影响。调节因子分析采用结构方程模型和多层分析方法，对嵌套数据结构进行控制。该分析允许调查哪些学生个体变量影响经验任务难度，以及在多大程度上可以确定特定领域的影响。这些结果将作为面向实践的建议的基础，以便为特定学科的客观、可靠和有效的诊断和成绩测试设计适当的任务。

In the school subjects of mathematics and science, assessment tasks are frequently used to diagnose and measure students’ subject-specific competences and to provide them with feedback on their learning progress. Because these tasks are often presented in written form, linguistic competences are necessary pre-requisites in order to demonstrate subject-specific cognitive competences. But even though linguistic competences play a vital role for mental model building and the activation of previous knowledge, it is still largely unclear on empirical grounds in how far the linguistic requirements of assessment tasks influence their level of difficulty and how linguistic and subject-specific cognitive item features interact with each other.The aim of this study is therefore to investigate in an experimental design, how subject-specific cognitive requirements (UV1) and linguistic requirements (UV2) of assessment items in the domains of mathematics and physics influence empirical task difficulty. Besides the main effects (UV1, UV2), effects of interaction (UV1*UV2) are measured. The goal is to investigate if, and if yes, which individual learner variables (e.g. subject-specific and linguistic competences) influence empirical task difficulty. The treatment design of the experimental study is as follows: Students solve assessment tasks differing systematically in their subject-specific cognitive and linguistic requirements. The situational context is kept stable, as is the subject-specific domain in which the tasks are situated. Each assessment task is systematically varied on three subject-specific cognitive and three linguistic requirement levels (low, intermediate, high). This results in a 3x3 variation matrix for every task. 15 tasks are created for both subjects each, which in turn are varied within the 3x3 matrix design. The task variations are presented in rotated order in 9 test booklets. Students are randomly assigned to test booklets. Power analyses have identified an optimal sample size of 1350 students. The target group are 9th graders from three federal states of Germany. Methods of IRT are used for multidimensional scaling of the obtained data. This allows us to determine the task parameters for all test tasks and to analyse the effects of the subject-specific cognitive and the linguistics task requirements. Structural Equation Models and Multilevel Analysis which controls for the nested data structure are used for moderator analysis. This analysis allows to investigate which individual student variables influence the empirical task difficulty and in how far domain specific effects can be identified. The results will serve as the basis for practice-oriented recommendations for suitable task design for objective, reliable and valid diagnostic and achievement tests for subject-specific contexts.