Analysis of rater-(method-)effects in empirical educational research

该研究方案旨在对经验性教育研究中的评分者(或方法)效应进行系统和全面的分析。在本项目中，方法效应被定义为潜在的残差变量，它捕捉了教师对学生非认知特征的独特观点，并对学生的自我评价进行了修正。因此，潜在的方法效应带有教师不准确的系统性影响，需要深入研究，不应将其误解为错误的评价。利用适当的潜变量建模方法对嵌套结构不同方法的多水平MTMM设计进行分析，考察哪些预测变量能够在个体和班级水平上解释潜在方法(教师)效应。该项目将填补两个关键的研究空白。首先，它将提供有关教师在判断学生的非认知特征时不准确的决定因素的重要信息。到目前为止，人们对教育实证研究中方法(评分者)效应的潜在原因知之甚少。其次，本项目对CFA MTMM模型中潜在因素方法的充分分析做出了重要的方法论贡献。

The research proposal aims at a systematic and comprehensive analysis of rater (or method) effects in the empirical education research. In this project, method effects are defined as latent residual variables capturing teachers' unique perspective on students' non-cognitive characteristics corrected for students' self-evaluations. Hence, latent method effects carry systematic influences of teacher inaccuracy that needs to examined thoroughly and should not be misinterpreted as a false evaluation. Using adequate latent variable modeling approaches for the analysis of multilevel MTMM designs with nested structurally different methods, it will be examined what predictor variables can explain latent method (teacher) effect at the individual and the class level. This project will fill two key research gaps. First, it will provide important information with regard to the determinants of teacher inaccuracy when judging students' non-cognitive characteristics. So far, little is known about the potential causes of method (rater) effects in empirical educational research. Secondly, an important methodological contribution toward the adequate analysis of latent factors methods in CFA MTMM models is done in this project.