Analysis of rater-(method-)effects in empirical educational research

本研究计划旨在系统、全面地分析实证教育研究中评估者（或方法）的效果。在本项目中，方法效应被定义为潜在的剩余变量，它捕获了教师对学生非认知特征的独特视角，这些特征被学生的自我评价所纠正。因此，潜在的方法效应具有教师不准确性的系统性影响，需要彻底检查，不应被误解为错误的评价。利用充分的潜在变量建模方法来分析具有嵌套结构不同方法的多层MTMM设计，将检验哪些预测变量可以解释个人和班级水平上的潜在方法（教师）效应。这个项目将填补两个关键的研究空白。首先，它将提供关于教师在判断学生非认知特征时不准确的决定因素的重要信息。迄今为止，对实证教育研究中方法效应的潜在原因所知甚少。其次，本项目为充分分析CFA MTMM模型中的潜在因素方法做出了重要的方法学贡献。

The research proposal aims at a systematic and comprehensive analysis of rater (or method) effects in the empirical education research. In this project, method effects are defined as latent residual variables capturing teachers' unique perspective on students' non-cognitive characteristics corrected for students' self-evaluations. Hence, latent method effects carry systematic influences of teacher inaccuracy that needs to examined thoroughly and should not be misinterpreted as a false evaluation. Using adequate latent variable modeling approaches for the analysis of multilevel MTMM designs with nested structurally different methods, it will be examined what predictor variables can explain latent method (teacher) effect at the individual and the class level. This project will fill two key research gaps. First, it will provide important information with regard to the determinants of teacher inaccuracy when judging students' non-cognitive characteristics. So far, little is known about the potential causes of method (rater) effects in empirical educational research. Secondly, an important methodological contribution toward the adequate analysis of latent factors methods in CFA MTMM models is done in this project.