Explaining Inconsistent Effects of Teaching Quality on Educational Outcomes

解释教学质量对教育成果的不一致影响

• 批准号: 518236946
• 负责人: Dr. Carmen Köhler
• 金额: --
• 依托单位: DIPF I Leibniz-Institut für Bildungsforschung und Bildungsinformation
• 依托单位国家: 德国
• 项目类别: Research Grants
• 资助国家: 德国
• 项目状态: 未结题
• 来源: https://gepris.dfg.de/gepris/projekt/518236946?language=en
• 关键词: Explaining; Inconsistent; Effects; Teaching; Quality

The focus of educational effectiveness research is to identify aspects of teaching that foster student learning outcomes. The ultimate goal is to inform policy makers and practitioners in the field about teaching practices and educational initiatives that improve student learning. Various meta-analyses within the research field exist. They show large deviations in effect sizes across studies that investigated the same substantial research question, which implies replication failure. Inconsistent and, in part, even contradictory results hinder straightforward recommendations and raise doubts about the credibility of the research results. Reasons for deviations could be that the studies vary with respect to several aspects: (a) the setting, (b) the design, (c) the operationalization of the constructs, and (d) the methodology used in the statistical analyses. Meta-analyses and reviews showed that some of those aspects influenced the estimated effect of teaching quality on student outcome. Since the meta-analyses and reviews gathered information from already existing studies, however, they were unable to keep certain aspects such as the statistical model constant, and were thus unable to disentangle the different aspects that are responsible for heterogeneous effects. Conducting a systematic re-analysis of already existing data, our two main aims are (1) to identify and precisely define possible factors that might explain variance in effects of teaching quality, and (2) to provide estimates of the relative importance of those factors for heterogeneity of results. In pursuit of the first research objective, we will categorize data sets with respect to differences in (a) the design and setting and (b) the operationalization of constructs. We will further define methodological factors that might influence the estimated effect. To achieve the second research objective, we will harmonize the data sets, run the secondary analyses using different methodological approaches, extract the estimated effects from the individual analyses and transform them onto the same scale (in the form of an effect size), and estimate which aspects explain which proportion of variance in the effects. In the end, these findings aid in the planning of future generalizability analyses and conceptual replications that result in meaningful recommendations for practitioners.

教育有效性研究的重点是确定促进学生学习成果的教学方面。最终目标是向该领域的决策者和从业人员通报改善学生学习的教学实践和教育举措。研究领域内存在各种荟萃分析。他们显示，在研究相同的实质性研究问题的研究中，效应量存在很大的偏差，这意味着复制失败。不一致，甚至部分矛盾的结果阻碍了直接的建议，并对研究结果的可信度提出质疑。偏离的原因可能是研究在以下几个方面存在差异：（a）背景，（B）设计，（c）结构的可操作性，以及（d）统计分析中使用的方法。荟萃分析和综述表明，其中一些方面影响了教学质量对学生成绩的估计效果。然而，由于荟萃分析和综述从现有研究中收集信息，它们无法保持某些方面（如统计模型）不变，因此无法理清造成异质性效应的不同方面。通过对现有数据进行系统的再分析，我们的两个主要目标是：（1）识别并精确定义可能解释教学质量影响差异的可能因素;（2）估计这些因素对结果异质性的相对重要性。在追求第一个研究目标，我们将分类数据集方面的差异（a）的设计和设置和（B）的操作结构。我们将进一步定义可能影响估计效果的方法学因素。为了实现第二个研究目标，我们将协调数据集，使用不同的方法进行二次分析，从单个分析中提取估计的效应并将其转换为相同的尺度（以效应量的形式），并估计哪些方面解释了效应中的方差比例。最后，这些发现有助于规划未来的概括性分析和概念复制，从而为从业者提供有意义的建议。