Collaborative Research: Assessing the Longitudinal Impact of Noyce Awards on the Subject Matter Knowledge of Beginning STEM Teachers in the US: A Comparative Study

在2010年国家研究委员会的报告《教师培训：为健全的政策建立证据》中，强调了与STEM教师培训相关的领域，这些领域缺乏足够的研究基础。这个Noyce Track 4B研究项目试图解决在其研究罗伯特·Noyce教师奖学金(Noyce)资助的机构在多大程度上培养了比那些在Noyce资助周期中没有获得Noyce资助的机构更有知识的初任教师的领域之一，这是根据首次考生参加的Praxis II主题知识考试的结果来衡量的。通过石溪大学和肯纳索州立大学之间的合作研究工作，以及与教育考试服务机构(ETS)的合作，该项目将对过去15年的Praxis II考试成绩进行分类，以确定那些接受Noyce资助的机构在多大程度上培养了比没有Noyce奖项的机构更有知识的初任教师。NSF网站上的Noyce获奖者名单将用于对过去15年来来自Noyce和非Noyce机构的Praxis II考试考生进行分类。使用来自Praxis II考生的自我报告的人口统计数据，机构代码将与Noyce和非Noyce机构名单进行比较，并对每个机构的学生进行简单的统计。由于并不是所有的州都使用了Praxis II考试来获得执照，因此分析中只会涵盖Noyce投资组合的一部分。教师知识的衡量将在两个层面上进行分析，这些水平与Prxis II主题知识考试的结果有关。第一级：分析将使用各自年份考试的个别州减分。这将使人们看到，各机构是否正在为各自州预先确定的水平准备STEM教师候选人。第2级：分析Prxis II主题知识，以获得考试的原始正确分数。每次考试的平均分数将被计算出来，教师候选人将根据他们相对于平均成绩的表现被分组(即，高于平均水平1个标准差，低于平均水平1个标准差，等等)。将分析第一级和第二级所述的成果变量，以及按人口统计变量以及诺伊斯和非诺伊斯机构在业绩方面的其他差异。将回答的范例问题包括：与非诺伊斯学院相比，诺伊斯学院是否培养了更多的少数民族STEM教师候选人？根据STEM Prxis II内容知识考试，Noyce机构的女性是否比非Noyce机构的女性表现更好？与非诺伊斯学院的同龄人相比，诺伊斯学院第一次参加Praxis II考试的非传统学生人数是否更多？为了建立Noyce对不同院校影响的模型，在回归分析中将使用平均SAT成绩作为协变量，以控制机构的选择性。在描述性分析中，只会使用入学要求和SAT/ACT平均分数相近的学校。为了衡量在一所大学同时拥有诺伊斯学者和非诺伊斯学者的情况下，诺伊斯资助对学者质量的影响，将使用关于每个Praxis II STEM领域的考生人数、他们的平均GPA以及他们在诺伊斯资助之前是否有现场专业的数据来建立教师教育毕业生的数量和类型的基线。这将允许对后续数据进行分析，以确定这些模式在获得Noyce资金后是否发生了变化。一项实证性研究将仅限于诺伊斯学校的一个小样本，以考察诺伊斯毕业生占特定年份STEM教育毕业生总数的比例的影响。它将被作为一个系数纳入回归模型，根据比例大小调整诺伊斯机构之间的平均值比较。如果有证据表明该比率与院校层面的平均Prxis分数有关，则将在其他学校收集这一比率，并将其纳入Noyce和非Noyce院校的平均分数的比较中。最后，这项研究将考察诺伊斯计划对STEM初任教师在进入该专业时的教学知识深度的影响。

In the 2010 National Research Council's report, "Preparing Teachers: Building Evidence for Sound Policy," areas related to STEM teacher preparation that lacked a sufficient research base were highlighted. This Noyce Track 4B research project seeks to address one of the areas identified as it studies the extent to which those institutions receiving Robert Noyce Teacher Scholarship (Noyce) funding from the National Science Foundation have prepared more knowledgeable beginning teachers than those institutions which have not received Noyce grants during Noyce funding cycles as measured by outcomes on Praxis II subject matter knowledge exams given to first time test takers. Through a collaborative research effort between Stony Brook University and Kennesaw State University, and working with the Educational Testing Service (ETS), the project will undertake the disaggregation of Praxis II test scores over the past fifteen years to determine to what extent those institutions receiving Noyce funding have prepared more knowledgeable beginning teachers than those without Noyce awards. The list of Noyce awardees from the NSF website will be used to categorize Praxis II test takers coming from Noyce and non-Noyce institutions over the past fifteen years. Using the self-reported demographic data from Praxis II test takers, the institutional codes will be compared to the Noyce and non-Noyce institution list and a simple count of students at each institution will be performed. As not all states have used Praxis II exams for licensure, only a subset of the Noyce portfolio will be captured in the analysis. The measure of teacher knowledge will be analyzed at two levels related to outcomes on the Praxis II subject matter knowledge exams. Level 1: analysis will use individual state cut scores for the respective year exams. This will provide a look at whether institutions are preparing STEM teacher candidates for a level pre-determined by their respective states. Level 2: analysis of Praxis II subject matter knowledge for a raw percent correct score on the exam. An average exam score will be calculated for each exam and teacher candidates will be placed into groups based on their performance relative to the mean (i.e., 1 standard deviation above the mean, 1 standard deviation below the mean, etc.). The outcome variables described in Level 1 and Level 2 will be analyzed as well as additional differences in performance by demographic variables for the total population and also by Noyce and non-Noyce institutions. Example questions that will be answered include: Do Noyce institutions prepare more minority STEM teacher candidates compared to non-Noyce institutions? Do women at Noyce institutions outperform women at non-Noyce institutions based on STEM Praxis II content knowledge exams? Do Noyce institutions have a higher number of non-traditional first-time Praxis II test takers compared to their non-Noyce peers? In order to develop a model for impact of Noyce on different institutions, average incoming SAT scores will be used as a covariate in the regression analysis that will control for selectivity of the institution. In the descriptive analysis, only schools with similar entrance requirements and average SAT/ACT scores to each other will be used. In order to measure the impact of the Noyce funding related to the quality of the scholars in situations where a university has had both Noyce scholars and non-Noyce scholars, data on the number of test takers in each Praxis II STEM field, their average GPA, and whether or not they have in-field majors prior to Noyce funding will be used to establish a baseline for number and type of teacher education graduates. This will allow for analysis of subsequent data to determine whether these patterns changed after receiving Noyce funding. An empirical study will be conducted limited to a subsample of the NOYCE schools to examine the effect of the proportion of Noyce graduates to total STEM education graduates in the given year. It will be included as a coefficient in the regression models to adjust the comparison of averages between the Noyce institutions based on the size of the proportion. If there is evidence that the ratio is related to the average PRAXIS scores at the institution level, then it will be collected on additional schools and incorporated into the comparison of averages between Noyce and non-Noyce institutions. Ultimately, this study will examine the influence of the Noyce program on the depth of content knowledge for beginning STEM teachers as they enter the profession.