Beyond Evaluation Use: Determining the Effect of Individual Project Participation on the Influence of STEM Overall Program Evaluations

该项目的目的是对比不同的方案评价方法的类型和影响范围，以提高外地对在为具有国家和地方影响的大型机构进行评价时如何使用评价结果的理解。 该项目基于两个理论基础：1）评价使用，2）参与评价。 这些都是基于柯克哈特，格林，加里和马克，巴顿，伯克，金和劳伦斯和霍夫曼的理论工作。 评价使用的主要理论是Kirkhart的三维影响评价模型--影响源（过程或结果）、意图（有意和无意）和时间（即时、周期结束或长期）。 参与评估借鉴了Lawrenz和霍夫曼的连续体，由独立于项目内项目的实体进行的项目评估限制到项目独立确定评估程序和收集哪些数据的实体。“拟议的研究将调查参与的连续体及其与使用的关系。研究问题是：1） 在大型、多地点评价中存在哪些参与模式？2))在何种程度上，不同的项目参与程度会导致不同的评估使用模式？3)什么样的评估实践与增强评估的影响力最直接相关。将对四个NSF计划进行案例研究：ATE，LSC，CETP和MSP RETA。 在案例研究中，方法将包括人工分析，访谈，调查和现场访问。

The project is designed to contrast the type and scope of influence of different approaches to program evaluation in order to increase the field's understanding of how evaluation results are used when evaluations are conducted for a large agency with both national and local implications. The project is grounded in two theoretical bases: 1) evaluation use, and 2) participation in evaluation. These are based on the theoretical work of Kirkhart, Greene, Gary and Mark, Patton, Burke, King and Lawrenz and Huffman. The primary theory of evaluation use is Kirkhart's three-dimensional influence of evaluation model - source of influence (process orresults), intention (intended and unintended), and time (immediate, end-of-cycle, or long-term). Participation in evaluation draws on a continuum of Lawrenz and Huffman bounded by program evaluation conducted by an entity separate from the projects within the program to one when where the projects independently determine the evaluation procedures and what data to collect. `The proposed research will investigate the continuum of participation and how it relates to use.The research questions are:1) What patterns of participation exist in large, multi-site evaluation?2)) To what extent do different levels of project participation in program evaluations result in different patterns of evaluation use?3) What evaluation practices are most directly related to enhancing the influence of evaluations.Case studies will be carried out of four NSF Programs: ATE, LSC, CETP, and MSP RETAs. Within the case studies, methods will include artifact analysis, interview, surveys and site visits.