Beyond Evaluation Use: Determining the Effect of Individual Project Participation on the Influence of STEM Overall Program Evaluations

该项目的目的是对比不同方案评价方法的类型和影响范围，以便增加实地对在为具有国家和地方影响的大型机构进行评价时如何使用评价结果的理解。本项目建立在两个理论基础之上：1)评价使用，2)评价参与。这些都是基于Kirkhart， Greene, Gary and Mark, Patton, Burke， King and Lawrenz和Huffman的理论工作。评价使用的主要理论是Kirkhart的三维影响评价模型——影响来源（过程或结果）、意图（有意的和无意的）和时间（即时的、周期结束的或长期的）。参与评估借鉴了Lawrenz和Huffman的连续统一体，该统一体由一个独立于项目内项目的实体进行项目评估，当项目独立决定评估程序和收集什么数据时。“拟议的研究将调查参与的连续性以及它与使用的关系。研究的问题是：1)在大型、多站点的评价中存在什么样的参与模式？2)不同程度的项目参与计划评估会在多大程度上导致不同的评估使用模式？3)哪些评价实践与提高评价的影响力最直接相关。案例研究将在四个NSF项目中进行：ATE、LSC、CETP和MSP RETAs。在案例研究中，方法将包括人工制品分析、访谈、调查和现场访问。

The project is designed to contrast the type and scope of influence of different approaches to program evaluation in order to increase the field's understanding of how evaluation results are used when evaluations are conducted for a large agency with both national and local implications. The project is grounded in two theoretical bases: 1) evaluation use, and 2) participation in evaluation. These are based on the theoretical work of Kirkhart, Greene, Gary and Mark, Patton, Burke, King and Lawrenz and Huffman. The primary theory of evaluation use is Kirkhart's three-dimensional influence of evaluation model - source of influence (process orresults), intention (intended and unintended), and time (immediate, end-of-cycle, or long-term). Participation in evaluation draws on a continuum of Lawrenz and Huffman bounded by program evaluation conducted by an entity separate from the projects within the program to one when where the projects independently determine the evaluation procedures and what data to collect. `The proposed research will investigate the continuum of participation and how it relates to use.The research questions are:1) What patterns of participation exist in large, multi-site evaluation?2)) To what extent do different levels of project participation in program evaluations result in different patterns of evaluation use?3) What evaluation practices are most directly related to enhancing the influence of evaluations.Case studies will be carried out of four NSF Programs: ATE, LSC, CETP, and MSP RETAs. Within the case studies, methods will include artifact analysis, interview, surveys and site visits.