Building Evaluation System Capacity for STEM Programs: Enhancing Capability and Advancing Practice

这是一个评价系统开发项目。它是基于这样一种假设，即我们对评价系统是如何发展的或如何对其本身进行评价知之甚少。鉴于目前评价的方法和做法以及背景的多样性，评价需求的复杂性尤其如此。建议使用结构化参与式混合方法系统方法：确定STEM教育工作者和评估人员对其计划进行评估的需求;开发评估系统的正式模型以满足这些需求;确定该系统的主要组成部分以及它们如何协同工作;并实施一项或多项评估能力建设措施，以评估评估系统的发展。该模型和措施将在纽约市的一个试点项目中实施和测试，该项目包括正式和非正式的STEM教育项目。该项目旨在增进我们对评价系统的性质和结构、如何更有效地发展这些系统以及如何评价其发展的基本知识。就“评价系统”而言，该项目是指“各种组成部分的安排-价值观和原则、方法、资源和结构-旨在进行和支持评价能力”。通常，评价系统是支持许多单独评价项目的一种方式，其明确目的往往是能够综合和比较这些不同评价的结果。该项目并没有声称没有资源进行评价。相反，这是因为资源太多，或者资源质量参差不齐，没有经过筛选或调整，以供在特定情况下使用。这不是缺乏资源，而是缺乏支持评价的系统。因此，拟议项目的目的是：开发一个STEM评估系统需求模型以及如何解决这些需求;开发一个或多个可用于评估变革工作的措施或评估能力;在两个地方STEM项目中对能力建设工作进行试点测试，同时测试开发的评估能力建设评估。项目合作伙伴已表示支持，包括学校（生物教师研究所和非正式科学研究所）;康奈尔大学的STEM研究小组（纳米生物技术教育推广项目、科学家和工程师的综合研究生教育和研究实习项目、科学探究伙伴关系和环境探究IMD项目）。

This is an evaluation systems development project. It is based on the assumption that we know little about how evaluation systems are developed or can be themselves evaluated. This is especially so because of the complexity of evaluation demands given the range of current methods and approaches and contexts of evaluation. What is proposed is to use structured participatory mixed methods system approaches to: identify the needs that STEM educators and evaluators have for evaluation of their programs; develop a formal model of an evaluation system to address those needs; identify the major components of that system and how they might work together; and operationalize one or measures of evaluation capacity building that can be used to assess evaluation system development. The model and measures will be implemented and tested in a pilot project in a New York City that includes both formal and informal STEM education programs. The project is intended to contribute to our fundamental knowledge regarding the nature and structure of evaluation systems, how they can be developed more effectively and how to evaluate their development. By "evaluation system" the project means "an arrangement of various components - values and principles, methods, resources, and structures - designed to conduct and support an evaluation capability". Usually an evaluation system is a way to support many separate evaluation projects, and often with the express purpose of being able to integrate and compare the results of these distinct evaluations. The project does not claim there are no resources for evaluation. Rather it is that there are too many, or resources varying in quality and not screened or adapted for use in particular contexts. It is a lack not of resources but of systems for supporting evaluation. Thus, the proposed project is designed to: develop a model of STEM evaluation system needs and how they might be addressed; develop one or more measures or evaluation capacity that can be used to assess change efforts; and pilot test a capacity-building effort in two local STEM programs while simultaneously testing the evaluation capacity building assessments developed. Project partners have indicated support including the schools (the Biology Teachers Institute and the informal science Cornell Garden Mosaics project); STEM research groups at Cornell (nanobiotechnology project with education outreach, integrative graduate education and research traineeship project for scientists and engineers, science inquiry partnerships, and environmental inquiry IMD project.