Building Evaluation System Capacity for STEM Programs: Enhancing Capability and Advancing Practice

这是一个评估系统开发项目。它是基于这样的假设，即我们对评估系统是如何开发的或如何评估自己知之甚少。鉴于目前的各种方法和途径以及评价的背景，评价要求十分复杂，情况尤其如此。建议采用结构化的参与式混合方法和系统办法：确定STEM教育者和评估者对其方案的评估需求；制定评估系统的正式模式以满足这些需求；确定该系统的主要组成部分以及它们如何协同工作；以及实施可用于评估评估系统发展情况的一项或多项评估能力建设措施。该模式和措施将在纽约市的一个试点项目中实施和测试，该项目包括正式和非正式的STEM教育项目。该项目旨在帮助我们了解评价系统的性质和结构，如何更有效地开发它们，以及如何评估它们的发展。该项目所称的“评价系统”指的是“为开展和支持评价能力而设计的各种组成部分--价值观和原则、方法、资源和结构的安排”。通常，评价系统是支持许多单独评价项目的一种方式，其明确目的往往是能够综合和比较这些不同评价的结果。该项目并不声称没有可供评估的资源。相反，这是因为有太多或质量不同的资源，没有筛选或调整以供在特定环境中使用。这不是缺乏资源，而是缺乏支持评价的制度。因此，拟议的项目旨在：开发科技教育评估系统需求的模型以及如何满足这些需求；制定可用于评估变革努力的一项或多项措施或评估能力；在两个地方科技教育和教育项目中试行能力建设工作，同时测试所制定的评估能力建设评估。项目合作伙伴已表示支持的项目包括学校(生物学师范学院和非正式科学康奈尔花园马赛克项目)、康奈尔的STEM研究小组(具有教育宣传的纳米生物技术项目、科学家和工程师的综合研究生教育和研究培训项目、科学研究伙伴关系和环境调查国际管理发展学院项目)。

This is an evaluation systems development project. It is based on the assumption that we know little about how evaluation systems are developed or can be themselves evaluated. This is especially so because of the complexity of evaluation demands given the range of current methods and approaches and contexts of evaluation. What is proposed is to use structured participatory mixed methods system approaches to: identify the needs that STEM educators and evaluators have for evaluation of their programs; develop a formal model of an evaluation system to address those needs; identify the major components of that system and how they might work together; and operationalize one or measures of evaluation capacity building that can be used to assess evaluation system development. The model and measures will be implemented and tested in a pilot project in a New York City that includes both formal and informal STEM education programs. The project is intended to contribute to our fundamental knowledge regarding the nature and structure of evaluation systems, how they can be developed more effectively and how to evaluate their development. By "evaluation system" the project means "an arrangement of various components - values and principles, methods, resources, and structures - designed to conduct and support an evaluation capability". Usually an evaluation system is a way to support many separate evaluation projects, and often with the express purpose of being able to integrate and compare the results of these distinct evaluations. The project does not claim there are no resources for evaluation. Rather it is that there are too many, or resources varying in quality and not screened or adapted for use in particular contexts. It is a lack not of resources but of systems for supporting evaluation. Thus, the proposed project is designed to: develop a model of STEM evaluation system needs and how they might be addressed; develop one or more measures or evaluation capacity that can be used to assess change efforts; and pilot test a capacity-building effort in two local STEM programs while simultaneously testing the evaluation capacity building assessments developed. Project partners have indicated support including the schools (the Biology Teachers Institute and the informal science Cornell Garden Mosaics project); STEM research groups at Cornell (nanobiotechnology project with education outreach, integrative graduate education and research traineeship project for scientists and engineers, science inquiry partnerships, and environmental inquiry IMD project.