Lessons-Learned from GSE Extension Services Grantees, 2005-2009

智力优势：美国科学促进会（AAAS）推进科学工程能力中心将审查NSF科学与工程性别研究（GSE）计划的推广服务（EXT）部分的经验。审查的目的是汇编自2005年以来所有项目组的经验教训，并评价预期概念如何转化为实地经验。推广服务受赠者在五年内获得高达250万美元的奖励，这是一笔相对较小的项目预算的巨额投资。最初资助的两个项目已经过了三年半。共有七个项目代表了该领域超过17年的经验，测试了这一特定投资的概念（每个项目为期五年）EXT项目参与了改变教育价值观、实践和知识的过程，这一过程既不是线性的，也不是可预测的。评估新举措成功与否的模式从简短和权宜的情况介绍会到正式和全面的成果评价不等。作为新的倡议，EXT缺乏衡量成功的标准模板或控制。定期评估经验教训是这样做的一个方法。拟议的工作是探索性的结合方法的评价研究（正式的成果测量），系统分析（审查和文件的组织过程和功能，以一致的方式），管理审查（领导人描述他们的经验），新闻报道（有趣的故事和事实），和人类学（人们如何解释他们在做什么）。该方法本质上是新颖的，跨学科的，务实的。关键的投入包括与项目经理的访谈、项目文件、通过第三年反向现场访问收集的报告和评论、年度计划公告中阐述的概念分析以及与NSF计划主任的访谈。 与受赠人的访谈将采用访谈指南方法（根据凯洛格基金会评估手册）。目标是概述EXT团队如何在项目主任的指导下解释招标，他们如何设计推广服务的版本，以及当他们面对实施的现实时发生了什么。采访将由美国科学促进会能力中心的工作人员进行，他们在上述方法中有多种方法的经验，特别是评估理论和方法，他们熟悉NSF的目标和运作，GSE计划，实践社区和与SE中妇女问题有关的知识库，并且亲自为大学，公司和联邦客户制作了许多类似的报告。更广泛的影响：有关GSE推广服务的经验教训将有可能使该计划能够完善招标，指导新团队在组织服务方面采取富有成效的策略，并接触目前未参与的各种社区。在某种程度上，某些EXT活动和关系被认为是有希望的，它们可能会被改编成其他地方。经验教训不能替代严格的正式项目或计划评估;它在真实的时间里是权宜之计和务实的，阐明了对他人和NSF有用的“现场经验”，随着这些大型项目的发展，捕获了重要的经验。综合起来，这些经验教训构成了一个项目组合评估，可以指导当前和未来的执行者以及NSF管理。

Intellectual Merit: The American Association for the Advancement of Science (AAAS) Center for Advancing Science & Engineering Capacity will review experience with the Extension Services (EXT) component of the NSF Research on Gender in Science and Engineering (GSE) program. The purpose of the review is to compile lessons-learned among all project cohorts since 2005 and evaluate how the intended concept translated into field experience. Extension Services grantees are awarded up to $2.5 million over five years, a large investment for a relatively small program budget. The first two projects funded have passed a three and a half-year mark. Collectively seven projects represent over 17 years of experience in the field, testing the concept of this particular investment (each for five years).EXT projects are engaged in the process of changing educational values, practice, and knowledge, a process that is neither linear nor predictable. Models for assessing the success of new initiatives range from short and expedient, e.g. briefings, to formal, comprehensive evaluation of outcomes. As new initiatives, EXT's lack a standard template or control for gauging success. Periodically assessing lessons-learned is one approach for doing so. The proposed work is exploratory in combining methods of evaluation research (formal outcomes measurement), systems analysis (review and documentation of organizational processes and functions in a consistent manner), management review (leaders describe their experience), journalistic reporting (interesting stories and facts), and anthropology (how people explain what they are doing in situ). The method is inherently novel, interdisciplinary, and pragmatic. Among key inputs are interviews with project managers, project documentation, reports and reviews gathered via the third-year reverse site visit, analysis of the concept as articulated in annual program announcements, and interviews with the NSF Program Director. Interviews with grantees will employ an Interview Guide Approach (per the Kellogg Foundation Evaluation Handbook). The goal is to develop an overview of how the lead EXT teams interpreted the solicitation with guidance from the Program Director, how they designed their version of an Extension Service, and what happened when they confronted the realities of implementation. Interviews will be conducted by staff of the AAAS Capacity Center, who are experienced in multiple methods among those cited above, especially evaluation theory and methods, and who are familiar with NSF's objectives and operations, the GSE program, the community of practice and the knowledge base related to issues of women in S&E, and who have personally produced many similar reports for university, corporate, and federal clients. Broader Impacts: Lessons-learned about GSE's Extension Services will potentially enable the program to refine the solicitation, guiding new teams toward productive strategies in organizing their services and reaching various communities not currently participating. To the extent certain kinds of EXT events and relationships are found to be promising, they might be adapted elsewhere. Lessons-learned is not a substitute for rigorous formal project or program evaluation; it is expedient and pragmatic in real time, elucidating what is useful to others and NSF as "field experience," capturing significant experience as these large-scale projects evolve. Taken together, these lessons constitute a portfolio assessment of projects that can guide current and future performers as well as NSF management.