Development of Evaluation Plan, Tools, and Protocols for the Sacramento State RISE Program

总结 萨克拉门托州RISE计划（RISE）团队建议与萨克拉门托州合作 社会研究所（ISR）将建立一个评估基础设施， 目标：（1）开发测量工具、流程和长期评估战略 可以由RISE工作人员在此补助金结束后可持续地实施，以及（2）调整 并整合其他联邦资助的针对UR的项目的评估活动 在萨克拉门托州立大学就读STEM专业的学生， 学习可以在这些项目中共享和利用。 第一，将为提高妇女地位战略制定一项全面的评价计划， 和结果的措施，其中RISE工作人员可以每年实施。通知 根据评估计划，ISR将审查当前的RISE数据收集工作以及其他工具和数据 由校园联邦资助的项目收集，支持UR学生在STEM学科。 评估计划将包括详细的RISE项目逻辑模型、形成和结果 指标、测量工具、数据收集时间轴和数据分析计划。一 全面的评价计划将确保所有数据收集活动得到整合， 与RISE计划的学习目标和预期成果保持一致。纳入 评估计划包括以下确定的活动。 定性形成性评估发展。将制定焦点小组协议， 收集RISE培训生对他们的计划体验的看法，包括培训生的原因 攻读博士学位以及RISE计划如何帮助实现这一目标。结果将 确定该计划目前未解决的任何培训生需求或差距，以及如何 他们在RISE的经验与他们可能参与的其他UR支持计划交叉。 定量形成性评估发展。ISR将开展学生调查， 衡量学生的看法和短期成果，纳入类似的评估尺度 其他校园URM/STEM项目使用。1，2将对草稿调查工具进行实地测试。ISR将 在SPSS中开发代码手册和基本分析语法，以自动化调查结果报告。 设计明矾数据收集协议。ISR将制定一项方案，对所有RISE校友进行计数 以及所有萨克拉门托州立大学生物医学研究专业的毕业生， 程序.这包括获得化学，生物化学， 微生物学、细胞与分子生物学和普通生物学校友来自萨克拉门托州 机构研究办公室，然后以批处理方式搜索学生信息交换所。 ISR将制定协议，为批量处理准备学生记录数据，然后 处理国家安全委员会的数据进行分析。RISE工作人员每年可以使用这些协议来跟踪 实现战略目标和可衡量的目标。 ISR还将为RISE校友和所有萨克拉门托州立大学校友开展一项调查， 到生物医学研究，询问他们的职业和毕业后的教育轨迹。 这项调查将衡量具体目标和可衡量的目标， 在其他联邦资助的项目中，纵向衡量可比的关键成果 并提供非UR STEM专业的比较样本。 ISR将与RISE团队密切合作，以确保实现该赠款的目标。为 例如，将共同制定评价计划，以确保 RISE评估基础设施已经到位，从数据收集工具到分析计划。年底 在实施年度评估周期后，ISR将与RISE进行面谈 工作人员确保初步评估计划可行、有意义和可持续， ISR的持续支持。

Summary The Sacramento State RISE Program (RISE) team proposes to work with the Sacramento State Institute for Social Research (ISR) to build an evaluative infrastructure with two primary objectives: (1) to develop measurement tools, processes, and a long-term evaluation strategy that can be sustainably implemented by RISE staff after the end of this grant, and (2) to align and integrate the evaluation activities from other federally-funded programs aimed at UR students enrolled in STEM majors at Sacramento State so the measurement tools, data, and learning can be shared and leveraged across these programs. First, a comprehensive evaluation plan will be developed for RISE, incorporating both formative and outcome measures, which RISE staff can then implement on an annual basis. To inform the evaluation plan, ISR will review current RISE data collection efforts as well as other tools and data collected by on-campus federally-funded programs supporting UR students in STEM disciplines. The evaluation plan will include a detailed RISE program logic model, formative and outcome indicators, measurement instruments, a data collection timeline, and a data analysis plan. A comprehensive evaluation plan will ensure that all data collection activities are integrated and aligned with learning objectives and intended outcomes of the RISE program. Included in the evaluation plan are the following identified activities. Qualitative Formative Assessment Development. Focus group protocols will be developed to collect RISE Trainee perspectives on their program experiences, including Trainees’ reasons for pursuing a Ph.D. and how the RISE Program has helped toward that goal. Results will be used to identify any Trainee needs or gaps the program currently does not address, as well as how their experience in RISE intersects with other UR support programs they may also participate in. Quantitative Formative Assessment Development. ISR will develop a student survey in which to measure student perceptions and short-term outcomes, incorporating similar assessment scales used by other campus URM/STEM programs.1,2 A draft survey tool will be field tested. ISR will develop a code-book and basic analysis syntax in SPSS to automate survey results reporting. Designing Alum Data Collection Protocols. ISR will develop a protocol to count all RISE alums and all Sacramento State alums of majors related to biomedical research who go on to Ph.D. programs. This includes obtaining names and birth dates of Chemistry, Biochemistry, Microbiology, Cell & Molecular Biology, and General Biology alums from the Sacramento State Office of Institutional Research, then searching the Student Clearinghouse in a batch method. ISR will develop protocols to prepare the student record data for batch processing and then process the NSC data for analysis. These protocols can be used by RISE staff annually to track progress towards Strategic Aims and Measurable Objectives. ISR will also develop a survey for RISE alums and all Sacramento State alums of majors related to biomedical research, asking about their careers and educational trajectories post-graduation. This survey will measure Specific Aims and Measurable Objectives, as well as be leveraged across other federally-funded programs to measure comparable key outcomes longitudinally and to provide a comparison sample of non-UR STEM majors. ISR will work closely with the RISE team to ensure the objectives of this grant are achieved. For example, the evaluation plan will be co-developed in a way in which to ensure all elements of RISE evaluation infrastructure is in place, from data collection tools to analysis plans. At the end of the grant year and after implementing an annual evaluation cycle, ISR will interview RISE staff to ensure that the initial evaluation plan is feasible, meaningful, and sustainable without ongoing support from ISR.