Development of Evaluation Plan, Tools, and Protocols for the Sacramento State RISE Program

点击翻译按钮获取中文摘要

Summary The Sacramento State RISE Program (RISE) team proposes to work with the Sacramento State Institute for Social Research (ISR) to build an evaluative infrastructure with two primary objectives: (1) to develop measurement tools, processes, and a long-term evaluation strategy that can be sustainably implemented by RISE staff after the end of this grant, and (2) to align and integrate the evaluation activities from other federally-funded programs aimed at UR students enrolled in STEM majors at Sacramento State so the measurement tools, data, and learning can be shared and leveraged across these programs. First, a comprehensive evaluation plan will be developed for RISE, incorporating both formative and outcome measures, which RISE staff can then implement on an annual basis. To inform the evaluation plan, ISR will review current RISE data collection efforts as well as other tools and data collected by on-campus federally-funded programs supporting UR students in STEM disciplines. The evaluation plan will include a detailed RISE program logic model, formative and outcome indicators, measurement instruments, a data collection timeline, and a data analysis plan. A comprehensive evaluation plan will ensure that all data collection activities are integrated and aligned with learning objectives and intended outcomes of the RISE program. Included in the evaluation plan are the following identified activities. Qualitative Formative Assessment Development. Focus group protocols will be developed to collect RISE Trainee perspectives on their program experiences, including Trainees’ reasons for pursuing a Ph.D. and how the RISE Program has helped toward that goal. Results will be used to identify any Trainee needs or gaps the program currently does not address, as well as how their experience in RISE intersects with other UR support programs they may also participate in. Quantitative Formative Assessment Development. ISR will develop a student survey in which to measure student perceptions and short-term outcomes, incorporating similar assessment scales used by other campus URM/STEM programs.1,2 A draft survey tool will be field tested. ISR will develop a code-book and basic analysis syntax in SPSS to automate survey results reporting. Designing Alum Data Collection Protocols. ISR will develop a protocol to count all RISE alums and all Sacramento State alums of majors related to biomedical research who go on to Ph.D. programs. This includes obtaining names and birth dates of Chemistry, Biochemistry, Microbiology, Cell & Molecular Biology, and General Biology alums from the Sacramento State Office of Institutional Research, then searching the Student Clearinghouse in a batch method. ISR will develop protocols to prepare the student record data for batch processing and then process the NSC data for analysis. These protocols can be used by RISE staff annually to track progress towards Strategic Aims and Measurable Objectives. ISR will also develop a survey for RISE alums and all Sacramento State alums of majors related to biomedical research, asking about their careers and educational trajectories post-graduation. This survey will measure Specific Aims and Measurable Objectives, as well as be leveraged across other federally-funded programs to measure comparable key outcomes longitudinally and to provide a comparison sample of non-UR STEM majors. ISR will work closely with the RISE team to ensure the objectives of this grant are achieved. For example, the evaluation plan will be co-developed in a way in which to ensure all elements of RISE evaluation infrastructure is in place, from data collection tools to analysis plans. At the end of the grant year and after implementing an annual evaluation cycle, ISR will interview RISE staff to ensure that the initial evaluation plan is feasible, meaningful, and sustainable without ongoing support from ISR.