BCSER: Developing Expertise in Educational Testing to Support Characterizing the Impact of Organic Chemistry Instruction on Students' Ability to Reason with Representations

BCSER：发展教育测试方面的专业知识，以支持表征有机化学教学对学生用表征进行推理的能力的影响

• 批准号: 2025216
• 负责人: Maia Popova
• 金额: $ 34.46万
• 依托单位: University of North Carolina Greensboro
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-10-01 至 2023-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2025216&HistoricalAwards=false
• 关键词: BCSER; Developing; Expertise; Educational; Testing

Representational competence is the ability to use drawings, diagrams, and symbols to learn, communicate, and solve problems in a specific area, such as organic chemistry. Studies have shown that undergraduate students’ ability to reason with specific representations is important for their understanding of and success in organic chemistry. However, no validated assessment instruments currently exist that measure students’ representational competence in organic chemistry. With support from NSF’s EHR Core Research Building Capacity in STEM Education Research (ECR: BCSER) program, this project seeks to develop such an instrument. In the process of conducting this project, the principal investigator will develop competencies in educational testing and psychometrics. These new competencies will complement existing expertise in qualitative investigation of students’ understanding of chemistry concepts and characterization of instructors’ teaching beliefs and classroom practices. The high-quality assessment instrument developed by the project team is expected to be useful for further research on organic chemistry learning, as well as for helping college chemistry instructors make informed decisions about their teaching.This study will use a combination of classical test theory and factor analysis methods to develop and test the Organic-chemistry Representational Competence Assessment (ORCA). ORCA will measure student representational competence skills in the context of organic chemistry representations such as dash-wedge diagrams, Newman projections, and chair conformations. The targeted skills to be measured by ORCA include: the ability to (a) interpret these representations, (b) select an appropriate representation for a particular purpose, (c) make connections across these representations, and (d) use these representations to make predictions about chemical phenomena. The measurement quality framework, which targets select psychometric evidence informed by the Standards for Educational and Psychological Testing, will be used as the conceptual and methodological framework for evaluating the quality of ORCA’s items and its capacity to assist in drawing valid and reliable inferences about student representational competence skills. The instrument will be rigorously evaluated and tested with organic chemistry students from the University of North Carolina at Greensboro and the University of South Florida. The principal investigator will be mentored by an expert in the area of assessment of content in undergraduate chemistry courses, and will complete coursework to develop proficiency in psychometrics and measurement. The success of the project and achievement of project objectives will be assessed through periodic meetings with both the mentor and the advisory board. Dissemination of project work will be facilitated via publications in academic journals, presentations at relevant conferences, and workshops on assessment measurement design and evaluation. Results of this study will inform instructional practices and evidence-based design of organic chemistry curricula that supports the use of effective strategies to develop student representational competence. This project is supported through the EHR Core Research: Building Capacity in STEM Education Research (ECR: BCSER) competition that is designed to build individuals’ capacity to carry out high-quality, fundamental STEM education research in STEM learning and learning environments, broadening participation in STEM fields, and STEM workforce development.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

表征能力是指使用图画、图表和符号来学习、交流和解决特定领域的问题的能力，例如有机化学。研究表明，本科生用具体表征进行推理的能力对于他们对有机化学的理解和成功非常重要。然而，目前还没有有效的评估工具来衡量学生在有机化学中的表征能力。在NSF的STEM教育研究的EHR核心研究能力建设计划(ECR：BCSER)的支持下，该项目试图开发这样的工具。在进行这个项目的过程中，首席调查员将发展教育测试和心理测量学方面的能力。这些新的能力将补充现有的专门知识，定性调查学生对化学概念的理解，以及对教师的教学信念和课堂实践的表征。本研究采用经典测试理论和因子分析相结合的方法，对有机化学表象能力测评(ORCA)进行了研究和测试。ORCA将在有机化学表示的背景下衡量学生的表征能力，如短划线楔形图、纽曼投影和椅子构象。ORCA要衡量的目标技能包括：(A)解释这些表示的能力，(B)为特定目的选择适当的表示，(C)在这些表示之间建立联系，以及(D)使用这些表示对化学现象进行预测。以《教育和心理测试标准》提供的心理测量学证据为目标的测量质量框架，将被用作评估ORCA项目质量及其能力的概念和方法框架，以帮助对学生的表征能力技能做出有效和可靠的推断。该仪器将与北卡罗来纳大学格林斯伯勒分校和南佛罗里达大学的有机化学学生进行严格的评估和测试。首席调查员将由本科化学课程内容评估领域的专家指导，并将完成课程工作，以提高心理测量学和测量方面的熟练程度。项目的成功和项目目标的实现将通过与导师和咨询委员会的定期会议进行评估。将通过在学术期刊上发表文章、在相关会议上发言以及举办关于评估计量设计和评价的讲习班来促进项目工作的传播。这项研究的结果将为有机化学课程的教学实践和循证设计提供参考，以支持使用有效的策略来发展学生的表征能力。该项目由EHR核心研究：STEM教育研究中的能力建设(ECR：BCSER)竞赛提供支持，该竞赛旨在培养个人在STEM学习和学习环境中开展高质量的基本STEM教育研究的能力，扩大对STEM领域的参与，以及STEM劳动力发展。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。