BCSER: Developing Expertise in Educational Testing to Support Characterizing the Impact of Organic Chemistry Instruction on Students' Ability to Reason with Representations

表征能力是使用绘图，图表和符号来学习，交流和解决特定领域（如有机化学）问题的能力。研究表明，本科生的推理能力与特定的表示是重要的，他们的理解和成功的有机化学。然而，目前还没有有效的评估工具来衡量学生在有机化学中的表征能力。在NSF的EHR核心研究STEM教育研究能力建设（ECR：BCSER）计划的支持下，该项目旨在开发这样一种工具。在开展该项目的过程中，主要调查员将发展教育测试和心理测量方面的能力。 这些新的能力将补充现有的专业知识，定性调查学生的化学概念的理解和教师的教学信念和课堂实践的特点。本研究将运用经典测试理论和因子分析方法，对有机化学表征能力评价（ORCA）进行开发和测试。ORCA将衡量学生的表征能力技能的有机化学表示，如破折号楔形图，纽曼预测，椅子构象的背景下。ORCA衡量的目标技能包括：（a）解释这些表征的能力，（B）为特定目的选择适当的表征的能力，（c）在这些表征之间建立联系的能力，以及（d）使用这些表征对化学现象进行预测的能力。测量质量框架，其目标是选择心理测量学证据的教育和心理测试的标准，将被用来作为评估ORCA的项目的质量和能力，以协助绘制有效和可靠的推断学生的代表能力技能的概念和方法框架。该仪器将与来自格林斯伯勒的北卡罗来纳州大学和南佛罗里达大学的有机化学学生一起进行严格的评估和测试。主要研究者将由本科化学课程内容评估领域的专家指导，并将完成课程，以提高心理测量学和测量的熟练程度。将通过与导师和咨询委员会的定期会议评估项目的成功和项目目标的实现情况。将通过在学术刊物上发表文章、在有关会议上发言以及关于评估、计量、设计和评价的讲习班来促进项目工作的传播。本研究的结果将告知教学实践和循证设计的有机化学课程，支持使用有效的策略，以发展学生的代表能力。EHR核心研究：STEM教育研究能力建设（ECR：BCSER）竞赛，旨在培养个人在STEM学习和学习环境中开展高质量、基础STEM教育研究的能力，扩大STEM领域的参与，该奖项反映了NSF的法定使命，并通过使用基金会的智力价值进行评估，被认为值得支持和更广泛的影响审查标准。

Representational competence is the ability to use drawings, diagrams, and symbols to learn, communicate, and solve problems in a specific area, such as organic chemistry. Studies have shown that undergraduate students’ ability to reason with specific representations is important for their understanding of and success in organic chemistry. However, no validated assessment instruments currently exist that measure students’ representational competence in organic chemistry. With support from NSF’s EHR Core Research Building Capacity in STEM Education Research (ECR: BCSER) program, this project seeks to develop such an instrument. In the process of conducting this project, the principal investigator will develop competencies in educational testing and psychometrics. These new competencies will complement existing expertise in qualitative investigation of students’ understanding of chemistry concepts and characterization of instructors’ teaching beliefs and classroom practices. The high-quality assessment instrument developed by the project team is expected to be useful for further research on organic chemistry learning, as well as for helping college chemistry instructors make informed decisions about their teaching.This study will use a combination of classical test theory and factor analysis methods to develop and test the Organic-chemistry Representational Competence Assessment (ORCA). ORCA will measure student representational competence skills in the context of organic chemistry representations such as dash-wedge diagrams, Newman projections, and chair conformations. The targeted skills to be measured by ORCA include: the ability to (a) interpret these representations, (b) select an appropriate representation for a particular purpose, (c) make connections across these representations, and (d) use these representations to make predictions about chemical phenomena. The measurement quality framework, which targets select psychometric evidence informed by the Standards for Educational and Psychological Testing, will be used as the conceptual and methodological framework for evaluating the quality of ORCA’s items and its capacity to assist in drawing valid and reliable inferences about student representational competence skills. The instrument will be rigorously evaluated and tested with organic chemistry students from the University of North Carolina at Greensboro and the University of South Florida. The principal investigator will be mentored by an expert in the area of assessment of content in undergraduate chemistry courses, and will complete coursework to develop proficiency in psychometrics and measurement. The success of the project and achievement of project objectives will be assessed through periodic meetings with both the mentor and the advisory board. Dissemination of project work will be facilitated via publications in academic journals, presentations at relevant conferences, and workshops on assessment measurement design and evaluation. Results of this study will inform instructional practices and evidence-based design of organic chemistry curricula that supports the use of effective strategies to develop student representational competence. This project is supported through the EHR Core Research: Building Capacity in STEM Education Research (ECR: BCSER) competition that is designed to build individuals’ capacity to carry out high-quality, fundamental STEM education research in STEM learning and learning environments, broadening participation in STEM fields, and STEM workforce development.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.