Collaborative Research: Enhancing Comparative Assessment for Chemistry with or without Standardized Testing

为了应对教职员工和院系对学生学习进行更全面评估的日益增长的需求，该项目正在进行研究并开发工具，以在本科化学课程中对学生的学习进行评估。该项目的主要目标是使化学教师能够访问学生在化学试题上的成绩数据仓库，可以查询这些数据仓库，以将其学生的成绩与数据库中所有人的成绩进行比较。这项工作正在利用化学教育中长期存在的比较评估能力，这是通过美国化学协会考试研究所制作的常模参考考试提供的，软件开发人员的专业知识导致了20万化学专业学生使用的在线网络学习(OWL)系统的创建。智力优势：进行心理测量研究并为教师开发半自动方法以使测试项目与内容标准模板保持一致，正在促进化学教育和心理测量学的重大进步。具体的项目活动包括为整个本科化学课程制作锚定概念内容图(ACCM)，创建一个允许化学教师将自己的考题与ACCM连接并动态调整的门户网站，以及为教师建立一种机制，让教师将他们自己学生在当地书写的考试项目上的表现与全国学生成绩数据集进行比较。开发必要的软件工具可以提高对几个关键领域的理解，例如测试项目元标签的体系结构、测试数据仓库的效率、用于评估开发的用户友好的查询工具以及用户在评估结果报告中有效的数据组织和可视化。广泛的影响：这项工作正在为来自所有机构类型的化学教师开发实用工具，以改进他们对学生化学学习证据的整理。随着学生成绩评估数据库的增长，数据将允许对课程和教学改革的效果进行更完整和更有意义的比较。该项目的成果可用于为所有学科的科学教育研究提供信息。

In response to an increasing need for faculty members and departments to perform more thorough assessments of student learning, this project is conducting research on and developing tools to perform assessments of student learning in undergraduate chemistry courses. The overarching goal of the project is to provide chemistry instructors access to data warehouses of student performance on chemistry test items that can be queried to compare the performance of their students to the performance of the entire population in the database. This work is leveraging long-standing capacity in comparative assessment within chemistry education, as provided through norm-referenced examinations produced by the American Chemical Society Examinations Institute, with software developers, whose expertise led to the creation of the Online Web-based Learning (OWL) system used by 200,000 students of chemistry.Intellectual Merit: Conducting psychometric research and developing semi-automated methods for instructors to align test items to a content criterion template are facilitating significant progress in both chemistry education and psychometrics. Specific project activities include producing an anchoring concepts content map (ACCM) for the entire undergraduate chemistry curriculum, creating a portal that allows chemistry instructors to connect and dynamically align their own test items to the ACCM and establishing a mechanism for instructors to compare the performances of their own students on their locally-written test items with national data sets of student performance. Developing the requisite software tools advances understanding in several key areas, such as architecture for test item meta tags, efficiency of test data warehousing, user-friendly query tools for assessment development and effectual data organization and visualization in assessment outcomes reports for users.Broader Impacts: This work is developing practical tools for chemistry instructors from all institution types to improve their marshaling of evidence with regard to student learning in chemistry. As the database of student performance on assessments grows, data will permit for more complete and meaningful comparisons of the efficacy of curricular and pedagogical reforms. Outcomes from this project can be used to inform science education research across all disciplines.