Collaborative Research: Enhancing Comparative Assessment for Chemistry with or without Standardized Testing

为了响应教师和院系对学生学习进行更全面评估的日益增长的需求，该项目正在研究和开发工具来评估学生在本科化学课程中的学习情况。该项目的首要目标是为化学教师提供访问学生化学测试项目表现的数据仓库的权限，可以查询这些数据仓库，以便将其学生的表现与数据库中整个人群的表现进行比较。这项工作利用了化学教育中长期以来的比较评估能力，通过美国化学学会考试协会（American Chemical Society examinations Institute）与软件开发人员（他们的专业知识促成了20万名化学学生使用的基于网络的在线学习（OWL）系统）制作的标准参考考试提供了这一能力。智力优势：进行心理测量研究和开发半自动方法，使教师能够将测试项目与内容标准模板对齐，这促进了化学教育和心理测量学的重大进步。具体的项目活动包括为整个本科化学课程制作一个锚定概念内容图（ACCM），创建一个门户网站，允许化学教师将他们自己的测试项目与ACCM连接并动态调整，并建立一个机制，让教师将他们自己的学生在当地书面测试项目上的表现与国家学生表现数据集进行比较。开发必要的软件工具可以促进对几个关键领域的理解，例如测试项目元标签的体系结构、测试数据仓库的效率、用于评估开发的用户友好查询工具以及有效的数据组织和可视化评估结果报告。更广泛的影响：这项工作是为所有机构类型的化学教师开发实用工具，以改善他们对学生化学学习的证据整理。随着学生评价成绩数据库的增长，数据将允许对课程和教学改革的效果进行更全面和有意义的比较。该项目的成果可用于为所有学科的科学教育研究提供信息。

In response to an increasing need for faculty members and departments to perform more thorough assessments of student learning, this project is conducting research on and developing tools to perform assessments of student learning in undergraduate chemistry courses. The overarching goal of the project is to provide chemistry instructors access to data warehouses of student performance on chemistry test items that can be queried to compare the performance of their students to the performance of the entire population in the database. This work is leveraging long-standing capacity in comparative assessment within chemistry education, as provided through norm-referenced examinations produced by the American Chemical Society Examinations Institute, with software developers, whose expertise led to the creation of the Online Web-based Learning (OWL) system used by 200,000 students of chemistry.Intellectual Merit: Conducting psychometric research and developing semi-automated methods for instructors to align test items to a content criterion template are facilitating significant progress in both chemistry education and psychometrics. Specific project activities include producing an anchoring concepts content map (ACCM) for the entire undergraduate chemistry curriculum, creating a portal that allows chemistry instructors to connect and dynamically align their own test items to the ACCM and establishing a mechanism for instructors to compare the performances of their own students on their locally-written test items with national data sets of student performance. Developing the requisite software tools advances understanding in several key areas, such as architecture for test item meta tags, efficiency of test data warehousing, user-friendly query tools for assessment development and effectual data organization and visualization in assessment outcomes reports for users.Broader Impacts: This work is developing practical tools for chemistry instructors from all institution types to improve their marshaling of evidence with regard to student learning in chemistry. As the database of student performance on assessments grows, data will permit for more complete and meaningful comparisons of the efficacy of curricular and pedagogical reforms. Outcomes from this project can be used to inform science education research across all disciplines.