Collaborative Research: Enhancing Comparative Assessment for Chemistry with or without Standardized Testing

为了满足教师和部门对学生学习进行更全面评估的日益增长的需求，该项目正在进行研究和开发工具，以评估本科化学课程的学生学习。 该项目的总体目标是提供化学教师访问数据仓库的学生表现的化学测试项目，可以查询，以比较他们的学生的表现，在数据库中的整个人口的表现。 这项工作利用了化学教育中比较评估的长期能力，这种能力是通过美国化学学会考试研究所与软件开发人员一起编制的规范参照考试提供的，软件开发人员的专门知识导致创建了20万名化学学生使用的在线网络学习系统。进行心理测量学研究和开发半自动化的方法，教师调整测试项目的内容标准模板，促进化学教育和心理测量学的重大进展。 具体的项目活动包括为整个本科化学课程制作一个锚定概念内容图（ACCM），创建一个门户网站，允许化学教师将自己的测试项目与ACCM连接并动态调整，并建立一个机制，让教师将自己的学生在当地笔试项目上的表现与学生表现的国家数据集进行比较。 开发必要的软件工具可以促进对几个关键领域的理解，例如测试项目Meta标签的架构、测试数据仓库的效率、用于评估开发的用户友好型查询工具以及有效的数据组织和评估结果报告的可视化。这项工作正在为所有机构类型的化学教师开发实用工具，以改善他们对学生学习化学。 随着学生评估成绩数据库的增长，数据将允许对课程和教学改革的效果进行更全面和更有意义的比较。 该项目的成果可用于为所有学科的科学教育研究提供信息。

In response to an increasing need for faculty members and departments to perform more thorough assessments of student learning, this project is conducting research on and developing tools to perform assessments of student learning in undergraduate chemistry courses. The overarching goal of the project is to provide chemistry instructors access to data warehouses of student performance on chemistry test items that can be queried to compare the performance of their students to the performance of the entire population in the database. This work is leveraging long-standing capacity in comparative assessment within chemistry education, as provided through norm-referenced examinations produced by the American Chemical Society Examinations Institute, with software developers, whose expertise led to the creation of the Online Web-based Learning (OWL) system used by 200,000 students of chemistry.Intellectual Merit: Conducting psychometric research and developing semi-automated methods for instructors to align test items to a content criterion template are facilitating significant progress in both chemistry education and psychometrics. Specific project activities include producing an anchoring concepts content map (ACCM) for the entire undergraduate chemistry curriculum, creating a portal that allows chemistry instructors to connect and dynamically align their own test items to the ACCM and establishing a mechanism for instructors to compare the performances of their own students on their locally-written test items with national data sets of student performance. Developing the requisite software tools advances understanding in several key areas, such as architecture for test item meta tags, efficiency of test data warehousing, user-friendly query tools for assessment development and effectual data organization and visualization in assessment outcomes reports for users.Broader Impacts: This work is developing practical tools for chemistry instructors from all institution types to improve their marshaling of evidence with regard to student learning in chemistry. As the database of student performance on assessments grows, data will permit for more complete and meaningful comparisons of the efficacy of curricular and pedagogical reforms. Outcomes from this project can be used to inform science education research across all disciplines.