Collaborative Research: Building Assessment Capacity in Chemistry Education - The Chemistry Instrument Review and Assessment Library Project

该项目旨在通过促进采用以证据为基础的教学评估工具，为高质量的本科化学教育服务于国家利益。要了解教育倡议对学生学习的影响，需要来自与兴趣结果相一致的有效评估工具的数据。本项目旨在帮助化学教师和化学教育研究人员选择适合课堂使用的有效工具。具体地说，该项目将开发和托管一份供化学教育界使用的评估工具目录。该目录将以用户友好的格式提供，并托管在2012年成立的美国化学协会社区资源化学教育交换网站上。为了进一步支持选择高质量的工具，该目录还将：1)提供关于使用该工具收集的数据的有效性和可靠性的信息；2)编制与有效性和可靠性有关的概念和方法的词汇表；3)包括关于每项评估的现有证据的专家审查。通过这项工作，化学仪器审查和评估图书馆(手性)平台将被开发为化学教育界的中心资源。为了实现其目标，项目小组将系统地审查现有文献，以确定化学教育中使用的评估工具。一个专家小组将编写一份关于每项工具的总结报告，以帮助用户根据特定情况选择最合适的评估工具。除了基本的元数据，如乐器的主题和格式，手性平台中的每个乐器都将包括关于乐器有效性和可靠性的证据的概述。这些信息将包括开发人员和使用该仪器的其他研究的结果。此外，摘要和概览中包括的有效性和可靠性的关键原则将与动态词汇表建立超级链接。本词汇表将总结和参考现有的质量资源，这些资源以标准化的格式解释常见的心理测量学术语和方法，从基础主题到高级主题。为了支持手性平台对化学教育界的价值，用户将有多种方式对其功能提供反馈。手性平台收集和呈现的信息被预测为改善用户在评估和实施化学教育评估中的决策过程。为了衡量这种影响的程度，将进行一项定性评估研究，以衡量教师对假设工具的看法，并为其数据的有效性和可靠性提供不同程度的证据。在化学教育界，目录、同行评议过程和词汇表的结合有可能增加学习评估过程的严谨性，提高学术工作在仪器开发中的价值，并减少教育工作者和研究人员致力于根据其需要定位或制定适当评估的时间。因此，该项目有望加强课堂教学和学生在化学学习方面的教育研究。该项目得到了美国国家科学基金会改善本科生STEM教育计划：教育和人力资源(IUSE：EHR)的支持，该计划支持旨在提高所有学生STEM教育有效性的研究和开发项目。通过参与的学生学习路径，该计划支持有前景的实践和工具的创建、探索和实施。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This project aims to serve the national interest in high-quality undergraduate chemistry education by promoting the adoption of evidence-based tools for assessing instruction. Understanding the impact of an educational initiative on student learning requires data from valid assessment instruments that are aligned to the outcome of interest. This project aims to help chemistry instructors and chemistry education researchers select valid instruments for classroom use. Specifically, the project will develop and host a catalog of assessment instruments used by the chemistry education community. The catalog will be available in a user-friendly format and hosted on the Chemical Education Xchange, an American Chemical Society community resource established in 2012. To further support the selection of high-quality instruments, this catalog will also: 1) provide information about the validity and reliability of data collected using the instrument; 2) develop a glossary of the concepts and methods related to validity and reliability; and 3) include expert reviews regarding the available evidence for each assessment. Through this work, the CHemistry Instrument Review and Assessment Library (CHIRAL) platform will be developed as a central resource for the chemical education community. To meet its goals, the project team will systematically review available literature to identify assessment instruments that are used in chemistry education. A panel of experts will write a summary report about each instrument, to aid users in selecting the most appropriate assessment instrument for a given context. Along with basic metadata, such as the topic and format of an instrument, each instrument in the CHIRAL platform will include an overview of the evidence regarding the instrument's validity and reliability. This information will include results from the developers and from other studies that have used the instrument. Additionally, key principles of validity and reliability included in the summary and overview will be hyperlinked to a dynamic glossary. This glossary will summarize and reference existing quality resources that explain common psychometric terminology and methods in a standardized format that scaffolds from basic to advanced topics. To support the value of the CHIRAL platform to the chemistry education community, users will have multiple ways to provide feedback on its features. The information gathered and presented by the CHIRAL platform is predicted to improve users' decision-making processes in evaluating and implementing educational assessments in chemistry. To measure the extent of this impact, a qualitative evaluation study will be conducted to gauge faculty's perspectives of hypothetical instruments supported with varying levels of evidence for the validity and reliability of their data. Within the chemistry education community, the combination of the catalog, peer-review process, and the glossary have the potential to increase the rigor of the learning assessment process, enhance the value of scholarly work in instrument development, and reduce the time educators and researchers dedicate to locating or developing appropriate assessments for their needs. As a result, this project is expected to enhance both instruction at the classroom level and educational research on student learning in chemistry. This project is supported by the NSF Improving Undergraduate STEM Education Program: Education and Human Resources (IUSE: EHR), which supports research and development projects to improve the effectiveness of STEM education for all students. Through the Engaged Student Learning track, the program supports the creation, exploration, and implementation of promising practices and tools.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.