Developing a Next Generation Concept Inventory to Help Environmental Programs Evaluate Student Knowledge of Complex Food-Energy-Water Systems

本项目旨在通过改善本科环境专业的教学实践，为国家利益服务。为此，该项目将开发并测试一份书面回应概念清单，以衡量本科生对复杂食物-能源-水系统的了解。传统的概念量表被广泛应用于许多学科，以衡量学生对该学科关键思想的理解。这些概念清单使用选择题的答案来衡量学生的知识。然而，从要求学生用自己的话来展示他们的知识的问题中，可以更深入地了解学生的思维。该项目计划开发一个“下一代概念清单”，该清单将使用简答问题，并对学生对跨学科、系统级概念问题的回答进行自动评分。这个概念清单的可用性有望帮助教师评估学生对复杂环境概念的理解，以及评估和比较项目内部和项目之间的学习。反过来，这些信息可以用来提高环境项目的教学效率，并可以作为其他学科类似方法的典范。为了更好地为教授、课程决策和管理大学环境项目的人员提供信息，该项目将开发一个评估工具，以衡量环境专业学生的基础知识和对复杂系统级概念的理解。具体而言，该项目将应用已建立的机器学习方法来评估构建的回答（简短回答）问题，以创建下一代概念清单。这种概念清单构建方法将创建一套新的构建反应项目和相关的自动评分模型，重点关注复杂的食物-能源-水关联系统，这是环境项目中典型的主题。该项目将评估概念清单的有效性和可靠性，以确保它能够提供高质量的信息，供环境教育工作者和项目管理者使用，并用于STEM教育和评估的研究。项目团队将首先确定共同的食物-能源-水关系概念，并通过检查全国各地环境项目的入门课程材料来学习成果。针对这些概念/结果的下一代概念清单的开发将遵循既定的概念清单构建方法，包括使用与入门环境项目学生的访谈来揭示学生的先入之见，并开发构建的回答问题。通过审查美国各地的环境课程收集到的信息，结合下一代概念清单对学生学习情况的测量，可以为大学环境科学和研究项目的课程设置和人员配备决策提供信息。因此，该项目通过提供一个有效可靠的工具来评估学生的学习、课程成果和项目有效性，有可能使环境项目的教师和学生受益。NSF IUSE: EHR计划支持研究和开发项目，以提高所有学生STEM教育的有效性。通过参与学生学习轨道，该计划支持有前途的实践和工具的创建，探索和实施。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project aims to serve the national interest by improving teaching practices in undergraduate environmental programs. To do so, the project will develop and test a written response concept inventory to measure undergraduate students’ knowledge of complex Food-Energy-Water systems. Traditional concept inventories are widely used in many disciplines to measure student understanding of key ideas in the discipline. These concept inventories use answers to multiple choice questions to measure student knowledge. However, greater insight into student thinking can be gained from questions that require students to use their own words to demonstrate their knowledge. This project plans to develop a “Next Generation Concept Inventory” that will use short answer questions and automated scoring of student responses to questions about interdisciplinary, systems-level concepts. The availability of this concept inventory is expected to help faculty evaluate their students’ understanding of complex environmental concepts, as well as evaluate and compare learning within and between programs. This information, in turn, can be used to improve teaching effectiveness within environmental programs and may serve as a model for similar approaches in other disciplines.To better inform those who teach, make curricular decisions, and manage college-level environmental programs, the project will develop an assessment tool to measure environmental students’ foundational knowledge and understanding of complex systems-level concepts. Specifically, the project will apply an established machine learning method of evaluating constructed response (short answer) questions to create a Next Generation Concept Inventory. This approach to concept inventory construction will create a new set of constructed-response items and associated automated scoring models focused on complex Food-Energy-Water Nexus systems, a topic typically addressed in environmental programs. The project will evaluate the concept inventory’s validity and reliability to ensure that it will provide high-quality information that can be used by environmental educators and program administrators, as well as for research on STEM education and assessment. The project team will first determine common Food-Energy-Water Nexus concepts and learning outcomes through examination of introductory course materials from environmental programs across the nation. Development of the Next Generation Concept Inventory targeting these concepts/outcomes will follow established methods for concept inventory construction, including the use of interviews with introductory environmental program students to reveal student preconceptions and to develop the constructed response questions. The information gleaned from reviewing environmental curricula across the United States, combined with measurements of student learning from the Next Generation Concept Inventory, can inform curricular and staffing decisions regarding college environmental science and studies programs. Thus, the project has the potential to benefit faculty and students in environmental programs by providing a valid and reliable instrument for evaluating student learning, course outcomes, and program effectiveness. The NSF IUSE: EHR Program supports research and development projects to improve the effectiveness of STEM education for all students. Through the Engaged Student Learning track, the program supports the creation, exploration, and implementation of promising practices and tools.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.