Developing a Next Generation Concept Inventory to Help Environmental Programs Evaluate Student Knowledge of Complex Food-Energy-Water Systems

该项目旨在通过改进本科环境专业的教学实践来服务于国家利益。为此，该项目将开发和测试书面回应概念清单，以衡量本科生对复杂的食物-能量-水系统的知识。传统的概念量表在许多学科中被广泛使用，以衡量学生对该学科中关键思想的理解。这些概念清单使用多项选择题的答案来衡量学生的知识。然而，从要求学生用自己的语言来证明他们的知识的问题中，可以更好地洞察学生的思维。这个项目计划开发一个“下一代概念清单”，它将使用简答题和学生对跨学科、系统级概念问题的回答的自动评分。这份概念清单的可用性预计将帮助教师评估他们的学生对复杂环境概念的理解，以及评估和比较项目内和项目之间的学习。这些信息反过来可以用来提高环境专业的教学效率，也可以作为其他学科类似方法的典范。为了更好地为那些教授、制定课程决策和管理大学水平的环境课程的人提供信息，该项目将开发一种评估工具，以衡量环境学生的基础知识和对复杂系统级概念的理解。具体地说，该项目将应用一种已建立的评估构造回答(简答)问题的机器学习方法，以创建下一代概念清单。这种构建概念库存的方法将创建一套新的构建响应项目和相关的自动评分模型，专注于复杂的食品-能源-水关联系统，这是环境项目中通常涉及的一个主题。该项目将评估概念清单的有效性和可靠性，以确保它将提供高质量的信息，可供环境教育工作者和项目管理人员使用，以及用于STEM教育和评估的研究。该项目团队将首先通过检查全国环境项目的入门课程材料，确定常见的食物-能源-水关联概念和学习结果。针对这些概念/结果的下一代概念清单的开发将遵循构建概念清单的既定方法，包括使用与介绍性环境项目学生的访谈来揭示学生的先入为主的概念，并开发构建的回答问题。通过审查美国各地的环境课程收集的信息，结合从下一代概念清单中对学生学习情况的衡量，可以为有关大学环境科学和研究项目的课程和人员配备决策提供依据。因此，该项目通过提供一个有效和可靠的工具来评估学生的学习、课程结果和项目有效性，从而有可能使环境项目的教职员工和学生受益。NSF IUSE：EHR计划支持研究和开发项目，以提高所有学生的STEM教育的有效性。通过参与的学生学习路径，该计划支持有前景的实践和工具的创建、探索和实施。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This project aims to serve the national interest by improving teaching practices in undergraduate environmental programs. To do so, the project will develop and test a written response concept inventory to measure undergraduate students’ knowledge of complex Food-Energy-Water systems. Traditional concept inventories are widely used in many disciplines to measure student understanding of key ideas in the discipline. These concept inventories use answers to multiple choice questions to measure student knowledge. However, greater insight into student thinking can be gained from questions that require students to use their own words to demonstrate their knowledge. This project plans to develop a “Next Generation Concept Inventory” that will use short answer questions and automated scoring of student responses to questions about interdisciplinary, systems-level concepts. The availability of this concept inventory is expected to help faculty evaluate their students’ understanding of complex environmental concepts, as well as evaluate and compare learning within and between programs. This information, in turn, can be used to improve teaching effectiveness within environmental programs and may serve as a model for similar approaches in other disciplines.To better inform those who teach, make curricular decisions, and manage college-level environmental programs, the project will develop an assessment tool to measure environmental students’ foundational knowledge and understanding of complex systems-level concepts. Specifically, the project will apply an established machine learning method of evaluating constructed response (short answer) questions to create a Next Generation Concept Inventory. This approach to concept inventory construction will create a new set of constructed-response items and associated automated scoring models focused on complex Food-Energy-Water Nexus systems, a topic typically addressed in environmental programs. The project will evaluate the concept inventory’s validity and reliability to ensure that it will provide high-quality information that can be used by environmental educators and program administrators, as well as for research on STEM education and assessment. The project team will first determine common Food-Energy-Water Nexus concepts and learning outcomes through examination of introductory course materials from environmental programs across the nation. Development of the Next Generation Concept Inventory targeting these concepts/outcomes will follow established methods for concept inventory construction, including the use of interviews with introductory environmental program students to reveal student preconceptions and to develop the constructed response questions. The information gleaned from reviewing environmental curricula across the United States, combined with measurements of student learning from the Next Generation Concept Inventory, can inform curricular and staffing decisions regarding college environmental science and studies programs. Thus, the project has the potential to benefit faculty and students in environmental programs by providing a valid and reliable instrument for evaluating student learning, course outcomes, and program effectiveness. The NSF IUSE: EHR Program supports research and development projects to improve the effectiveness of STEM education for all students. Through the Engaged Student Learning track, the program supports the creation, exploration, and implementation of promising practices and tools.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.