Developing a Next Generation Concept Inventory to Help Environmental Programs Evaluate Student Knowledge of Complex Food-Energy-Water Systems

该项目旨在通过改善本科环境课程的教学实践来服务于国家利益。为此，该项目将开发和测试一个书面的反应概念清单，以衡量本科生对复杂的食物-能源-水系统的知识。传统的概念量表广泛应用于许多学科，以衡量学生对学科中关键思想的理解。 这些概念清单使用多项选择题的答案来衡量学生的知识。然而，从要求学生用自己的话来证明他们的知识的问题中，可以更深入地了解学生的想法。该项目计划开发一个“下一代概念清单”，将使用简短的答案问题和自动评分的学生回答有关跨学科，系统级的概念的问题。这个概念清单的可用性，预计将帮助教师评估他们的学生对复杂的环境概念的理解，以及评估和比较内和程序之间的学习。这些信息，反过来，可以用来提高教学效率在环境方案，并可能作为一个模型，类似的方法在其他学科。为了更好地通知那些谁教，制定课程的决定，并管理大学水平的环境计划，该项目将开发一个评估工具，以衡量环境学生的基础知识和复杂的系统级概念的理解。具体而言，该项目将应用一种已建立的机器学习方法来评估构建的响应（简答）问题，以创建下一代概念清单。这种概念库存构建方法将创建一组新的构建响应项目和相关的自动评分模型，重点关注复杂的食品-能源-水关系系统，这是环境计划中通常涉及的主题。该项目将评估概念清单的有效性和可靠性，以确保它将提供高质量的信息，可用于环境教育工作者和项目管理人员，以及对STEM教育和评估的研究。项目团队将首先通过检查全国各地环境项目的入门课程材料，确定常见的食品-能源-水关系概念和学习成果。针对这些概念/成果的下一代概念清单的开发将遵循概念清单构建的既定方法，包括使用与介绍性环境项目学生的访谈来揭示学生的先入之见并开发构建的响应问题。从审查美国各地的环境课程收集的信息，结合学生学习的测量从下一代概念清单，可以告知有关大学环境科学和研究计划的课程和人员配置的决定。因此，该项目有可能通过提供一个有效和可靠的工具来评估学生的学习，课程成果和方案的有效性，以有利于教师和学生在环境方案。NSF IUSE：EHR计划支持研究和开发项目，以提高所有学生STEM教育的有效性。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project aims to serve the national interest by improving teaching practices in undergraduate environmental programs. To do so, the project will develop and test a written response concept inventory to measure undergraduate students’ knowledge of complex Food-Energy-Water systems. Traditional concept inventories are widely used in many disciplines to measure student understanding of key ideas in the discipline. These concept inventories use answers to multiple choice questions to measure student knowledge. However, greater insight into student thinking can be gained from questions that require students to use their own words to demonstrate their knowledge. This project plans to develop a “Next Generation Concept Inventory” that will use short answer questions and automated scoring of student responses to questions about interdisciplinary, systems-level concepts. The availability of this concept inventory is expected to help faculty evaluate their students’ understanding of complex environmental concepts, as well as evaluate and compare learning within and between programs. This information, in turn, can be used to improve teaching effectiveness within environmental programs and may serve as a model for similar approaches in other disciplines.To better inform those who teach, make curricular decisions, and manage college-level environmental programs, the project will develop an assessment tool to measure environmental students’ foundational knowledge and understanding of complex systems-level concepts. Specifically, the project will apply an established machine learning method of evaluating constructed response (short answer) questions to create a Next Generation Concept Inventory. This approach to concept inventory construction will create a new set of constructed-response items and associated automated scoring models focused on complex Food-Energy-Water Nexus systems, a topic typically addressed in environmental programs. The project will evaluate the concept inventory’s validity and reliability to ensure that it will provide high-quality information that can be used by environmental educators and program administrators, as well as for research on STEM education and assessment. The project team will first determine common Food-Energy-Water Nexus concepts and learning outcomes through examination of introductory course materials from environmental programs across the nation. Development of the Next Generation Concept Inventory targeting these concepts/outcomes will follow established methods for concept inventory construction, including the use of interviews with introductory environmental program students to reveal student preconceptions and to develop the constructed response questions. The information gleaned from reviewing environmental curricula across the United States, combined with measurements of student learning from the Next Generation Concept Inventory, can inform curricular and staffing decisions regarding college environmental science and studies programs. Thus, the project has the potential to benefit faculty and students in environmental programs by providing a valid and reliable instrument for evaluating student learning, course outcomes, and program effectiveness. The NSF IUSE: EHR Program supports research and development projects to improve the effectiveness of STEM education for all students. Through the Engaged Student Learning track, the program supports the creation, exploration, and implementation of promising practices and tools.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.