Development of the Biology Lab Inventory of Critical Thinking for Ecology (Eco-BLIC)

在美国国家科学基金会改善本科STEM教育计划：教育与人力资源（IUSE: EHR）的支持下，该项目旨在通过开发一种新的评估工具来服务于国家利益，该工具名为“生态学批判性思维的生物实验室清单”或“Eco-BLIC”，生物教师可以使用该工具来衡量本科生在野外环境中收集的生态学数据的批判性思维技能。批判性思维在这里被定义为以证据为基础的方式，在这种方式下，个人决定该相信什么，该做什么。虽然迄今为止的教育研究主要集中在学生从讲座和背诵中学习，但对批判性思维学习成果的研究却少得多，而批判性思维学习成果通常是教学实验室和实地课程的既定目标。这种不平等的平衡留下了许多悬而未决的研究问题，比如学生目前从这些课程中学到了什么，学生可能会学到什么，以及如何衡量这种学习。此外，通过参与研究项目来帮助学生进行批判性思维的生态学实验室和实地课程可能是院系课程中资源最密集的部分。它们通常需要专门的设施，广泛的协调，并占用大量的学生和教学人员的时间。这些后勤方面的挑战和资源，加上很少有研究来评估学生在这些课程中学到的批判性思维技能，导致了本科阶段真正的实验室和野外生态学课程的减少。有了Eco-BLIC，各院系将能够衡量学生批判性思维技能的潜在增长，这反过来可以帮助他们确定如何最好地争取和利用资源，以提高他们的生态学实验室和实地研究成果。虽然有相关的工具，但没有一种工具能有效地满足本科生态学实验室和实地课程中评估以实验为重点的批判性思维的需要。拟议的eco - lic仪器将使生物学教育研究界能够响应建立关于实验室和实地课程教学的证据的呼吁，并产生关于学生核心能力和批判性思维困难的新知识。Eco-BLIC的设计将基于最近开发的批判性思维物理实验室清单（PLIC）的工作。基于PLIC的设计，我们将创建实验情境和问题套件，以探索学生对生态学实地研究数据的批判性思维技能。这些背景和问题将用于评估来自多个机构的一系列学生，并将按照标准程序进行研究验证，包括比较开放式和封闭式回答版本，采访学生，并从专家那里获得反馈。在问题开发过程中，专家反馈将检查eco - lic问题的清晰度和准确性。专家回答还将用于：1)制定评分方案，2)通过比较本科生和专家批判性思维技能的差异来评估工具的歧视，以及3)测量学生对专家回答的同意前后的变化。Eco-BLIC的试点版本将提供给多个院校类型的学生，我们将使用一套统计技术（例如，测试-重测稳定性的信度系数、项目分析、测试分析、因素和网络分析）来收集评估有效性和可靠性的证据，并确定问题能够衡量不同学生群体结果的程度。我们还将比较学生对PLIC和eco - lic的回答，以确定和研究跨多学科的批判性思维主题。NSF IUSE: EHR计划支持研究和开发项目，以提高所有学生STEM教育的有效性。通过参与学生学习轨道，该计划支持有前途的实践和工具的创建，探索和实施。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

With support from the NSF Improving Undergraduate STEM Education Program: Education and Human Resources (IUSE: EHR), this project aims to serve the national interest by developing a new assessment instrument, titled the Biology Lab Inventory of Critical thinking for Ecology or Eco-BLIC, that biology instructors can use to measure undergraduate students' critical thinking skills in the context of ecology data collected from field settings. Critical thinking is defined here as the evidence-based ways in which individuals make decisions about what to trust and what to do. While education research to date has largely focused on student learning from lectures and recitations, there is far less research on the critical thinking learning outcomes that are often stated goals of instructional laboratory and field courses. This unequal balance leaves many open research questions regarding what students are currently learning from these courses, what students could be learning, and how to measure such learning. Furthermore, ecology lab and field courses that help students engage in critical thinking through participation in research projects can be some of the most resource intensive components of a departmental curriculum. They often require specialized facilities, extensive coordination, and occupy a significant amount of student and instructional staff time. These logistical challenges and resources, along with little research to evaluate what critical thinking skills students are learning in these courses, have caused a decline in offerings of authentic laboratory and field ecology courses at the undergraduate level. With the Eco-BLIC, departments will be able to measure potential growth in students' critical thinking skills, which in turn can help them determine how to best argue for and use resources to enhance their ecology laboratory and field-based offerings.While there are related instruments, no single instrument effectively meets the needs of assessing experimentation-focused critical thinking in undergraduate ecology lab and field courses. The proposed Eco-BLIC instrument will allow the biology education research community to respond to calls to build evidence about laboratory and field course instruction, and to generate new knowledge about student core competencies and difficulties in critical thinking. The design of the Eco-BLIC will build from recent work to develop a Physics Lab Inventory of Critical thinking (PLIC). Based on the design of the PLIC, we will create experimental contexts and suites of questions that probe students' critical thinking skills about ecology field research data. The contexts and questions will be intended to assess a range of students from multiple institutions and will be research-validated following standard procedures, including comparing open- and closed-response versions, interviewing students, and getting feedback from experts. During the question development process, the expert feedback will check the clarity and accuracy of the Eco-BLIC questions. Expert answers will also be used to: 1) develop the scoring scheme, 2) evaluate discrimination of the instrument by comparing differences in undergraduate and expert critical thinking skills, and 3) measure pre/post changes in students' agreement with the expert responses. Pilot versions of Eco-BLIC will be given to students at multiple institution types and we will use a suite of statistical techniques (e.g., reliability coefficients for test-retest stability, item analysis, test analysis, factor and network analysis) to collect evidence of assessment validity and reliability, and to determine the extent to which the questions are able to measure outcomes in various student populations. We will also compare student answers on the PLIC versus the Eco-BLIC to identify and study themes about critical thinking that span multiple disciplines. The NSF IUSE: EHR Program supports research and development projects to improve the effectiveness of STEM education for all students. Through the Engaged Student Learning track, the program supports the creation, exploration, and implementation of promising practices and tools.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.