Development of the Biology Lab Inventory of Critical Thinking for Ecology (Eco-BLIC)

在NSF改进本科STEM教育计划：教育和人力资源（IUSE：EHR）的支持下，该项目旨在通过开发一种新的评估工具来服务于国家利益，该工具名为生态学批判性思维的生物学实验室清单或Eco-BLIC，生物学教师可以用来测量本科生在从现场环境收集的生态学数据背景下的批判性思维技能。批判性思维在这里被定义为一种基于证据的方式，个人在这种方式下决定信任什么和做什么。虽然迄今为止的教育研究主要集中在学生从讲座和背诵中学习，但对批判性思维学习成果的研究却少得多，而批判性思维学习成果往往是教学实验室和实地课程的既定目标。这种不平等的平衡留下了许多开放的研究问题，学生目前正在学习这些课程，学生可以学习，以及如何衡量这种学习。此外，生态学实验室和实地课程，帮助学生通过参与研究项目进行批判性思维，可以是部门课程中资源最密集的部分。他们往往需要专门的设施，广泛的协调，并占用大量的学生和教学人员的时间。这些后勤方面的挑战和资源，沿着很少有研究来评估学生在这些课程中学习到了什么样的批判性思维技能，导致了本科阶段真实的实验室和野外生态学课程的数量下降。有了Eco-BLIC，各系将能够衡量学生批判性思维能力的潜在增长，这反过来又可以帮助他们确定如何最好地论证和使用资源，以提高他们的生态学实验室和实地课程。虽然有相关的仪器，但没有一种仪器可以有效地满足评估本科生态学实验室和实地课程中以实验为重点的批判性思维的需求。拟议的Eco-BLIC工具将使生物教育研究界能够响应建立有关实验室和现场课程教学的证据的呼吁，并产生有关学生核心能力和批判性思维困难的新知识。生态BLIC的设计将建立在最近的工作，以开发批判性思维的物理实验室库存（PLIC）。根据PLIC的设计，我们将创建实验背景和问题集，以探讨学生对生态学实地研究数据的批判性思维能力。背景和问题将用于评估来自多个机构的一系列学生，并将按照标准程序进行研究验证，包括比较开放式和封闭式回答版本，采访学生以及从专家那里获得反馈。在问题开发过程中，专家反馈将检查Eco-BLIC问题的清晰度和准确性。专家的回答也将用于：1）制定评分方案，2）通过比较本科生和专家批判性思维技能的差异来评估仪器的区分度，以及3）测量学生与专家回答一致性的前/后变化。Eco-BLIC的试点版本将提供给多个机构类型的学生，我们将使用一套统计技术（例如，信度系数的重测稳定性，项目分析，测试分析，因素和网络分析），以收集证据的评估有效性和可靠性，并确定在何种程度上的问题是能够衡量的结果，在不同的学生群体。我们还将比较学生对PLIC与Eco-BLIC的回答，以确定和研究跨越多个学科的批判性思维主题。 NSF IUSE：EHR计划支持研究和开发项目，以提高所有学生STEM教育的有效性。 该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

With support from the NSF Improving Undergraduate STEM Education Program: Education and Human Resources (IUSE: EHR), this project aims to serve the national interest by developing a new assessment instrument, titled the Biology Lab Inventory of Critical thinking for Ecology or Eco-BLIC, that biology instructors can use to measure undergraduate students' critical thinking skills in the context of ecology data collected from field settings. Critical thinking is defined here as the evidence-based ways in which individuals make decisions about what to trust and what to do. While education research to date has largely focused on student learning from lectures and recitations, there is far less research on the critical thinking learning outcomes that are often stated goals of instructional laboratory and field courses. This unequal balance leaves many open research questions regarding what students are currently learning from these courses, what students could be learning, and how to measure such learning. Furthermore, ecology lab and field courses that help students engage in critical thinking through participation in research projects can be some of the most resource intensive components of a departmental curriculum. They often require specialized facilities, extensive coordination, and occupy a significant amount of student and instructional staff time. These logistical challenges and resources, along with little research to evaluate what critical thinking skills students are learning in these courses, have caused a decline in offerings of authentic laboratory and field ecology courses at the undergraduate level. With the Eco-BLIC, departments will be able to measure potential growth in students' critical thinking skills, which in turn can help them determine how to best argue for and use resources to enhance their ecology laboratory and field-based offerings.While there are related instruments, no single instrument effectively meets the needs of assessing experimentation-focused critical thinking in undergraduate ecology lab and field courses. The proposed Eco-BLIC instrument will allow the biology education research community to respond to calls to build evidence about laboratory and field course instruction, and to generate new knowledge about student core competencies and difficulties in critical thinking. The design of the Eco-BLIC will build from recent work to develop a Physics Lab Inventory of Critical thinking (PLIC). Based on the design of the PLIC, we will create experimental contexts and suites of questions that probe students' critical thinking skills about ecology field research data. The contexts and questions will be intended to assess a range of students from multiple institutions and will be research-validated following standard procedures, including comparing open- and closed-response versions, interviewing students, and getting feedback from experts. During the question development process, the expert feedback will check the clarity and accuracy of the Eco-BLIC questions. Expert answers will also be used to: 1) develop the scoring scheme, 2) evaluate discrimination of the instrument by comparing differences in undergraduate and expert critical thinking skills, and 3) measure pre/post changes in students' agreement with the expert responses. Pilot versions of Eco-BLIC will be given to students at multiple institution types and we will use a suite of statistical techniques (e.g., reliability coefficients for test-retest stability, item analysis, test analysis, factor and network analysis) to collect evidence of assessment validity and reliability, and to determine the extent to which the questions are able to measure outcomes in various student populations. We will also compare student answers on the PLIC versus the Eco-BLIC to identify and study themes about critical thinking that span multiple disciplines. The NSF IUSE: EHR Program supports research and development projects to improve the effectiveness of STEM education for all students. Through the Engaged Student Learning track, the program supports the creation, exploration, and implementation of promising practices and tools.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.