Development of a classroom observation format and protocol to assess student participation and interactivity in undergraduate STEM courses

在高等教育中，课堂观察通常用于审查和评估教师的表现。这种方法存在两个主要问题。首先，不同的观察者对什么是“好的教学”有自己的看法，并且经常依赖这些个人标准来判断。因此，原则上，不同的观察者将以不同的方式回顾同一个类。更糟糕的是，这种评估给被观察的教师提供了次优反馈，因为它通常不是基于与学生积极成果相关的循证方法。该项目的重点是测试一种评估性观察工具，这种工具比传统方法更客观、更翔实。这项正在进行的工作被称为学生参与观察工具（SPOT）。它被设计为一种计算机化的观察格式和协议，它将允许观察者在观察期间对课堂行为进行分类，从而产生关于课堂进展的详细时间戳信息。SPOT的设计目的是获取有关促进学生积极成果的教学实践使用的数据，以便对评估人员和教师都有用。从设计上考虑，SPOT将易于使用，因此将有潜力在教师开发人员和其他日常进行课堂观察的人员中广泛采用。这个探索性项目集成了研究和设计组件，以1)开发基于一系列设计目标的观察工具，以及2)评估工具的有效性（它满足设计目标的程度）。设计目标是利用教育研究和现有的观察工具来开发一种格式和协议，以便收集与学生积极学习成果相关的学生和教师行为的实时数据。评估目标是评估观察格式和协议的可用性和有用性，为教师和评估人员提供关于课堂实践的有效和可靠的数据，以便他们可以对STEM教育做出基于证据的决策。SPOT是基于一个类似的经过良好测试的工具开发的，该工具可以实时对课堂行为进行分类。该协议的制定是基于对K-12文献和特定学科的高等教育研究的回顾，这些研究涉及教师和学生的课堂行为，这些行为会导致积极的学生成果。随后将在圣何塞州立大学广泛的课程中对SPOT进行实地测试，以提高可用性，并确定该工具的有效性和可靠性。

In higher education, classroom observations are typically conducted to review and evaluate an instructor's performance. There are two main problems with this approach. First, different observers have their own ideas of what constitutes "good teaching," and often rely on these personal standards in their judgments. Thus, in principle, different observers will review the same class differently. Worse, the evaluation provides suboptimal feedback to the observed instructors because it is not typically based on evidence-based methods linked to positive student outcomes. The focus of this project is the testing of an evaluative observation tool that is more object and informative than traditional methods. This work in progress is known as the Student Participation Observation Tool (SPOT). It is being designed as a computerized observing format and protocol that will allow an observer to categorize classroom actions during an observation, resulting in detailed time-stamped information about the progression of the class. SPOT is being designed to capture data about the use of teaching practices that promote positive student outcomes, in order to be useful to both evaluators and instructors. As a design consideration, SPOT will be easy to use and therefore will have potential for widespread adoption among faculty developers and others who routinely conduct classroom observations. This exploratory project integrates research and design components to 1) develop the observation tool based on a series of design goals, and 2) evaluate the effectiveness of the tool (how well it meets the design goals). The design objective is drawing on education research and existing observation tools to develop a format and protocol that allows collecting real-time data about student and teacher behaviors that are linked to positive student learning outcomes. The evaluation objective is to assess the usability and usefulness of the observation format and protocol to provide valid and reliable data to instructors and evaluators about classroom practices so that they may make evidence-based decisions regarding STEM education. SPOT is being developed based on a similar well-tested tool that categorizes classroom behaviors in real time. The protocol is being developed based on a review of K-12 literature and discipline-specific higher education research regarding instructor and student classroom behaviors that result in positive student outcomes. Subsequent field-testing of SPOT in a wide-range of courses at San Jose State University will be undertaken to improve usability, and establish the validity and reliability of the tool.