Development of a classroom observation format and protocol to assess student participation and interactivity in undergraduate STEM courses

在高等教育中，课堂观察通常被用来回顾和评估教师的表现。这种方法有两个主要问题。首先，不同的观察者对什么是“好的教学”有他们自己的想法，并经常依靠这些个人标准来判断。因此，原则上，不同的观察员将以不同的方式审查相同的课程。更糟糕的是，评估向被观察的教师提供的反馈不是最优的，因为它通常不是基于与积极的学生结果相关的循证方法。这个项目的重点是测试一种比传统方法更客观、更有信息量的评价性观察工具。这项正在进行的工作被称为学生参与观察工具(SPOT)。它被设计为一种计算机化的观察格式和协议，使观察员能够对观察期间的课堂行为进行分类，从而产生关于课堂进展的详细时间戳信息。Spot的设计是为了收集有关使用教学实践的数据，以促进积极的学生结果，以便对评估者和教师都有用。作为设计考虑，Spot将易于使用，因此有可能在教员开发人员和其他经常进行课堂观察的人中广泛采用。这个探索性项目集成了研究和设计组件，以1)基于一系列设计目标开发观察工具，以及2)评估工具的有效性(它在多大程度上满足设计目标)。设计目标是利用教育研究和现有的观察工具来开发一种格式和协议，允许收集与积极的学生学习结果相关的学生和教师行为的实时数据。评估的目标是评估观察格式和协议的可用性和有用性，以便向教师和评估者提供关于课堂实践的有效和可靠的数据，以便他们可以就STEM教育做出基于证据的决定。Spot正在基于一个类似的经过充分测试的工具开发，该工具可以对课堂行为进行实时分类。该协议是基于对K-12文献和特定学科的高等教育研究的回顾而制定的，这些研究涉及教师和学生的课堂行为，这些行为会给学生带来积极的结果。随后将在圣何塞州立大学的一系列课程中对Spot进行实地测试，以提高可用性，并确定该工具的有效性和可靠性。

In higher education, classroom observations are typically conducted to review and evaluate an instructor's performance. There are two main problems with this approach. First, different observers have their own ideas of what constitutes "good teaching," and often rely on these personal standards in their judgments. Thus, in principle, different observers will review the same class differently. Worse, the evaluation provides suboptimal feedback to the observed instructors because it is not typically based on evidence-based methods linked to positive student outcomes. The focus of this project is the testing of an evaluative observation tool that is more object and informative than traditional methods. This work in progress is known as the Student Participation Observation Tool (SPOT). It is being designed as a computerized observing format and protocol that will allow an observer to categorize classroom actions during an observation, resulting in detailed time-stamped information about the progression of the class. SPOT is being designed to capture data about the use of teaching practices that promote positive student outcomes, in order to be useful to both evaluators and instructors. As a design consideration, SPOT will be easy to use and therefore will have potential for widespread adoption among faculty developers and others who routinely conduct classroom observations. This exploratory project integrates research and design components to 1) develop the observation tool based on a series of design goals, and 2) evaluate the effectiveness of the tool (how well it meets the design goals). The design objective is drawing on education research and existing observation tools to develop a format and protocol that allows collecting real-time data about student and teacher behaviors that are linked to positive student learning outcomes. The evaluation objective is to assess the usability and usefulness of the observation format and protocol to provide valid and reliable data to instructors and evaluators about classroom practices so that they may make evidence-based decisions regarding STEM education. SPOT is being developed based on a similar well-tested tool that categorizes classroom behaviors in real time. The protocol is being developed based on a review of K-12 literature and discipline-specific higher education research regarding instructor and student classroom behaviors that result in positive student outcomes. Subsequent field-testing of SPOT in a wide-range of courses at San Jose State University will be undertaken to improve usability, and establish the validity and reliability of the tool.