Development of a classroom observation format and protocol to assess student participation and interactivity in undergraduate STEM courses

在高等教育中，课堂观察通常是为了审查和评估教师的表现。这种方法有两个主要问题。首先，不同的观察者对什么是“好的教学”有自己的看法，而且在他们的判断中往往依赖这些个人标准。因此，原则上，不同的观察者会以不同的方式审查同一类。 更糟糕的是，评估提供了次优反馈给观察教师，因为它通常不是基于与学生积极成果相关的循证方法。该项目的重点是测试一种比传统方法更客观、信息量更大的评价性观察工具。 这项正在进行的工作被称为学生参与观察工具。它被设计成一种计算机化的观察格式和协议，允许观察者在观察期间对课堂行动进行分类，从而获得有关课堂进展的详细时间戳信息。SPOT的目的是收集关于促进学生取得积极成果的教学做法使用情况的数据，以便对评估人员和教员都有用。作为一个设计考虑，SPOT将易于使用，因此将有可能在教师开发人员和其他经常进行课堂观察的人中广泛采用。该探索性项目整合了研究和设计组件，以1）基于一系列设计目标开发观察工具，2）评估工具的有效性（满足设计目标的程度）。设计目标是利用教育研究和现有的观察工具，开发一种格式和协议，允许收集有关学生和教师行为的实时数据，这些数据与学生的积极学习成果有关。评估目标是评估观察格式和协议的可用性和有用性，为教师和评估人员提供有关课堂实践的有效和可靠数据，以便他们可以就STEM教育做出基于证据的决策。 SPOT是基于一个类似的经过良好测试的工具开发的，该工具可以对课堂行为进行真实的分类。该协议是基于K-12文献和学科特定的高等教育研究的审查，教师和学生的课堂行为，导致积极的学生成果。随后将在圣何塞州立大学的一系列课程中对SPOT进行实地测试，以提高该工具的可用性，并确定其有效性和可靠性。

In higher education, classroom observations are typically conducted to review and evaluate an instructor's performance. There are two main problems with this approach. First, different observers have their own ideas of what constitutes "good teaching," and often rely on these personal standards in their judgments. Thus, in principle, different observers will review the same class differently. Worse, the evaluation provides suboptimal feedback to the observed instructors because it is not typically based on evidence-based methods linked to positive student outcomes. The focus of this project is the testing of an evaluative observation tool that is more object and informative than traditional methods. This work in progress is known as the Student Participation Observation Tool (SPOT). It is being designed as a computerized observing format and protocol that will allow an observer to categorize classroom actions during an observation, resulting in detailed time-stamped information about the progression of the class. SPOT is being designed to capture data about the use of teaching practices that promote positive student outcomes, in order to be useful to both evaluators and instructors. As a design consideration, SPOT will be easy to use and therefore will have potential for widespread adoption among faculty developers and others who routinely conduct classroom observations. This exploratory project integrates research and design components to 1) develop the observation tool based on a series of design goals, and 2) evaluate the effectiveness of the tool (how well it meets the design goals). The design objective is drawing on education research and existing observation tools to develop a format and protocol that allows collecting real-time data about student and teacher behaviors that are linked to positive student learning outcomes. The evaluation objective is to assess the usability and usefulness of the observation format and protocol to provide valid and reliable data to instructors and evaluators about classroom practices so that they may make evidence-based decisions regarding STEM education. SPOT is being developed based on a similar well-tested tool that categorizes classroom behaviors in real time. The protocol is being developed based on a review of K-12 literature and discipline-specific higher education research regarding instructor and student classroom behaviors that result in positive student outcomes. Subsequent field-testing of SPOT in a wide-range of courses at San Jose State University will be undertaken to improve usability, and establish the validity and reliability of the tool.