Using Neural Networks for Automated Classification of Elementary Mathematics Instructional Activities

该研究项目由EHR核心研究（ECR）计划提供支持，该计划支持推进STEM学习和学习环境的基础研究，扩大STEM的参与度，以及STEM劳动力发展。在过去的十年中，使用视频进行教师准备和研究教学质量的情况大幅增加。对教学候选人和初任教师进行总结性评价的突出方法是对教学实践进行录像。此外，大规模的研究还包括教学录像。教师准备计划越来越多地使用视频方法课程和形成性评估的目的。此外，越来越多地使用交互式模拟在职前准备和在职专业发展依赖于具有视频记录的示例性教学实践。尽管近年来使用视频来衡量和促进教学质量的显着增长，但大规模使用视频仍然存在一些关键挑战。首先，经过培训的人工评分员观看数百小时的视频记录课程非常耗时。其次，每个视频使用多个人工评分员会增加财务成本和时间需求。第三，手动编目、标记和索引大量课堂视频以供以后观看是耗时的。计算机视觉、机器学习和深度学习的最新进展可能为这些挑战提供解决方案，并使分析和评分视频的过程更加高效。特别是，深度学习已经成为与分析视频内容相关的问题的最新选择。这项拟议的NSF核心研究将利用为两项NSF资助的研究收集的小学数学教学视频，这些研究采用了数学扫描（M-Scan）课堂观察工具。本研究将使用这些视频探索深度神经网络可以用于对数学教学视频中的教学活动进行分类的几种方法。本研究将通过检查三种类型的人工神经网络对小学数学教学视频中的（a）对象和（B）教学活动进行准确分类的程度来推进知识和理解。例如，神经网络可以直接确定小学教师是否参与讲座与促进与学生的讨论。另一方面，这样的网络可能更难评估教师代表数学内容的方式，他们的问题的性质，以及学生的手势是否意味着理解。研究设计涉及使用计算机视觉、机器学习和深度学习的三个方面：（a）神经网络的类型，（B）视频标签的类型（即，对象标签和教学活动标签），以及（c）数学教学的主题（例如，数与运算;模式、函数与代数;几何）。一些类型的神经网络最近被证明对视频分类有效：卷积神经网络（CNN），长短期记忆（LSTM）神经网络和混合CNN-LSTM。最终，该项目旨在开始建立系统的基础设施，以有效和负担得起的方式对课堂教学视频进行分类。研究结果将可能有关键的影响（a）大规模的研究，教学视频和（B）职前教师准备计划，在职专业发展活动，并努力评估教学候选人和执业教师。特别是，这项研究的结果将为有关可用于正确分类教学视频的神经网络类型以及为此目的使用网络的实际限制的决策提供信息。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估来支持。

This research project is supported by the EHR Core Research (ECR) program, which supports work that advances fundamental research on STEM learning and learning environments, broadening participation in STEM, and STEM workforce development.In the last decade, there has been a tremendous increase in the use of video for preparing teachers and studying teaching quality. Prominent approaches to summative evaluation of teaching candidates and beginning teachers feature video recordings of instructional practices. In addition, large-scale research studies have included video of instruction. Teacher preparation programs increasingly use video in methods courses and for formative assessment purposes. Moreover, the growing use of interactive simulation in pre-service preparation and in-service professional development relies on having video recorded examples of exemplary instructional practices. Despite the significant growth in the use of video to measure and promote instructional quality in recent years, there remain some key challenges to employing it at scale. First, it is very time-consuming for trained human raters to view hundreds of hours of video recorded lessons. Second, financial costs and time demands increase with the use of multiple human raters per video. Third, manually cataloging, labeling, and indexing large volumes of classroom video for later viewing is time consuming. Recent advances in computer vision, machine learning, and deep learning may provide solutions to these challenges and could make the process of analyzing and scoring videos more efficient. In particular, deep learning has become the state-of-the-art choice in problems related to analyzing the content of video. This proposed NSF Core Research study will draw on videos of elementary mathematics instruction that were collected for two NSF-funded studies that featured the Mathematics-Scan (M-Scan) classroom observation tool. The research will use these videos to explore several ways that deep neural networks can be used to classify instructional activities in videos of math instruction.This study will advance knowledge and understanding by examining the degree to which three types of artificial neural networks can accurately classify (a) objects and (b) instructional activities in videos of elementary mathematics instruction. For example, it may be straightforward for neural networks to determine whether an elementary teacher is engaged in lecture vs. facilitating discussion with students. On the other hand, it may be harder for such networks to assess the ways teachers represent math content, the nature of their questions, and whether student gestures signify understanding. The research design concerns three aspects of using computer vision, machine learning, and deep learning: (a) the type of neural network, (b) the type of video label (i.e., object labels and instructional activity labels), and (c) the subject of math instruction (e.g., number and operations; patterns, functions, and algebra; geometry). A few types of neural networks have recently proven effective for video classification: convolutional neural networks (CNNs), long short-term memory (LSTM) neural networks, and hybrid CNN-LSTMs. Ultimately, this project is aimed at beginning to build systematic infrastructure for classifying videos of classroom instruction at scale in efficient and affordable ways. The findings will potentially have key implications for (a) large-scale research studies that feature videos of instruction and (b) pre-service teacher preparation programs, in-service professional development activities, and efforts to evaluate teaching candidates and practicing teachers. In particular, the results from this study will inform decisions about the types of neural networks that can be used to correctly classify videos of instruction and the practical limitations of using networks for this purpose.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.