Automatically Measuring Question Authenticity in Real-World Classrooms
Automatically Measuring Question Authenticity in Real-World Classrooms
复制标题
DOI：
10.3102/0013189x18785613
复制
发表时间：
2018-06
影响因子：
8.2
通讯作者：
Sean Kelly;A. Olney;P. Donnelly;M. Nystrand;S. D’Mello
中科院分区：
文献类型：
--
作者：
Sean Kelly;A. Olney;P. Donnelly;M. Nystrand;S. D’Mello
分析课堂话语的质量是教育研究和改进工作的核心。特别是，真实的教师问题的存在，答案不是由教师预先确定的，有助于构成和作为一个标记的生产性课堂话语。此外，可以培养真实的问题，以提高教学效果，从而提高学生的成绩。不幸的是，目前的方法来衡量问题的真实性不规模，因为他们依赖于人类的观察或编码的教师话语。为了应对这一挑战，我们开始使用自动语音识别、自然语言处理和机器学习来训练计算机自动检测现实世界教室中的真实问题。我们的方法使用来自两个来源的教室音频和人类编码的观察数据进行了迭代改进：（a）来自112个教室的451个观察结果的文本记录的大型档案数据库;以及（B）来自27个教室的132个高质量音频记录的新收集样本，在预期大规模自动数据收集和分析的技术限制下获得。人类编码和计算机编码的真实性之间的相关性在课堂上是足够高的（r = 0.602的档案成绩单和0.687的录音），以提供一个有价值的补充人类编码的研究工作。
Analyzing the quality of classroom talk is central to educational research and improvement efforts. In particular, the presence of authentic teacher questions, where answers are not predetermined by the teacher, helps constitute and serves as a marker of productive classroom discourse. Further, authentic questions can be cultivated to improve teaching effectiveness and consequently student achievement. Unfortunately, current methods to measure question authenticity do not scale because they rely on human observations or coding of teacher discourse. To address this challenge, we set out to use automatic speech recognition, natural language processing, and machine learning to train computers to detect authentic questions in real-world classrooms automatically. Our methods were iteratively refined using classroom audio and human-coded observational data from two sources: (a) a large archival database of text transcripts of 451 observations from 112 classrooms; and (b) a newly collected sample of 132 high-quality audio recordings from 27 classrooms, obtained under technical constraints that anticipate large-scale automated data collection and analysis. Correlations between human-coded and computer-coded authenticity at the classroom level were sufficiently high (r = .602 for archival transcripts and .687 for audio recordings) to provide a valuable complement to human coding in research efforts.