聋哑学生课堂视频的机器理解与效能分析

本项目提出了聋哑学生课堂视频机器理解和效能分析的理论框架和全新方法，探索和研究聋哑学生课堂学习认知情绪的机器识别、聋哑学生课堂行为姿态的机器识别、聋哑学生课堂互动场景的机器理解、以及基于机器理解的学习效能分析理论模型和关键技术。项目将基于中央差分卷积神经网络模型和多尺度注意力融合机制，研究聋哑学生课堂学习认知情绪细粒度表征和识别算法；融合不同层级卷积神经网络特征和长短时记忆网络对聋哑学生课堂视频进行时空特征提取和可视化概念识别；采用多通道卷积神经网络和长短时记忆网络架构，研究聋哑学生课堂视频语义描述模型和典型互动场景的机器理解；研究基于机器理解的学习效能分析以提供智能的个性化学习支持服务。项目将基于真实的聋哑学生课堂视频数据，对提出的理论模型和算法进行实证分析。本项研究属于人工智能和教育科学的交叉领域，对于特殊教育的研究与应用具有重要的理论及现实意义。

The theory framework and new methods of machine understanding and effectiveness analysis of classroom videos for the deaf-mute students are proposed in this project. The focus of the research will be the theoretical models and key technologies in machine recognition of classroom learning cognitive emotions and behavioral stances, machine understanding of classroom interaction scenarios, and learning effectiveness analysis for the deaf-mute students. In this project, we will construct fine-grained feature representation and recognition algorithms of classroom learning cognitive emotions for the deaf-mute students based on central difference convolution neural networks and multiscale attention fusion mechanism. We will combine different level features of convolutional neural networks and long short-term memory networks to obtain the temporal features and recognize the visualization concepts of classroom videos of the deaf-mute students. We will use convolutional neural networks with multi-channels and long short-term memory networks to construct a semantic description model of classroom videos of the deaf-mute students and enable machines to understand typical classroom interaction scenarios. The project will build learning effectiveness analysis model based on machine understandings to provide smart personalized learning support services. The project will present example analysis for proposed theoretical models and algorithms based on real data of classroom videos of the deaf-mute students. The project stands at the crossroads of artificial intelligence and education, this study has both theoretical and practical importance for special education.