Automatic assessment of cognitive and emotional states in virtual reality‐based flexibility training for four adolescents with autism
Automatic assessment of cognitive and emotional states in virtual reality‐based flexibility training for four adolescents with autism
复制标题
基于虚拟现实的灵活性训练对四名自闭症青少年的认知和情绪状态进行自动评估
DOI：
10.1111/bjet.13005
复制
发表时间：
2020
影响因子：
6.6
通讯作者：
Sokolikj, Zlatko
中科院分区：
文献类型：
--
作者：
Moon, Jewoong;Ke, Fengfeng;Sokolikj, Zlatko
跟踪学生的学习状态以提供定制的学习者支持是自适应学习系统的关键要素。本研究探讨了自动评估如何能够在基于虚拟现实（VR）的表征灵活性训练中跟踪学习者的认知和情绪状态。这个基于VR的培训计划旨在促进自闭症谱系障碍（ASD）青少年在STEM相关设计问题解决过程中解释，选择和创建多模态表示的灵活性。对于自动评估，我们使用自然语言处理（NLP）和机器学习技术来开发多标签分类模型。然后，我们用来自四名ASD青少年的66次音频和视频训练课程的数据训练模型。为了验证模型，我们实施了k折交叉验证和专家评审员的手动评估。研究结果表明，实施NLP和机器学习驱动的自动评估来跟踪和评估ASD患者在基于VR的灵活性训练期间的认知和情绪状态是可行的。研究结果还表明，在高度互动的数字学习环境中，提供适应性支持以保持学习者的认知和情感参与的重要性和可行性。
Tracking students’ learning states to provide tailored learner support is a critical element of an adaptive learning system. This study explores how an automatic assessment is capable of tracking learners’ cognitive and emotional states during virtual reality (VR)‐based representational‐flexibility training. This VR‐based training program aims to promote the flexibility of adolescents with autism spectrum disorder (ASD) in interpreting, selecting and creating multimodal representations during STEM‐related design problem solving. For the automatic assessment, we used both natural language processing (NLP) and machine‐learning techniques to develop a multi‐label classification model. We then trained the model with the data from a total of audio‐ and video‐recorded 66 training sessions of four adolescents with ASD. To validate the model, we implemented both k‐fold cross‐validations and the manual evaluations by expert reviewers. The study finding suggests the feasibility of implementing the NLP and machine‐learning driven automatic assessment to track and assess the cognitive and emotional states of individuals with ASD during VR‐based flexibility training. The study finding also denotes the importance and viability of providing adaptive supports to maintain learners’ cognitive and affective engagement in a highly interactive digital learning environment.