An Open Source Video Analytics Tool for Analyzing Learner Navigation in Immersive Simulated Environments
An Open Source Video Analytics Tool for Analyzing Learner Navigation in Immersive Simulated Environments
复制标题
DOI：
--
复制
发表时间：
2023-05
期刊：
影响因子：
--
通讯作者：
Noah Soriano;Ashkan Negahban;S. Ozden;O. Ashour
中科院分区：
文献类型：
--
作者：
Noah Soriano;Ashkan Negahban;S. Ozden;O. Ashour
在教育研究中，用户模拟交互越来越重要，因为它为基于模拟的学习和沉浸式技术的有效性提供了关键见解。研究用户模拟交互的常见方法包括实时手动分析参与者交互或通过视频记录，这是一个繁琐的过程。调查/问卷也是常用的方法，但存在主观性，只能提供定性数据。本文中提出的工具，我们称之为环境检测用户模拟交互测量（EDUSIM），是一个公开的视频分析工具，接收屏幕录制的视频输入从参与者与模拟环境进行交互，并输出统计数据相关的时间花费在预定义的领域内的模拟模型。所提出的工具利用机器学习，即多分类卷积神经网络，提供一个有效的，自动化的过程中提取这样的导航数据。EDUSIM还实现了一个二进制分类模型，以标记不完美的输入视频数据，例如指定模拟环境之外的视频帧。为了评估该工具的有效性，我们在本科数据库课程中实现了一组沉浸式基于模拟的学习（ISBL）模块，学习者在与模拟交互以完成ISBL任务时记录他们的屏幕。然后，我们使用EDUSIM工具来分析收集的视频，并将该工具的输出与手动分析视频获得的预期结果进行比较。
In educational research, user-simulation interaction is gaining importance as it provides key insights into the effectiveness of simulation-based learning and immersive technologies. A common approach to study user-simulation interaction involves manually analyzing participant interaction in real-time or via video recordings, which is a tedious process. Surveys/questionnaires are also commonly used but are open to subjectivity and only provide qualitative data. The tool proposed in this paper, which we call Environmental Detection for User-Simulation Interaction Measurement (EDUSIM), is a publicly available video analytics tool that receives screen-recorded video input from participants interacting with a simulated environment and outputs statistical data related to time spent in pre-defined areas of interest within the simulation model. The proposed tool utilizes machine learning, namely multi-classification Convolutional Neural Networks, to provide an efficient, automated process for extracting such navigation data. EDUSIM also implements a binary classification model to flag imperfect input video data such as video frames that are outside the specified simulation environment. To assess the efficacy of the tool, we implement a set of immersive simulation-based learning (ISBL) modules in an undergraduate database course, where learners record their screens as they interact with a simulation to complete their ISBL assignments. We then use the EDUSIM tool to analyze the videos collected and compare the tool’s outputs with the expected results obtained by manually analyzing the videos.