Multimodal assessment of learning through educational games

与UNLTD Inc.合作，拟议的研究项目将创建一个框架，通过游戏化远程向儿童提供交互式教育内容。2019冠状病毒病大流行凸显了对智能平台的需求，以实现学习要素的自动交付。在这个项目中，目标是通过交互式3D（常规和增强/虚拟现实）游戏将一些核心学习元素游戏化。最近的证据表明，与传统方法相比，游戏化（即严肃游戏）具有实现和维持更高水平的学习动机和参与度的潜力。这种基于游戏的评估的一个重要方面是对学习结果的评估，这通常与准确评估学习情绪有关。在这个项目中，研究的一个主要方面将是如何通过生理（如心电图、脑电图）和行为（如面部表情、声音）数据，以稳健有效的方式评估这种学习情绪。我们将重点关注三个关键方面：1)基于人工智能的学习情绪的鲁棒多模态评估：我们将确保开发的算法可以在资源受限的环境下工作，其中一种或多种数据模式可能会缺失。2)数据驱动情绪评估的客观注释：由于AI算法是数据驱动的，我们将创建基于证据的任务驱动游戏应用程序，这可以帮助我们弥合主观和客观注释之间的差距。3)基于云的快速原型框架：我们将创建一个易于扩展的模块，可以帮助UNLTD快速有效地将项目扩展到多个游戏。拟议项目的潜在结果是一个强大而通用的框架，可以有效地提供互动教育游戏。该项目将培养2名博士，2名硕士。，以及多名研究助理和软件开发人员。该项目可以为加拿大儿童在课堂上传统学习之外的补充学习创造一个吸引人的渠道。技术创新还将增强安大略省和加拿大作为技术中心的地位，为参与该项目的hqp创造新的就业机会。

In partnership with UNLTD Inc., the proposed research project will create a framework to remotely deliver interactive educational content to children through gamification. The COVID-19 pandemic has highlighted the need for intelligent platforms for automated delivery of learning elements. In this project, the aim is to gamify some of the core learning elements through interactive 3D (regular and augmented/virtual reality) games. Recent evidence has shown that gamification (i.e., serious games) has the potential to achieve and sustain a higher level of motivation and engagement in learning compared to traditional means. An important aspect in such game-based evaluation is the assessment of learning outcomes, which is often tied to assessing learning emotions accurately. In this project, a major aspect of the research will be on how to assess such learning emotions in a robust and efficient manner through physiological (e.g. ECG, EEG) and behavioral (e.g. facial expression, voice) data. We will focus on three key aspects: 1) Robust AI-based multimodal assessment of learning emotions: we will ensure that the developed algorithms can work under resource-constrained environments where one or more modalities of data could be missing. 2) Objective annotation for data-driven assessment of emotions: Since AI algorithms are data-driven, we will create evidence-based task-driven gaming applications which can help us in bridging the gap between subjective and objective annotations. 3) Cloud-based framework for rapid prototyping: we will create an easily scalable module that can help UNLTD in scaling up the project quickly and efficiently to multiple gaming titles. The potential outcome of the proposed project is a robust and universal framework for delivering interactive educational games efficiently. The project will train two PhD, two MSc., and multiple research assistants and software developers. The project can create an engaging channel for supplemental learning for children in Canada in addition to traditional learning in the classroom. The technical innovations will also enhance Ontario and Canada's position as a technical hub, creating new job opportunities for the HQPs involved in the project.