Collaborative Research: GeoGaze: Gaze-Driven Adaptive Multimedia to Augment Geoscience Learning for Neurodiverse Learners

今天，很多学习都是在多媒体环境中进行的。这个项目将设计和测试GeoGaze，这是一项新技术，使用眼球跟踪来实时通过多媒体增强学习。GeoGaze根据工作记忆容量的差异为学生分析和预测有效的策略，并改变学习材料的呈现方式，帮助每个人查看和学习信息。研究人员将首先研究哪些视觉注意策略对不同工作记忆能力水平的学习者最有效。这些信息将被用来建立一种凝视驱动的技术，通过实时为每个学习者调整学习材料来帮助每个学习者使用最有效的策略。这方面的例子包括预测学习者将在屏幕上看哪里，他们将以什么顺序浏览内容，然后突出显示要查看的序列中最重要的元素，或者在适当的时候显示之前模糊不太相关的内容。这个项目的结果将帮助我们理解视觉注意力和工作记忆容量的个体差异如何影响多媒体学习，以及如何调整学习材料以提高实时学习能力。这个项目旨在让更多的学生参与到地球科学的学习中来，这个领域的学生和研究人员的多样性并不是很好地代表了这个领域。多媒体学习的研究传统上侧重于最终的学习结果，但在学习过程中发生了什么(例如，整合多媒体的视觉注意策略)往往没有被探索。这个项目解决了这些缺点，以获得一个稳健的视角，了解具有不同工作记忆能力的学生如何在多媒体环境中学习真实的STEM内容(地球科学)，以及如何使用凝视驱动的自适应学习技术GeoGaze来增强他们的认知。该方法基于一种新的行为和心理生理数据的集成以及多层反向传播神经网络分析来预测媒体整合策略。GeoGaze代表了一种新颖的凝视驱动技术，用于根据每个学习者的有效媒体整合策略实时调整多媒体学习。这项研究将为使用实时凝视驱动的学习材料改编来增强认知和改善学习产生新的实证证据。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Much of learning occurs today in multimedia environments. This project will design and test GeoGaze, a novel technology that uses eye tracking to augment learning with multimedia in real time. GeoGaze analyzes and predicts effective strategies for students based on differences in working memory capacity and changes the presentation of learning materials to help everyone view and learn information. Investigators will first study what visual attention strategies work best for learners with different levels of working memory capacity. This information will be used to build a gaze-driven technology to help each learner use the most effective strategies by adapting the learning materials in real time and for each learner. Examples of this include predicting where learners will look on the screen, what order they will explore the content, and then highlighting the most important elements in the sequence to be viewed or blurring the less relevant content before revealing when appropriate. The results of this project will help us understand how individual differences in visual attention and working memory capacity influence multimedia learning and how to adapt learning materials to improve learning in real time. This project seeks to engage more students in learning geoscience, a field not well represented by a diversity of students and researchers.Studies of multimedia learning traditionally focus on final learning outcomes but what happens during the learning process (e.g., visual attention strategies to integrate multimedia) is often unexplored. This project addresses these shortcomings to gain a robust perspective of how students with different working memory capacities learn authentic STEM content (geoscience) in multimedia environments, and how their cognition can be enhanced using gaze-driven adaptive learning technology, GeoGaze. This approach is based on a novel integration of behavioral and psychophysiological data as well as multi-layer backpropagation neural network analysis to predict media integration strategies. GeoGaze represents a novel gaze-driven technology for adapting multimedia learning in real time based on effective media integration strategies for each learner. This research will generate new empirical evidence for using real-time gaze-driven adaptation of learning materials to augment cognition and improve learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.