Collaborative Research: GeoGaze: Gaze-Driven Adaptive Multimedia to Augment Geoscience Learning for Neurodiverse Learners

今天，许多学习都是在多媒体环境中进行的。这个项目将设计和测试GeoGaze，这是一项利用眼球追踪来实时增强多媒体学习的新技术。GeoGaze根据学生工作记忆容量的差异分析和预测有效的策略，并改变学习材料的呈现方式，帮助每个人查看和学习信息。研究者将首先研究什么样的视觉注意策略对具有不同工作记忆能力水平的学习者最有效。这些信息将被用来建立一个注视驱动的技术，帮助每个学习者使用最有效的策略，实时调整学习材料，并为每个学习者服务。这方面的例子包括预测学习者在屏幕上的位置，他们探索内容的顺序，然后突出显示要查看的顺序中最重要的元素，或者模糊不相关的内容，然后在适当的时候显示。该项目的研究结果将有助于我们了解视觉注意和工作记忆容量的个体差异如何影响多媒体学习，以及如何调整学习材料以实时提高学习效果。该项目旨在吸引更多的学生学习地球科学，这是一个学生和研究人员多样性不足的领域。传统上对多媒体学习的研究主要集中在最终的学习结果上，但对学习过程中发生的事情（如整合多媒体的视觉注意策略）的研究却很少。该项目解决了这些缺点，以获得一个强有力的视角，了解具有不同工作记忆能力的学生如何在多媒体环境中学习真实的STEM内容（地球科学），以及如何使用注视驱动的自适应学习技术GeoGaze来增强他们的认知。该方法基于行为和心理生理数据的新颖整合以及多层反向传播神经网络分析来预测媒体整合策略。GeoGaze代表了一种新颖的注视驱动技术，它基于对每个学习者有效的媒体整合策略来实时适应多媒体学习。本研究将为使用实时注视驱动的学习材料适应来增强认知和改善学习提供新的经验证据。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Much of learning occurs today in multimedia environments. This project will design and test GeoGaze, a novel technology that uses eye tracking to augment learning with multimedia in real time. GeoGaze analyzes and predicts effective strategies for students based on differences in working memory capacity and changes the presentation of learning materials to help everyone view and learn information. Investigators will first study what visual attention strategies work best for learners with different levels of working memory capacity. This information will be used to build a gaze-driven technology to help each learner use the most effective strategies by adapting the learning materials in real time and for each learner. Examples of this include predicting where learners will look on the screen, what order they will explore the content, and then highlighting the most important elements in the sequence to be viewed or blurring the less relevant content before revealing when appropriate. The results of this project will help us understand how individual differences in visual attention and working memory capacity influence multimedia learning and how to adapt learning materials to improve learning in real time. This project seeks to engage more students in learning geoscience, a field not well represented by a diversity of students and researchers.Studies of multimedia learning traditionally focus on final learning outcomes but what happens during the learning process (e.g., visual attention strategies to integrate multimedia) is often unexplored. This project addresses these shortcomings to gain a robust perspective of how students with different working memory capacities learn authentic STEM content (geoscience) in multimedia environments, and how their cognition can be enhanced using gaze-driven adaptive learning technology, GeoGaze. This approach is based on a novel integration of behavioral and psychophysiological data as well as multi-layer backpropagation neural network analysis to predict media integration strategies. GeoGaze represents a novel gaze-driven technology for adapting multimedia learning in real time based on effective media integration strategies for each learner. This research will generate new empirical evidence for using real-time gaze-driven adaptation of learning materials to augment cognition and improve learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.