Collaborative Research: GeoGaze: Gaze-Driven Adaptive Multimedia to Augment Geoscience Learning for Neurodiverse Learners

今天，许多学习都发生在多媒体环境中。 该项目将设计和测试GeoGaze，这是一种新颖的技术，使用眼球跟踪来增强真实的多媒体学习。GeoGaze根据工作记忆容量的差异为学生分析和预测有效的策略，并改变学习材料的呈现方式，以帮助每个人查看和学习信息。研究人员将首先研究哪些视觉注意策略对不同工作记忆能力水平的学习者最有效。这些信息将用于建立一个凝视驱动的技术，以帮助每个学习者使用最有效的策略，通过调整学习材料在真实的时间和每个学习者。这方面的例子包括预测学习者将在屏幕上看什么地方，他们将以什么顺序探索内容，然后突出显示要查看的序列中最重要的元素或模糊不相关的内容，然后在适当的时候显示。这个项目的结果将帮助我们了解视觉注意力和工作记忆能力的个体差异如何影响多媒体学习，以及如何调整学习材料以改善真实的学习。这个项目旨在让更多的学生参与学习地球科学，这是一个学生和研究人员都不太多的领域。多媒体学习的研究传统上关注最终的学习成果，但在学习过程中发生了什么（例如，整合多媒体的视觉注意力策略）常常未被探索。该项目解决了这些缺点，以获得具有不同工作记忆能力的学生如何在多媒体环境中学习真实STEM内容（地球科学）的强大视角，以及如何使用凝视驱动的自适应学习技术GeoGaze增强他们的认知。 这种方法是基于一种新的整合行为和心理生理数据以及多层反向传播神经网络分析，以预测媒体整合策略。 GeoGaze代表了一种新颖的凝视驱动技术，用于基于针对每个学习者的有效媒体集成策略来真实的适应多媒体学习。 这项研究将产生新的经验证据，使用实时凝视驱动的适应学习材料，以增强认知和改善learning.This奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估的支持。

Much of learning occurs today in multimedia environments. This project will design and test GeoGaze, a novel technology that uses eye tracking to augment learning with multimedia in real time. GeoGaze analyzes and predicts effective strategies for students based on differences in working memory capacity and changes the presentation of learning materials to help everyone view and learn information. Investigators will first study what visual attention strategies work best for learners with different levels of working memory capacity. This information will be used to build a gaze-driven technology to help each learner use the most effective strategies by adapting the learning materials in real time and for each learner. Examples of this include predicting where learners will look on the screen, what order they will explore the content, and then highlighting the most important elements in the sequence to be viewed or blurring the less relevant content before revealing when appropriate. The results of this project will help us understand how individual differences in visual attention and working memory capacity influence multimedia learning and how to adapt learning materials to improve learning in real time. This project seeks to engage more students in learning geoscience, a field not well represented by a diversity of students and researchers.Studies of multimedia learning traditionally focus on final learning outcomes but what happens during the learning process (e.g., visual attention strategies to integrate multimedia) is often unexplored. This project addresses these shortcomings to gain a robust perspective of how students with different working memory capacities learn authentic STEM content (geoscience) in multimedia environments, and how their cognition can be enhanced using gaze-driven adaptive learning technology, GeoGaze. This approach is based on a novel integration of behavioral and psychophysiological data as well as multi-layer backpropagation neural network analysis to predict media integration strategies. GeoGaze represents a novel gaze-driven technology for adapting multimedia learning in real time based on effective media integration strategies for each learner. This research will generate new empirical evidence for using real-time gaze-driven adaptation of learning materials to augment cognition and improve learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.