Using Augmented Reality to Enhance Attention in STEM Learning for Students with Executive Function Disabilities

使用增强现实来提高执行功能障碍学生在科学、技术、工程和数学学习中的注意力执行功能代表了一套广泛的技能，对STEM学习和学业成功至关重要，包括与注意力、坚持不懈、情绪调节和抑制控制相关的技能。对于有执行功能障碍的学生来说，在家庭作业和其他独立学习任务中集中注意力是一种挑战。许多现有的辅助技术补充学生的组织能力是昂贵的和排他性的，依赖于学生组织或启动自己的任务监控的能力。该项目提供了一种创新的、可扩展的干预措施，通过使用增强现实技术支持个人自我监控其注意力并在下班后重新参与内容的能力，使那些在执行功能方面存在困难的个人能够在学术和工作环境中保持并茁壮成长。从长远来看，该系统有潜力成为一种广泛使用的学习技术，可以改善有执行功能障碍和没有执行功能障碍的学生的结果。该项目将通过增加目前在STEM教育和STEM劳动力中代表性不足的人口的参与来服务于公众利益。该项目将通过两个总体项目目标实现，对计算机科学和学习科学作出新的贡献。首先，利用对头部位置、头部方向和凝视方向的时间分析，该项目团队将开发一个基于深度学习算法的开源工具，该工具可以检测本科生在处理数学作业问题时的任务外行为。其次，该团队将使用增强现实技术，以重定向或中断的形式向学生提供适当的反馈，这些反馈与他们自己的注意力水平和分心程度有关。该系统的开发将涉及一个迭代的原型和测试过程，并将以早期疗效研究结束。为了探索最有效的提示，有执行功能问题的学生将作为系统的联合设计者参与进来。用于检测脱机行为的深度学习算法是对计算机科学领域的一项新贡献，对其他学科领域和其他类型的学习困难具有潜在的应用价值。此外，由此产生的自我监控和重定向提示是对学习科学的贡献，可以使大多数学习者受益。因此，这项研究将是将增强现实应用于学习的重大发展，因为它探索了学生在复杂问题空间中与教育技术交互的最有效方式。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Using Augmented Reality to Enhance Attention in Science, Technology, Engineering and Mathematics Learning for Students with Executive Function DisabilitiesExecutive functioning represents a broad skill set that is central to STEM learning and academic success, including skills related to attention, persistence, emotion regulation, and inhibition control. For students with executive function difficulties, focusing during homework and other independent learning tasks is a challenge. Many existing assistive technologies supplementing a student’s organizational capabilities are expensive and exclusionary and rely on students’ ability to organize or initiate their own task monitoring. This project provides an innovative, scalable intervention that enables individuals who struggle with executive functioning to persist and thrive in academic and workplace settings by using Augmented Reality technology to support an individual’s ability to self-monitor their attention and re-engage with the content when they are off task. In the long term, the system has the potential to become a widely used learning technology that improves outcomes for students with and without executive functioning difficulties. The project will serve the public interest by increasing the participation of a population that is currently underrepresented in STEM education and the STEM workforce.This project will make a novel contribution to both the computer sciences and the learning sciences, achieved through two overarching project goals. First, using a temporal analysis of head position, head orientation, and gaze orientation, the project team will develop an open-source tool based on deep learning algorithms that can detect off-task behavior as undergraduate students work on math homework problems. Second, the team will use augmented reality to provide appropriate feedback to students in the form of redirection or breaks related to their own level of focus and distractibility. Development of this system will involve an iterative prototyping and testing process, and will conclude with an early efficacy study. To explore the most effective prompts, students with executive functioning issues will participate as co-designers of the system. The deep learning algorithm for the detection of off-task behavior is a novel contribution to the computer science field that has potential applications for other subject areas and other types of learning difficulties. Furthermore, the resultant self-monitoring and redirection prompts are a contribution to the learning sciences that can benefit most learners. Thus, this research will be a significant development in the application of augmented reality to learning as it explores the most productive means of student interaction with educational technology in a complex problem space.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.