Using Augmented Reality to Enhance Attention in STEM Learning for Students with Executive Function Disabilities

使用增强现实技术提高执行功能障碍学生在科学、技术、工程和数学学习中的注意力执行功能代表了一套广泛的技能，是STEM学习和学业成功的核心，包括与注意力、持久性、情绪调节和抑制控制相关的技能。对于有执行功能障碍的学生来说，在作业和其他独立学习任务中集中注意力是一个挑战。许多现有的辅助技术补充学生的组织能力是昂贵的和排他性的，并且依赖于学生组织或启动他们自己的任务监控的能力。该项目提供了一种创新的、可扩展的干预措施，通过使用增强现实技术来支持个人自我监控注意力的能力，并在他们离开任务时重新参与内容，使那些在执行功能方面挣扎的个人能够在学术和工作环境中坚持并茁壮成长。从长远来看，该系统有可能成为一种广泛使用的学习技术，为有或没有执行功能障碍的学生改善学习效果。该项目将通过增加目前在STEM教育和STEM劳动力中代表性不足的人口的参与来服务于公众利益。本项目将对计算机科学和学习科学做出新的贡献，通过两个总体项目目标来实现。首先，通过对头部位置、头部方向和凝视方向的时间分析，项目团队将开发一个基于深度学习算法的开源工具，该工具可以检测本科生在做数学作业时的非任务行为。其次，团队将使用增强现实技术，以重新定向或休息的形式为学生提供适当的反馈，这些反馈与他们自己的注意力和分心程度有关。该系统的开发将涉及一个迭代原型和测试过程，并将以早期功效研究结束。为了探索最有效的提示，有执行功能问题的学生将作为系统的共同设计者参与其中。用于检测非任务行为的深度学习算法是对计算机科学领域的一项新贡献，在其他学科领域和其他类型的学习困难中具有潜在的应用前景。此外，由此产生的自我监控和重定向提示是对学习科学的贡献，可以使大多数学习者受益。因此，这项研究将是增强现实应用于学习的重大发展，因为它探索了在复杂问题空间中学生与教育技术互动的最有效手段。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Using Augmented Reality to Enhance Attention in Science, Technology, Engineering and Mathematics Learning for Students with Executive Function DisabilitiesExecutive functioning represents a broad skill set that is central to STEM learning and academic success, including skills related to attention, persistence, emotion regulation, and inhibition control. For students with executive function difficulties, focusing during homework and other independent learning tasks is a challenge. Many existing assistive technologies supplementing a student’s organizational capabilities are expensive and exclusionary and rely on students’ ability to organize or initiate their own task monitoring. This project provides an innovative, scalable intervention that enables individuals who struggle with executive functioning to persist and thrive in academic and workplace settings by using Augmented Reality technology to support an individual’s ability to self-monitor their attention and re-engage with the content when they are off task. In the long term, the system has the potential to become a widely used learning technology that improves outcomes for students with and without executive functioning difficulties. The project will serve the public interest by increasing the participation of a population that is currently underrepresented in STEM education and the STEM workforce.This project will make a novel contribution to both the computer sciences and the learning sciences, achieved through two overarching project goals. First, using a temporal analysis of head position, head orientation, and gaze orientation, the project team will develop an open-source tool based on deep learning algorithms that can detect off-task behavior as undergraduate students work on math homework problems. Second, the team will use augmented reality to provide appropriate feedback to students in the form of redirection or breaks related to their own level of focus and distractibility. Development of this system will involve an iterative prototyping and testing process, and will conclude with an early efficacy study. To explore the most effective prompts, students with executive functioning issues will participate as co-designers of the system. The deep learning algorithm for the detection of off-task behavior is a novel contribution to the computer science field that has potential applications for other subject areas and other types of learning difficulties. Furthermore, the resultant self-monitoring and redirection prompts are a contribution to the learning sciences that can benefit most learners. Thus, this research will be a significant development in the application of augmented reality to learning as it explores the most productive means of student interaction with educational technology in a complex problem space.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.