CHS: Small: Enhancing Educational Virtual Reality with Headset-based Eye Tracking

虚拟现实(VR)可以为由于位置、预算或日程安排而无法访问物理站点的学生带来类似实验室或实地旅行的体验。与传统教学工具相比，这些体验的潜在优势包括提高学生的参与度和积极性，更直接地查看建模对象的大小和空间关系，以及对内容的更强记忆。新兴的消费类VR设备正开始为家庭和学校使用提供足够的质量和负担能力，这最终将使教育VR体验得到广泛应用。未来的消费类VR头戴设备预计将包括更强的传感功能，例如用来确定用户注视方向的眼球跟踪摄像头，以及用来检测面部表情的应变计。可以对传感器数据进行分析，以洞察用户的注意力和情感。该项目将调查如何利用这种对学生注意力的洞察，通过设计回应个别学生注意力的个性化教育环境来改进教育虚拟现实。该项目还将开发使用传感器数据的技术，使教师能够增强对学生活动和行为模式的实时了解，以帮助他们提供更好的教师指导的VR体验。这将涉及为教育虚拟现实技术和实验开发新的方法，以产生应用这些方法的基本知识和指导方针。除了改善教育的潜在长期好处外，该项目还将提供一些更直接、更直接的教育好处。该团队将把这项工作纳入关于人机交互和VR的课程和本科生研究经验，以及针对路易斯安那州各地高中生的外联活动和暑期计划。该团队将设计和评估方法，包括：1)响应学生眼睛注视的教育内容，以获得更具响应性和吸引力的演示；2)基于检测到的眼睛行为，鼓励学生注意VR环境中的特定内容的视觉效果或指示器；以及3)学生眼睛凝视的可视化，使用原始和处理过的凝视数据来帮助教师理解和指导学生。为了了解方法之间的权衡，并为更广泛地开发和使用这些技术制定指导方针，将从行为、主观体验和学习方面研究影响。最有希望的方法将被应用于一个联网VR界面的案例研究，该界面允许教师通过沉浸式教育VR环境来监控和指导学生。为了做到这一点，该团队将建立在他们现有的教育VR框架的基础上，该框架之前已经部署在地区高中和数千名学生的外联活动中。该项目预计将提高这种虚拟现实系统的有效性，以及教师监督和帮助学生的能力。由此产生的方法和原则将为教育VR和其他相关应用(如基于模拟的培训和可访问性)中基于耳机的眼睛跟踪奠定基础。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Virtual reality (VR) can bring lab or field-trip-like experiences to students who are unable to visit physical sites because of location, budget, or schedule. Potential advantages of these experiences over traditional teaching tools include increased student engagement and motivation, more direct viewing of size and spatial relationships of modeled objects, and stronger memories of the content. Emerging consumer VR devices are starting to provide sufficient quality and affordability for home and school use, and this will eventually make educational VR experiences broadly available. Future consumer VR headsets are expected to include increased sensing, such as eye tracking cameras to determine where users are looking and strain gauges to detect facial expressions. The sensor data can be analyzed for insight into users' attention and emotional affect. The project will investigate how such insight into student attention can be used to improve educational VR through the design of personalized educational environments that respond to individual students' attention. The project will also develop techniques for using sensor data to give teachers enhanced real-time insight into student activities and behavior patterns to help them provide better teacher-guided VR experiences. This will involve development of new approaches for educational VR technology and experiments that generate fundamental knowledge and guidelines for applying such approaches. In addition to the potential long-term benefit of improving education, the project will provide a number of more immediate, direct educational benefits. The team will incorporate the work into courses and undergraduate research experiences on human-computer interaction and VR, as well as outreach activities and summer programs aimed at high school students across Louisiana. The team will design and assess methods including the following: 1) educational content that responds to student eye gaze for more responsive and engaging presentation; 2) visual effects or indicators, based on detected eye behaviors, to encourage student attention to particular content in a VR environment; and 3) visualizations of student eye gaze that use both raw and processed gaze data to help teachers understand and guide students. To understand the tradeoffs between approaches and to develop guidelines for wider development and use of these techniques, effects will be studied in terms of behavior, subjective experience, and learning. The most promising methods will be applied to a case study of a networked VR interface that allows teachers to monitor and guide students through an immersive educational VR environment. To do this, the team will build on their existing educational VR framework that has previously been deployed at regional high schools and to thousands of students at outreach events. The project is expected to improve the effectiveness of such VR systems and of teachers' ability to supervise and assist students. Resulting methods and principles will provide a foundation for headset-based eye tracking in educational VR and in other related applications such as simulation-based training and accessibility.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.