CHS: Small: Enhancing Educational Virtual Reality with Headset-based Eye Tracking

CHS：小型：通过基于耳机的眼动追踪增强教育虚拟现实

基本信息

批准号：
1815976
负责人：
Christoph Borst
金额：
$ 49.98万
依托单位：
University of Louisiana at Lafayette
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2018
资助国家：
美国
起止时间：
2018-08-01 至 2023-07-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1815976&HistoricalAwards=false
关键词：
CHS Small Enhancing Educational Virtual

项目摘要

Virtual reality (VR) can bring lab or field-trip-like experiences to students who are unable to visit physical sites because of location, budget, or schedule. Potential advantages of these experiences over traditional teaching tools include increased student engagement and motivation, more direct viewing of size and spatial relationships of modeled objects, and stronger memories of the content. Emerging consumer VR devices are starting to provide sufficient quality and affordability for home and school use, and this will eventually make educational VR experiences broadly available. Future consumer VR headsets are expected to include increased sensing, such as eye tracking cameras to determine where users are looking and strain gauges to detect facial expressions. The sensor data can be analyzed for insight into users' attention and emotional affect. The project will investigate how such insight into student attention can be used to improve educational VR through the design of personalized educational environments that respond to individual students' attention. The project will also develop techniques for using sensor data to give teachers enhanced real-time insight into student activities and behavior patterns to help them provide better teacher-guided VR experiences. This will involve development of new approaches for educational VR technology and experiments that generate fundamental knowledge and guidelines for applying such approaches. In addition to the potential long-term benefit of improving education, the project will provide a number of more immediate, direct educational benefits. The team will incorporate the work into courses and undergraduate research experiences on human-computer interaction and VR, as well as outreach activities and summer programs aimed at high school students across Louisiana. The team will design and assess methods including the following: 1) educational content that responds to student eye gaze for more responsive and engaging presentation; 2) visual effects or indicators, based on detected eye behaviors, to encourage student attention to particular content in a VR environment; and 3) visualizations of student eye gaze that use both raw and processed gaze data to help teachers understand and guide students. To understand the tradeoffs between approaches and to develop guidelines for wider development and use of these techniques, effects will be studied in terms of behavior, subjective experience, and learning. The most promising methods will be applied to a case study of a networked VR interface that allows teachers to monitor and guide students through an immersive educational VR environment. To do this, the team will build on their existing educational VR framework that has previously been deployed at regional high schools and to thousands of students at outreach events. The project is expected to improve the effectiveness of such VR systems and of teachers' ability to supervise and assist students. Resulting methods and principles will provide a foundation for headset-based eye tracking in educational VR and in other related applications such as simulation-based training and accessibility.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

虚拟现实（VR）可以将实验室或野外旅行的经验带给因位置，预算或时间表而无法访问实体站点的学生。这些经验比传统教学工具的潜在优势包括增加学生的参与度和动力，更直接地观察建模对象的大小和空间关系以及对内容的更强烈的记忆。新兴的消费者VR设备开始为家庭和学校使用提供足够的质量和负担能力，这最终将使教育的VR体验广泛可用。预计未来的消费者VR耳机将包括增加的感应，例如眼睛跟踪摄像头，以确定用户的位置和应变表以检测面部表情。可以分析传感器数据，以了解用户的注意力和情感影响。该项目将通过设计个性化的教育环境来调查对学生注意力的这种洞察力如何来改善教育VR，从而响应个人学生的注意力。该项目还将开发用于使用传感器数据为教师增强对学生活动和行为模式的实时见解的技术，以帮助他们提供更好的教师指导的VR体验。这将涉及开发用于教育VR技术的新方法，并实验产生基本知识和应用此类方法的准则。除了改善教育的潜在长期利益外，该项目还将提供许多更直接的直接教育福利。该团队将将这项工作纳入有关人类计算机互动和VR的课程和本科研究经验，以及针对路易斯安那州高中生的外展活动和夏季课程。该团队将设计和评估方法，包括以下方法：1）对学生眼睛注视的响应，以进行更敏感和引人入胜的演示； 2）基于检测到的眼睛行为的视觉效果或指标，以鼓励学生在VR环境中注意特定内容； 3）使用原始和加工的凝视数据来帮助教师理解和指导学生的学生眼目光的可视化。要了解方法之间的权衡并制定了这些技术的更广泛发展和使用指南，将根据行为，主观经验和学习来研究效果。最有希望的方法将应用于网络VR接口的案例研究，该案例研究使教师可以监视和指导学生通过沉浸式教育VR环境。为此，该团队将建立在他们现有的教育VR框架上，该框架以前已在区域高中部署，并在宣传活动中为成千上万的学生部署。预计该项目将提高此类VR系统的有效性以及教师监督和协助学生的能力。最终的方法和原理将为教育VR中的基于耳机的眼镜跟踪以及其他相关应用（例如基于模拟的培训和可访问性）提供基础。该奖项反映了NSF的法定任务，并认为值得通过基金会的知识分子优点和更广泛的影响来通过评估来提供支持。

项目成果

期刊论文数量（30）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

A Study of Class Meetings in VR: Student Experiences of Attending Lectures and of Giving a Project Presentation

VR 课堂会议研究：学生参加讲座和进行项目演示的体验

DOI：
10.3389/frvir.2021.648619
发表时间：
2021
期刊：
Frontiers in Virtual Reality
影响因子：
0
作者：
Yoshimura, Andrew;Borst, Christoph W.
通讯作者：
Borst, Christoph W.

Exploring Eye Gaze Visualization Techniques for Identifying Distracted Students in Educational VR

探索用于识别教育 VR 中分心学生的眼睛注视可视化技术

DOI：
10.1109/vr46266.2020.00009
发表时间：
2020
期刊：
2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR
影响因子：
0
作者：
Rahman, Yitoshee;Asish, Sarker M.;Fisher, Nicholas P.;Bruce, Ethan C.;Kulshreshth, Arun K.;Borst, Christoph W.
通讯作者：
Borst, Christoph W.

Evaluation of Headset-based Viewing and Desktop-based Viewing of Remote Lectures in a Social VR Platform

社交 VR 平台中基于耳机的远程讲座观看和基于桌面的观看评估

DOI：
10.1145/3385956.3422124
发表时间：
2020
期刊：
VRST '20: 26th ACM Symposium on Virtual Reality Software and Technology
影响因子：
0
作者：
Yoshimura, Andrew;Borst, Christoph W.
通讯作者：
Borst, Christoph W.

Detecting Distracted Students in an Educational VR Environment Utilizing Machine Learning on EEG and Eye-Gaze Data

利用脑电图和眼睛注视数据的机器学习来检测教育 VR 环境中分心的学生

DOI：
10.1109/vrw58643.2023.00194
发表时间：
2023
期刊：
IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops
影响因子：
0
作者：
Asish, Sarker M.;Kulshreshth, Arun K;Borst, Christoph W
通讯作者：
Borst, Christoph W

Modifying Pedagogical Agent Spatial Guidance Sequences to Respond to Eye-Tracked Student Gaze in VR

修改教学代理空间引导序列以响应 VR 中的眼球追踪学生注视