EAGER: Mixed Reality for Improved Science Reading Comprehension

EARLY的探索性研究（EAGER）拨款是东卡罗莱纳大学（ECU），Spark Plug Games，坎伯兰县学校和Teresa C Berrien小学之间的合作伙伴关系，将开发和执行混合现实（MR）的初步测试，以提高对基础科学文本中学术语言的理解。理解科学词汇对小学生学习和享受科学至关重要，但许多学生未能充分理解科学素养所需的基本科学词汇。混合现实（MR）是增强现实（AR）的扩展，其中虚拟对象和体验覆盖或叠加在物理世界上。在MR中，用户可以与虚拟对象以及物理世界进行交互。 该项目计划在适当的时候激活科学文本中的虚拟内容，以帮助学习者阅读和理解新词汇。MR技术将使科学阅读模块的科学内容变得生动，具有3D图像，移动图表和声音。 我们的目标是通过使用混合现实（MR）开发引人入胜的个性化学习系统来提高学生对科学文本的理解。该项目是一项探索性研究，旨在开发和初步测试一个原型系统，以证明其实用性和有效性。将确定需要设计和开发MR学习环境，特别是专注于提高小学高年级学生的科学阅读理解。该项目将引领高质量MR学习体验的发展，并最终开发出适合在世界各地的小学、图书馆和家庭中广泛实施的负担得起的无障碍产品。在设计和开发的迭代方法中利用定性和定量数据，这项EAGER基金准备创建和测试一个基于Unity游戏引擎并通过Microsoft HoloLens眼镜部署的工作原型。 一个阅读模块将被选择，密切配合在学习的北卡罗来纳州标准课程的内容为五年级的科学和传达具有挑战性的抽象概念与学术语言和困难的词汇。该技术将在适当的时候激活科学文本中的内容，以帮助学习者阅读和理解新词汇。适当的时间将在操作上被定义为当读者的眼睛跟踪预定的科学词汇术语（例如，势能）。这部分工作将需要Sparkplug Games创建一个软件集成层，将几个经过验证但独立的技术缝合到一个系统中。头戴式显示器将监视学生的方向和注视的持续时间，以通知系统当前重要的内容。 研究将分两个阶段进行。在第一阶段，硬件和软件将被集成，以创建MR增强科学模块。这一阶段将包括收集学生数据（朗读和思考）和课程专家/教师访谈，以告知文本的增强（例如，哪些概念应该增强，解释有多大帮助，学生如何激活和停用进一步的解释）。措施将包括“大声思考”和研究人员的观察，以确定哪些词汇和概念是最困难的学生处理和采访教师和地区人员，以调整材料，以北卡罗来纳州和其他标准。第二阶段将测试并迭代改进5年级学生（N = 20-30）的原型MR体验，以确保该技术顺利工作，并为低于年级、处于年级和高于年级的学生阅读提供可访问和引人入胜的体验。措施将包括“大声思考”使用MR增强文本，观察比较阅读与和没有MR增强文本，学生和教师访谈，以及前和后口语阅读流畅性测试分析使用非参数统计措施。不同的参与者将被选择来测试原型与个人反映频谱的阅读技能，性别，种族和民族多样性的五年级学生。五年级教师以及其他学校和学区工作人员（例如教学教练和管理人员）也将参与其中--在设计过程中将征求他们的观点，以创造与他们预期教授的内容相一致且可行的MR体验。课堂实施。预计该项目将在三个方面产生广泛的社会影响。首先，它将制定一个框架，利用数据开发MR材料，将科学学习和兴趣与有效的小学识字实践联系起来，并补充传统的科学教学工具（例如工具包），这些工具在某些环境中难以有效实施。其次，该项目旨在为教师开发课堂工具，以令人兴奋的方式使用新兴技术，提高参与度，改善不同学生的学习成果。第三，该项目旨在培养对未来STEM学习成功至关重要的技能。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This EArly grant for Exploratory Research (EAGER) is a partnership between East Carolina University (ECU), Spark Plug Games, Cumberland County Schools, and Teresa C Berrien Elementary School that will develop and perform preliminary testing of mixed reality (MR) to improve understanding of academic language in elementary science text. Understanding science vocabulary is crucial for elementary school students to learn and enjoy science but many students fail to fully comprehend fundamental science vocabulary needed for scientific literacy. Mixed reality (MR) is an extension of augmented reality (AR) in that the virtual objects and experiences overlay or are superimposed on the physical world. In MR, the user can interact with the virtual objects as well as the physical world. This project plans technology that will activate virtual content in science texts at the appropriate time to assist the learner in reading and understanding new vocabulary. The MR technology will make the science content of a science reading module come alive with 3D images, moving diagrams, and sound. The goal is to improve students' comprehension of science text by using mixed reality (MR) to develop engaging personalized learning systems. The project is an exploratory study to develop and conduct preliminary tests of a prototype system meant to demonstrate its usefulness and effectiveness. It will be determined what is needed to design and develop MR learning environments, specifically focused on improving upper elementary grade students' science reading comprehension. The project will lead the way for development of quality MR learning experiences and eventually for affordable accessible products suitable for a broader implementation in elementary schools, libraries, and homes across the world.Utilizing both qualitative and quantitative data in an iterative approach to design and development, this EAGER grant is poised to create and test a working prototype built on the Unity Game Engine and deployed through Microsoft HoloLens glasses. A reading module will be chosen that closely aligns with the content in the North Carolina Standard Course of Study for 5th grade science and conveys challenging abstract concepts with academic language and difficult vocabulary. The technology will activate content in science texts at the appropriate time to assist the learner in reading and understanding new vocabulary. The appropriate time will be operationally defined as when the reader's eye track over a predetermined science vocabulary term (e.g., Potential Energy). This part of the effort will require the creation of a software integration layer by Sparkplug Games to stitch together several proven but discrete technologies into one system. The head mounted display will monitor the student's direction and duration of gaze to inform the system of what content is currently important. The research will take place in two phases. In the first phase, hardware and software will be integrated to create the MR enhanced science module. This phase will include collecting student data (read and think aloud) and curriculum specialist/teacher interviews to inform the enhancement of the text (e.g. which concepts should be enhanced, how much explanation is helpful, how does the student activate and deactivate further explanations). Measures will include "Think Aloud" and researcher observations to determine which vocabulary and concepts are most difficult for students to process and interviews with teachers and district personnel to align the materials to North Carolina and other standards. The second phase will test and iteratively improve the prototype MR experience with 5th grade students (N = 20-30) to ensure that the technology works smoothly and provides an accessible and engaging experience for students reading below, at, and above grade level. Measures will include "Think Aloud" use of MR enhanced text, observation to compare reading with and without MR enhanced text, student and teacher interviews, and a pre- and post- Oral Reading Fluency test analyzed using non-parametric statistical measures. Diverse participants will be selected to test the prototype with individuals reflecting the spectrum of reading skills, gender, racial, and ethnic diversity of the 5th graders. Fifth grade teachers and other school and district staff such as instructional coaches and administrators will also participate -- their perspectives will be solicited during the design process in order to create an MR experience that aligns to the content they are expected to teach and is workable in terms of classroom implementation. This project is expected to have a broad societal impact in three ways. First, it will develop a framework for using data to develop MR materials that bridge science learning and interest into effective elementary literacy practices and supplement traditional science instructional tools (e.g. kits), which have been difficult to implement effectively in some environments. Second, this project aims to develop classroom tools for teachers to use emerging technology in exciting ways, increasing engagement and improving learning outcomes for diverse students. Third, this project aims to develop skills that are fundamental to success in future STEM learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.