EAGER: Mixed Reality for Improved Science Reading Comprehension

探索性研究（EAGER）的早期拨款是东卡罗莱纳大学（ECU）、火花塞游戏、坎伯兰县学校和Teresa C Berrien小学之间的合作伙伴关系，将开发和执行混合现实（MR）的初步测试，以提高对小学科学教科书中学术语言的理解。理解科学词汇对于小学生学习和享受科学至关重要，但许多学生未能充分理解科学素养所需的基础科学词汇。混合现实（MR）是增强现实（AR）的扩展，虚拟对象和体验覆盖或叠加在物理世界上。在MR中，用户可以与虚拟物体以及物理世界进行交互。该项目计划在适当的时间激活科学文本中的虚拟内容，以帮助学习者阅读和理解新词汇的技术。磁共振技术将使科学阅读模块的科学内容通过3D图像、动态图表和声音变得生动起来。目标是通过使用混合现实（MR）开发引人入胜的个性化学习系统来提高学生对科学文本的理解。该项目是一项探索性研究，旨在开发和进行原型系统的初步测试，以证明其有用性和有效性。它将确定设计和开发MR学习环境所需要的东西，特别是侧重于提高小学高年级学生的科学阅读理解。该项目将引领高质量MR学习体验的发展，并最终开发出可负担得起的产品，适合在世界各地的小学、图书馆和家庭中更广泛地实施。在设计和开发的迭代方法中利用定性和定量数据，该EAGER赠款准备创建和测试基于Unity游戏引擎的工作原型，并通过微软HoloLens眼镜部署。阅读模块将与北卡罗来纳五年级科学标准课程的内容紧密结合，用学术语言和困难的词汇传达具有挑战性的抽象概念。该技术将在适当的时间激活科学文本中的内容，以帮助学习者阅读和理解新词汇。适当的时间将在操作上定义为当读者的目光追踪到预定的科学词汇术语（例如，势能）时。这部分工作需要Sparkplug Games创建一个软件集成层，将多个经过验证但离散的技术整合到一个系统中。头戴式显示器将监控学生凝视的方向和持续时间，以告知系统当前重要的内容。这项研究将分两个阶段进行。在第一阶段，硬件和软件将被集成以创建MR增强科学模块。这一阶段将包括收集学生数据（朗读和大声思考）和课程专家/教师访谈，以告知文本的增强（例如，哪些概念应该增强，多少解释是有帮助的，学生如何激活和不激活进一步的解释）。措施包括“大声思考”和研究人员的观察，以确定哪些词汇和概念对学生来说是最难处理的，并与教师和地区工作人员面谈，以使材料与北卡罗来纳和其他标准保持一致。第二阶段将在五年级学生（N = 20-30）中测试和迭代改进原型MR体验，以确保该技术能够顺利运行，并为低于、等于或高于年级的学生提供易于访问和引人入胜的体验。测量方法将包括使用磁共振增强文本的“大声思考”，观察对比使用和不使用磁共振增强文本的阅读，学生和教师访谈，以及使用非参数统计方法分析口语阅读流畅性测试前和后。将选择不同的参与者来测试原型，这些参与者反映了五年级学生的阅读技能、性别、种族和民族多样性。五年级教师和其他学校和地区的工作人员，如教学教练和行政人员也将参与其中——他们的观点将在设计过程中被征求，以创造一个MR体验，与他们期望教授的内容保持一致，并在课堂实施方面是可行的。该项目预计将在三个方面产生广泛的社会影响。首先，它将开发一个使用数据开发MR材料的框架，这些材料将科学学习和兴趣与有效的初级识字实践联系起来，并补充在某些环境中难以有效实施的传统科学教学工具（如工具包）。其次，该项目旨在为教师开发课堂工具，以令人兴奋的方式使用新兴技术，提高不同学生的参与度并改善学习成果。第三，该项目旨在培养在未来STEM学习中取得成功的基本技能。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This EArly grant for Exploratory Research (EAGER) is a partnership between East Carolina University (ECU), Spark Plug Games, Cumberland County Schools, and Teresa C Berrien Elementary School that will develop and perform preliminary testing of mixed reality (MR) to improve understanding of academic language in elementary science text. Understanding science vocabulary is crucial for elementary school students to learn and enjoy science but many students fail to fully comprehend fundamental science vocabulary needed for scientific literacy. Mixed reality (MR) is an extension of augmented reality (AR) in that the virtual objects and experiences overlay or are superimposed on the physical world. In MR, the user can interact with the virtual objects as well as the physical world. This project plans technology that will activate virtual content in science texts at the appropriate time to assist the learner in reading and understanding new vocabulary. The MR technology will make the science content of a science reading module come alive with 3D images, moving diagrams, and sound. The goal is to improve students' comprehension of science text by using mixed reality (MR) to develop engaging personalized learning systems. The project is an exploratory study to develop and conduct preliminary tests of a prototype system meant to demonstrate its usefulness and effectiveness. It will be determined what is needed to design and develop MR learning environments, specifically focused on improving upper elementary grade students' science reading comprehension. The project will lead the way for development of quality MR learning experiences and eventually for affordable accessible products suitable for a broader implementation in elementary schools, libraries, and homes across the world.Utilizing both qualitative and quantitative data in an iterative approach to design and development, this EAGER grant is poised to create and test a working prototype built on the Unity Game Engine and deployed through Microsoft HoloLens glasses. A reading module will be chosen that closely aligns with the content in the North Carolina Standard Course of Study for 5th grade science and conveys challenging abstract concepts with academic language and difficult vocabulary. The technology will activate content in science texts at the appropriate time to assist the learner in reading and understanding new vocabulary. The appropriate time will be operationally defined as when the reader's eye track over a predetermined science vocabulary term (e.g., Potential Energy). This part of the effort will require the creation of a software integration layer by Sparkplug Games to stitch together several proven but discrete technologies into one system. The head mounted display will monitor the student's direction and duration of gaze to inform the system of what content is currently important. The research will take place in two phases. In the first phase, hardware and software will be integrated to create the MR enhanced science module. This phase will include collecting student data (read and think aloud) and curriculum specialist/teacher interviews to inform the enhancement of the text (e.g. which concepts should be enhanced, how much explanation is helpful, how does the student activate and deactivate further explanations). Measures will include "Think Aloud" and researcher observations to determine which vocabulary and concepts are most difficult for students to process and interviews with teachers and district personnel to align the materials to North Carolina and other standards. The second phase will test and iteratively improve the prototype MR experience with 5th grade students (N = 20-30) to ensure that the technology works smoothly and provides an accessible and engaging experience for students reading below, at, and above grade level. Measures will include "Think Aloud" use of MR enhanced text, observation to compare reading with and without MR enhanced text, student and teacher interviews, and a pre- and post- Oral Reading Fluency test analyzed using non-parametric statistical measures. Diverse participants will be selected to test the prototype with individuals reflecting the spectrum of reading skills, gender, racial, and ethnic diversity of the 5th graders. Fifth grade teachers and other school and district staff such as instructional coaches and administrators will also participate -- their perspectives will be solicited during the design process in order to create an MR experience that aligns to the content they are expected to teach and is workable in terms of classroom implementation. This project is expected to have a broad societal impact in three ways. First, it will develop a framework for using data to develop MR materials that bridge science learning and interest into effective elementary literacy practices and supplement traditional science instructional tools (e.g. kits), which have been difficult to implement effectively in some environments. Second, this project aims to develop classroom tools for teachers to use emerging technology in exciting ways, increasing engagement and improving learning outcomes for diverse students. Third, this project aims to develop skills that are fundamental to success in future STEM learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.