Collaborative Research: Haptic Technology Innovations to Guide Student Gesturing with Dynamic STEM Visualizations

该项目旨在确定传递到学生手中的感官反馈如何帮助他们理解计算机模拟中显示的图片、图表和其他表示的含义。为了实现这一目标，项目团队将设计和开发一种定制的触觉手套，与生物细胞的计算机可视化相互作用。与之前的研究不同，该项目旨在通过引导理科生思考当他们在生物细胞的可视化图像上移动时，他们的手所感受到的意义，来构建理科生的学习。生物学的背景很重要，因为它将学生的学习置于微观和亚微观学科中，在这些学科中，我们对触觉对学习的影响的了解远远少于我们对使用大型现实世界物体学习的了解。许多过去的研究已经证明，在类似的科学背景下，使用身体和手来学习可以有所帮助，但这个项目旨在通过使用一种新颖的神经成像方法（功能性近红外光谱或fNIRS）来测试身体本身有助于学生学习的说法，该方法允许学生在学习时移动他们的身体，然后确定大脑的运动区域在以后解决问题时是否保持活跃。该项目为科学教育提供了附加价值，因为它将确定何时以及如何将特定的手部动作（即手势）与触觉手套的反馈相结合，使科学学习超越手势或感官反馈（即触摸）。这些知识将有助于解决学习科学中的一个突出问题，该问题涉及何时以及如何使用符号表示（例如，方程或数字）或其他具体表示（例如，塑料手持模型）。该项目采用混合方法，旨在解决计算机可视化的局限性，即无法向学生传递超出他们感官的小世界的感觉。此外，该项目团队旨在生产的技术将大大降低商用手套的价格。最后，该项目汇集了一个多学科团队，将成果传播给包括科学教师、私营企业和学习型科学家在内的不同利益相关者。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project aims to determine how sensory feedback delivered to students’ hands helps them understand the meanings of pictures, graphs, and other representations displayed in computer simulations. To accomplish this aim, the project team will design and develop a custom haptic glove that interacts with a computer visualization of a biological cell. Unlike prior research, the project aims to structure science students’ learning by guiding them to think about the meaning of what their hands feel when they move across a visualization of a biological cell. The context of biology matters because it situates students’ learning in micro- and sub-microscopic disciplines where we know much less about the effects of haptics on learning than we know about learning with large, real-world objects. Many past studies have demonstrated that using the body and hands to learn can help in similar science contexts, but this project aims to additionally test the claim that the body itself helps students learning by using a novel neuroimaging method (functional Near Infrared Spectroscopy or fNIRS) that allows students to move their bodies while learning and then determine if motor areas of the brain remain active during later problem solving. This project delivers added value to science education because it will determine when and how specific hand movements (i.e., gestures) combined with feedback from the haptic glove enable science learning above and beyond gestures or sensory feedback (i.e., touch) alone. This knowledge will help address an outstanding problem in the science of learning that involves when and how to use symbolic representations (e.g., equations or numbers) or else concrete representations (e.g., plastic hand-held models). The project adopts a blended approach that aims to address limitations of computer visualizations that fail to deliver to students a feeling for the small world beyond their senses. Moreover, the technology the project team aims to produce will result in a significant cost reduction to the price of commercial gloves. Last, the project brings together a multi-disciplinary team that will disseminate results to diverse stakeholders that include science instructors, private industry, and learning scientists.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.