Collaborative Research: Haptic Technology Innovations to Guide Student Gesturing with Dynamic STEM Visualizations

这个项目的目的是确定如何感官反馈传递到学生的手帮助他们理解的图片，图形和其他表示在计算机模拟显示的含义。为了实现这一目标，项目团队将设计和开发一种定制的触觉手套，它可以与生物细胞的计算机可视化进行交互。与之前的研究不同，该项目旨在通过引导学生思考当他们在生物细胞的可视化中移动时，他们的手感觉到的意义来构建理科学生的学习。生物学的背景很重要，因为它将学生的学习置于微观和亚微观学科中，在这些学科中，我们对触觉对学习的影响的了解远少于我们对使用大型现实世界物体学习的了解。许多过去的研究已经证明，使用身体和手来学习可以在类似的科学背景下有所帮助，但这个项目的目的是通过使用一种新的神经成像方法（功能性近红外光谱或fNIRS）来测试身体本身帮助学生学习的说法，该方法允许学生在学习时移动身体，然后确定大脑的运动区域是否在以后的问题解决过程中保持活跃。该项目为科学教育提供了附加值，因为它将确定何时以及如何进行特定的手部运动（即，手势）与来自触觉手套的反馈相结合使得能够进行手势或感觉反馈之外的科学学习（即，触摸）独自一人。这些知识将有助于解决学习科学中的一个突出问题，即何时以及如何使用符号表示（例如，等式或数字）或者具体表示（例如，塑料手持模型）。该项目采用了一种混合方法，旨在解决计算机可视化的局限性，这些局限性无法让学生感受到他们感官之外的小世界。此外，项目团队旨在生产的技术将导致商业手套价格的成本大幅降低。最后，该项目汇集了一个多学科团队，将向包括科学教师，私营企业和学习科学家在内的不同利益相关者传播成果。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project aims to determine how sensory feedback delivered to students’ hands helps them understand the meanings of pictures, graphs, and other representations displayed in computer simulations. To accomplish this aim, the project team will design and develop a custom haptic glove that interacts with a computer visualization of a biological cell. Unlike prior research, the project aims to structure science students’ learning by guiding them to think about the meaning of what their hands feel when they move across a visualization of a biological cell. The context of biology matters because it situates students’ learning in micro- and sub-microscopic disciplines where we know much less about the effects of haptics on learning than we know about learning with large, real-world objects. Many past studies have demonstrated that using the body and hands to learn can help in similar science contexts, but this project aims to additionally test the claim that the body itself helps students learning by using a novel neuroimaging method (functional Near Infrared Spectroscopy or fNIRS) that allows students to move their bodies while learning and then determine if motor areas of the brain remain active during later problem solving. This project delivers added value to science education because it will determine when and how specific hand movements (i.e., gestures) combined with feedback from the haptic glove enable science learning above and beyond gestures or sensory feedback (i.e., touch) alone. This knowledge will help address an outstanding problem in the science of learning that involves when and how to use symbolic representations (e.g., equations or numbers) or else concrete representations (e.g., plastic hand-held models). The project adopts a blended approach that aims to address limitations of computer visualizations that fail to deliver to students a feeling for the small world beyond their senses. Moreover, the technology the project team aims to produce will result in a significant cost reduction to the price of commercial gloves. Last, the project brings together a multi-disciplinary team that will disseminate results to diverse stakeholders that include science instructors, private industry, and learning scientists.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.