Collaborative Research: Haptic Technology Innovations to Guide Student Gesturing with Dynamic STEM Visualizations

这个项目旨在确定传递到学生手中的感觉反馈如何帮助他们理解计算机模拟中显示的图片、图形和其他表示的含义。为了实现这一目标，项目团队将设计和开发一种定制的触觉手套，它可以与生物细胞的计算机可视化交互。与之前的研究不同，该项目旨在通过引导理科学生思考手在生物细胞可视化中移动时的感觉的意义，来构建理科学生的学习结构。生物学的背景很重要，因为它将学生的学习置于微观和亚微观学科中，在这些学科中，我们对触觉对学习的影响的了解远远少于我们对大型现实世界物体的学习。过去的许多研究表明，在类似的科学背景下，使用身体和手进行学习可以有所帮助，但该项目旨在进一步验证身体本身通过使用一种新的神经成像方法(功能性近红外光谱或fNIRS)帮助学生学习的说法，这种方法允许学生在学习时移动身体，然后确定大脑的运动区在以后解决问题时是否保持活跃。该项目为科学教育带来了附加值，因为它将确定何时以及如何将特定的手部动作(即手势)与来自触觉手套的反馈相结合，从而使科学学习超越手势或感觉反馈(即触摸)。这一知识将有助于解决学习科学中的一个悬而未决的问题，即何时以及如何使用符号表示法(例如，方程式或数字)或具体表示法(例如，塑料手持模型)。该项目采用了一种混合的方法，旨在解决计算机可视化的局限性，这些限制无法向学生传达一种超出他们感官的小世界的感觉。此外，项目团队计划生产的技术将显著降低商业手套的价格。最后，该项目汇集了一个多学科团队，将成果传播给不同的利益相关者，包括科学教师、私营行业和学习科学家。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This project aims to determine how sensory feedback delivered to students’ hands helps them understand the meanings of pictures, graphs, and other representations displayed in computer simulations. To accomplish this aim, the project team will design and develop a custom haptic glove that interacts with a computer visualization of a biological cell. Unlike prior research, the project aims to structure science students’ learning by guiding them to think about the meaning of what their hands feel when they move across a visualization of a biological cell. The context of biology matters because it situates students’ learning in micro- and sub-microscopic disciplines where we know much less about the effects of haptics on learning than we know about learning with large, real-world objects. Many past studies have demonstrated that using the body and hands to learn can help in similar science contexts, but this project aims to additionally test the claim that the body itself helps students learning by using a novel neuroimaging method (functional Near Infrared Spectroscopy or fNIRS) that allows students to move their bodies while learning and then determine if motor areas of the brain remain active during later problem solving. This project delivers added value to science education because it will determine when and how specific hand movements (i.e., gestures) combined with feedback from the haptic glove enable science learning above and beyond gestures or sensory feedback (i.e., touch) alone. This knowledge will help address an outstanding problem in the science of learning that involves when and how to use symbolic representations (e.g., equations or numbers) or else concrete representations (e.g., plastic hand-held models). The project adopts a blended approach that aims to address limitations of computer visualizations that fail to deliver to students a feeling for the small world beyond their senses. Moreover, the technology the project team aims to produce will result in a significant cost reduction to the price of commercial gloves. Last, the project brings together a multi-disciplinary team that will disseminate results to diverse stakeholders that include science instructors, private industry, and learning scientists.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.