Bimodal Haptic-Mixed Reality (HMR) Needle Insertion Simulation for Hand-Eye Skills

虽然进行静脉（IV）插入是一种非常常见的医疗程序，但从技术上讲，很难掌握，因为35%-50%的失败率导致了重新插入的负面循环，从而增加了患者伤害和医疗保健系统的成本。真实条件下的Fauclide IV插入与静脉变量（静脉滚动或抗穿刺）和患者变量（触摸皮肤、皮肤着色）相关。护理教育专家提倡自主集成基于模拟的技术，以便在接受错误纠正的即时反馈的同时，有意识地练习IV技能。然而，使用目前开发的模拟器或假人手臂未能捕捉到实际的现实主义和适应变化的心理学技术，需要获得程序掌握的技能。为了改善目前的学习环境，需要技术进步，以建立一个具有可变性的现实学习平台，最大限度地提高技能转移和长期保留。拟议的工作是填补差距，开发一种新的模拟系统，使用触觉和混合现实（HMR）和调查的学习影响。这项工作是重要的，因为目前的触觉技术结合延展实境还没有提供足够的现实主义和可变性，有效地发展精细的运动技能。此外，还没有进行关于具有可变条件的双峰HMR模拟的教育影响的研究，所述可变条件可以在训练期间自适应地创建逼真的患者环境。在开发拟议研究的成功性质后，将提供对有效学习技术的新见解以及手眼技能学习改善的原因，这可能用于改善类似环境中的学习或对其他领域具有变革性，例如网络教学和学习，工作中的手部技能培训，沉浸式灵巧界面，残疾人运动技能开发，STEM学习，该项目将开发一个双模态HMR系统，使用新兴技术，触觉和MR，模拟在可变条件下的静脉针插入，为学生创造一个逼真的学习环境，让他们掌握使用双手的插入触觉技能;并调查实践中的可变性（废用理论）是否会提高针头插入技能。为了实现这些目标，该项目将分为两个阶段：第一阶段和第二阶段。第一阶段将重点开发双峰触觉模拟使用两个免费的触觉设备，触觉手套和触笔触觉设备，与MR集成，以模拟虚拟患者和IV针插入可变的训练条件（肤色和硬度，静脉滚动，或耐穿刺）。在II期研究中，360名（每年180名）护理专业学生将被随机分配到三种模式（HMR静态、HMR可变、假人手臂）中的一种模式下进行培训。为了衡量学习者的IV插入技能，来自护理学院的经过培训的评估人员（教师）将通过考试根据既定的IV插入技能清单观察和评估参与者的技能。将收集培训后调查的真实性和用户体验（可用性），这些数据将用于持续改进HMR系统。这项研究将推进与使用新兴技术开发创新的学习和教学环境相关的知识，并提供影响学习表现的有影响力的变量的经验证据。开发的平台作为一个自动自我实践系统，将为医疗保健或相关社区的教师和学生提供免费访问该媒体的机会，即使在大流行病下也可以扩展和使用，扩大参与，该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查进行评估，被认为值得支持的搜索.

Although performing intravenous (IV) insertion is a very common medical procedure, it is technically difficult to master as demonstrated by the 35%-50% failure rate resulting in a negative cycle of re-insertions leading to increased patient harm and costs to the healthcare system. Faulty IV insertions in real-world conditions are related to vein variables (vein rolling or resistant to puncture) and patient variables (touch skin, skin coloring). Experts in nursing education have advocated for self-paced integration of simulation-based technologies to deliberately practice IV skills while receiving immediate feedback for error correction. However, using currently developed simulators or manikin arms fails to capture the actual realism and psychomotor techniques adaptive to variability needed to gain procedural mastery of the skill. To enhance the current learning environment, technological advances are needed to create a realistic learning platform with variability that maximizes the skill transfer and long-term retention. The proposed work is to fill the gap by developing a novel simulation system using haptics and mixed reality (HMR) and investigating the learning impacts. This work is significant because current haptic technologies combined with extended reality do not yet provide sufficient realism and variability to effectively develop the fine motor skills. Further, studies have not been conducted on the educational impact of bimodal HMR simulation with variable conditions that can adaptively create realistic patient environments during training. Upon developing the successful nature of the proposed research, new insight into effective learning technology as well as causes of improved learning in hand-eye skills will be provided, which may be used to improve learning in similar settings or be transformative to other fields such as cyber teaching and learning, hand skill training at work, immersive dexterous interfaces, motor skill development for people with disabilities, STEM learning, robotic surgery, and medical training.This project will develop a bimodal HMR system, using emerging technologies, haptics and MR, to simulate IV needle insertion with variable conditions that will create a realistic learning environment for students to master insertion tactile skills using two hands; and investigate whether variability in practice (disuse theory) improves needle insertion skills. To achieve these goals, the project will be divided into two phases: Phase I and II. Phase I will focus on developing the bimodal haptic simulation using two complimentary haptic devices, a haptic glove and a stylus haptic device, integrated with MR to simulate virtual patients and IV needle insertion with variable training conditions (skin color and stiffness, vein rolled, or resistant to puncture). In Phase II studies, 360 (180 per year) nursing students will be randomly assigned to experience training sessions in one of the three modes (HMR-static, HMR-variable, manikin arm). To measure learners’ IV insertion skills, trained evaluators (faculty members) from the College of Nursing will observe and evaluate participants’ skills based on an established IV insertion skill checklist through exams. Post training surveys will be collected in terms of the realism and the user experience (usability) and those data will be used for continuously improving the HMR system. This research will advance the knowledge related to developing innovative learning and teaching environments using emerging technologies and provide empirical evidence of impactful variables that affect learning performance. The developed platform as an automatic self-practice system will provide free access to this medium for instructors and students alike in healthcare or related communities to extend and use even under a pandemic, for broadening participation for under-represented and financially challenged groups.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.