Bimodal Haptic-Mixed Reality (HMR) Needle Insertion Simulation for Hand-Eye Skills

尽管静脉(IV)插入是一种非常常见的医疗程序，但它在技术上很难掌握，35%-50%的失败率导致重复插入的负面循环，从而增加了患者的伤害和医疗保健系统的成本。在现实条件下，静脉插管的错误与静脉变量(静脉滚动或抗穿刺性)和患者变量(触摸皮肤、皮肤颜色)有关。护理教育领域的专家们主张将基于模拟的技术进行自定进度的整合，以刻意练习IV技能，同时接收即时反馈以进行纠错。然而，使用目前开发的模拟器或人体手臂无法捕捉到实际的现实主义和适应变异性的精神运动技术，这是获得程序上的技能掌握所需的。为了改善目前的学习环境，需要技术进步来创造一个具有可变性的现实学习平台，使技能转移和长期保留最大化。为了填补这一空白，我们开发了一种使用触觉和混合现实技术(HMR)的新型模拟系统，并研究了该系统对学习的影响。这项工作意义重大，因为目前的触觉技术与扩展现实相结合，还不能提供足够的真实感和可变性来有效地发展精细运动技能。此外，还没有进行关于双峰HMR模拟的教育影响的研究，该模拟具有可变的条件，可以在训练期间自适应地创建真实的患者环境。在发展拟议研究的成功性质后，将提供对有效学习技术以及手眼技能学习改善原因的新见解，可用于改善类似环境下的学习，或对其他领域产生变革，如网络教学和学习、工作中的手技能训练、沉浸式灵活界面、残疾人士的运动技能发展、STEM学习、机器人手术和医学培训。该项目将开发一个双模式HMR系统，使用新兴技术、触觉和MR来模拟可变条件下的静脉插针，为学生创造一个现实的学习环境，让学生用双手掌握插入触觉技能；并调查实践中的可变性(废弃理论)是否提高了针插入技能。为了实现这些目标，该项目将分为两个阶段：第一阶段和第二阶段。第一阶段将专注于开发双峰触觉模拟，使用两个免费的触觉设备，一个触觉手套和一个触笔触觉设备，与MR集成以模拟虚拟患者和不同训练条件(皮肤颜色和僵硬、静脉滚动或抗穿刺性)的IV针插入。在第二阶段的研究中，360名(每年180名)护生将被随机分配到三种模式(HMR-静态、HMR-可变、仿人手臂)中的一种模式下进行培训。为了测量学习者的IV插入技能，护理学院的训练有素的评估员(教职员工)将根据通过考试建立的IV插入技能检查表来观察和评估参与者的技能。将从现实性和用户体验(可用性)方面收集培训后调查，这些数据将用于不断改进人力资源管理系统。这项研究将推进与利用新兴技术发展创新的学习和教学环境相关的知识，并为影响学习绩效的影响变量提供经验证据。开发的平台作为一个自动自我练习系统，将为医疗保健或相关社区的教师和学生提供免费访问这一媒介的机会，即使在大流行的情况下也可以扩展和使用，以扩大代表不足和经济困难群体的参与。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Although performing intravenous (IV) insertion is a very common medical procedure, it is technically difficult to master as demonstrated by the 35%-50% failure rate resulting in a negative cycle of re-insertions leading to increased patient harm and costs to the healthcare system. Faulty IV insertions in real-world conditions are related to vein variables (vein rolling or resistant to puncture) and patient variables (touch skin, skin coloring). Experts in nursing education have advocated for self-paced integration of simulation-based technologies to deliberately practice IV skills while receiving immediate feedback for error correction. However, using currently developed simulators or manikin arms fails to capture the actual realism and psychomotor techniques adaptive to variability needed to gain procedural mastery of the skill. To enhance the current learning environment, technological advances are needed to create a realistic learning platform with variability that maximizes the skill transfer and long-term retention. The proposed work is to fill the gap by developing a novel simulation system using haptics and mixed reality (HMR) and investigating the learning impacts. This work is significant because current haptic technologies combined with extended reality do not yet provide sufficient realism and variability to effectively develop the fine motor skills. Further, studies have not been conducted on the educational impact of bimodal HMR simulation with variable conditions that can adaptively create realistic patient environments during training. Upon developing the successful nature of the proposed research, new insight into effective learning technology as well as causes of improved learning in hand-eye skills will be provided, which may be used to improve learning in similar settings or be transformative to other fields such as cyber teaching and learning, hand skill training at work, immersive dexterous interfaces, motor skill development for people with disabilities, STEM learning, robotic surgery, and medical training.This project will develop a bimodal HMR system, using emerging technologies, haptics and MR, to simulate IV needle insertion with variable conditions that will create a realistic learning environment for students to master insertion tactile skills using two hands; and investigate whether variability in practice (disuse theory) improves needle insertion skills. To achieve these goals, the project will be divided into two phases: Phase I and II. Phase I will focus on developing the bimodal haptic simulation using two complimentary haptic devices, a haptic glove and a stylus haptic device, integrated with MR to simulate virtual patients and IV needle insertion with variable training conditions (skin color and stiffness, vein rolled, or resistant to puncture). In Phase II studies, 360 (180 per year) nursing students will be randomly assigned to experience training sessions in one of the three modes (HMR-static, HMR-variable, manikin arm). To measure learners’ IV insertion skills, trained evaluators (faculty members) from the College of Nursing will observe and evaluate participants’ skills based on an established IV insertion skill checklist through exams. Post training surveys will be collected in terms of the realism and the user experience (usability) and those data will be used for continuously improving the HMR system. This research will advance the knowledge related to developing innovative learning and teaching environments using emerging technologies and provide empirical evidence of impactful variables that affect learning performance. The developed platform as an automatic self-practice system will provide free access to this medium for instructors and students alike in healthcare or related communities to extend and use even under a pandemic, for broadening participation for under-represented and financially challenged groups.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.