Bimodal Haptic-Mixed Reality (HMR) Needle Insertion Simulation for Hand-Eye Skills

虽然静脉（IV）插入是一种非常常见的医疗程序，但技术上很难掌握，因为35%-50%的失败率导致重新插入的负循环，从而增加了患者的伤害和医疗保健系统的成本。在现实条件下，错误的静脉注射与静脉变量（静脉滚动或抗穿刺）和患者变量（触摸皮肤，皮肤颜色）有关。护理教育专家提倡将基于模拟的技术自定节奏地整合在一起，有意识地练习静脉注射技能，同时获得即时反馈以纠正错误。然而，使用目前开发的模拟器或假人手臂无法捕捉到实际的真实感和适应变化的心理运动技术，从而获得对技能的程序性掌握。为了改善当前的学习环境，需要技术进步来创造一个具有可变性的现实学习平台，以最大限度地提高技能转移和长期保留。提出的工作是通过开发一种使用触觉和混合现实（HMR）的新型仿真系统并研究学习影响来填补这一空白。这项工作是重要的，因为目前的触觉技术与扩展现实相结合，还没有提供足够的真实感和可变性，以有效地发展精细运动技能。此外，还没有研究双峰HMR模拟的教育影响，该模拟具有可变条件，可以在训练期间自适应地创建逼真的患者环境。在发展拟议研究的成功性质后，将提供对有效学习技术的新见解以及改善手眼技能学习的原因，这可能用于改善类似环境中的学习，或对其他领域具有变革性，如网络教学和学习，工作中的手技能培训，沉浸式灵巧界面，残疾人运动技能开发，STEM学习，机器人手术和医疗培训。本项目将开发一个双峰式HMR系统，利用新兴技术、触觉学和MR，模拟可变条件下静脉注射针头的插入，为学生创造一个真实的学习环境，掌握用双手插入的触觉技能；并调查实践中的变异性（弃用理论）是否能提高针插技能。为了实现这些目标，该项目将分为两个阶段：第一阶段和第二阶段。第一阶段将重点开发双峰触觉模拟，使用两个互补的触觉设备，一个触觉手套和一个触控笔触觉设备，与MR集成，模拟虚拟患者和静脉针插入，具有可变的训练条件（肤色和硬度，静脉滚动，或抗穿刺）。在第二阶段研究中，360名（每年180名）护理专业学生将被随机分配到三种模式（hmr静态、hmr可变、假臂）中的一种进行培训。为了评估学习者的静脉注射技能，来自护理学院的训练有素的评估人员（教职员工）将根据既定的静脉注射技能检查表通过考试观察和评估参与者的技能。将根据现实情况和用户体验（可用性）收集培训后调查，这些数据将用于不断改进人力资源管理系统。本研究将推进与利用新兴技术开发创新学习和教学环境相关的知识，并提供影响学习绩效的影响变量的实证证据。开发的平台作为自动自我实践系统，将为医疗保健或相关社区的教师和学生免费提供这一媒介，即使在大流行期间也可以扩展和使用，以扩大代表性不足和经济困难群体的参与。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Although performing intravenous (IV) insertion is a very common medical procedure, it is technically difficult to master as demonstrated by the 35%-50% failure rate resulting in a negative cycle of re-insertions leading to increased patient harm and costs to the healthcare system. Faulty IV insertions in real-world conditions are related to vein variables (vein rolling or resistant to puncture) and patient variables (touch skin, skin coloring). Experts in nursing education have advocated for self-paced integration of simulation-based technologies to deliberately practice IV skills while receiving immediate feedback for error correction. However, using currently developed simulators or manikin arms fails to capture the actual realism and psychomotor techniques adaptive to variability needed to gain procedural mastery of the skill. To enhance the current learning environment, technological advances are needed to create a realistic learning platform with variability that maximizes the skill transfer and long-term retention. The proposed work is to fill the gap by developing a novel simulation system using haptics and mixed reality (HMR) and investigating the learning impacts. This work is significant because current haptic technologies combined with extended reality do not yet provide sufficient realism and variability to effectively develop the fine motor skills. Further, studies have not been conducted on the educational impact of bimodal HMR simulation with variable conditions that can adaptively create realistic patient environments during training. Upon developing the successful nature of the proposed research, new insight into effective learning technology as well as causes of improved learning in hand-eye skills will be provided, which may be used to improve learning in similar settings or be transformative to other fields such as cyber teaching and learning, hand skill training at work, immersive dexterous interfaces, motor skill development for people with disabilities, STEM learning, robotic surgery, and medical training.This project will develop a bimodal HMR system, using emerging technologies, haptics and MR, to simulate IV needle insertion with variable conditions that will create a realistic learning environment for students to master insertion tactile skills using two hands; and investigate whether variability in practice (disuse theory) improves needle insertion skills. To achieve these goals, the project will be divided into two phases: Phase I and II. Phase I will focus on developing the bimodal haptic simulation using two complimentary haptic devices, a haptic glove and a stylus haptic device, integrated with MR to simulate virtual patients and IV needle insertion with variable training conditions (skin color and stiffness, vein rolled, or resistant to puncture). In Phase II studies, 360 (180 per year) nursing students will be randomly assigned to experience training sessions in one of the three modes (HMR-static, HMR-variable, manikin arm). To measure learners’ IV insertion skills, trained evaluators (faculty members) from the College of Nursing will observe and evaluate participants’ skills based on an established IV insertion skill checklist through exams. Post training surveys will be collected in terms of the realism and the user experience (usability) and those data will be used for continuously improving the HMR system. This research will advance the knowledge related to developing innovative learning and teaching environments using emerging technologies and provide empirical evidence of impactful variables that affect learning performance. The developed platform as an automatic self-practice system will provide free access to this medium for instructors and students alike in healthcare or related communities to extend and use even under a pandemic, for broadening participation for under-represented and financially challenged groups.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.