ITR: Collaborative Research: Modeling and Display of Haptic Information for Enhanced Performance of Computer-Integrated Surgery

信息技术在手术室中取得了巨大的进步。在微创手术的实践中，三维建模技术使术前规划、新的图像采集和显示技术以及通过遥控机器人系统的上级灵活性成为可能。然而，在这些技术进步中，有一个感官通道目前被忽视了：触觉。在手动手术中，触觉反馈对于触诊、缝合操作和穿刺事件检测至关重要。力的感觉对于有创工具-组织相互作用任务（例如抓取、切割、剥离和经皮（GCDP）治疗）尤其重要。触觉信息的缺乏使外科医生所需的视觉感测过载，需要苛刻的注意力水平。为了使信息技术真正提高外科手术的实践，必须利用多个感官通道。 我们建议开发仪器和算法，用于对四种常见手术任务中工具与组织相互作用的触觉方面进行建模，即：抓取、切割、解剖和经皮治疗。这一系统将从三个方面大大加强信息显示。首先，在基于模型的远程操作过程中，真实的时间采集的数据将用于向外科医生提供反馈，从而增加透明度。机器人辅助手术系统其次，实时建模技术将使基于模型的遥操作成为可能，消除了传统直接遥操作中时间延迟所带来的严格限制。第三，逼真的手术模拟将改善培训，提高外科医生的能力和病人的安全。仪器和建模算法将用于确定不同组织类型的参数值，特别是肝脏、前列腺、脾脏和肾脏。这些实验将使用离体组织和体模水凝胶组织。一旦仪器和建模技术的发展，我们将继续进行广泛的验证增强信息显示的三个应用程序。 性能实验将验证直接和基于模型的遥控操作的准确性和精度的提高，计算机视觉/力传感系统将确定为手术模拟开发的力和变形模型的真实性。该提案解决了ITR的挑战，开发一个信息增强显示与计算，模拟和数据分析方法建模常见的手术GCDP任务，目前还没有系统的模型。 智力优点：该项目的具体目标是通过在微创手术中的工具-组织相互作用期间感测和获取代表触觉信息的模型来显著改善信息增强的手术室。这项研究将产生重大影响：（a）反映手术期间发生的实际力和变形的工具-组织相互作用模型，（B）在直接或基于模型的远程操作期间向外科医生提供触觉反馈，从而改善手术结果，c）开发用于培训当前和未来的卫生保健专业人员的逼真模拟，以及d）开发用于传统微创手术和机器人辅助手术的仪器化智能工具。 更广泛的影响：这两个机构的PI都有通过本科生研究经验（REU）让本科生（男性和女性）参与研究项目的优秀历史。此外，Drexel Research Experience for Teachers（RET）网站建议由NSF资助，沿着约翰霍普金斯大学的既定RET计划，将涉及高中数学和科学教师参加与此建议相关的夏季研究项目。这些活动将加强市中心学校代表性不足的群体的参与，并导致更广泛地向学生和教师传播科学和技术教育。对于研究生来说，这项研究的跨学科性质为教育提供了新的机会，扩大了机械工程师和医疗专业人员的互动。最后，拟议的研究将导致手术结果的改善，使患者和整个社会受益。 由于该项目所需的研究领域不同，拟议的研究将受益于德雷克塞尔和JHU的PI的合作。 Drexel在触觉、FEM建模、抓取/解剖任务和体模组织方面拥有专业知识，而JHU在基于现实的切割/经皮治疗建模、触觉和人机系统验证实验方面拥有专业知识。这两个机构独特的设施和合作，沿着与医学专业人士的密切联系，使这项工作适合作为一个合作研究项目。

Information Technology is making great progress in the operating room. In the practice of minimally invasive surgery, three-dimensional modeling techniques are enabling preoperative planning, new image acquisition and display techniques, and superior dexterity through teleoperated robotic systems. However, one sensory channel is presently ignored in these technological improvements: touch. In manual surgeries, haptic feedback is crucial for palpation, suture manipulation, and detection of puncture events. Sensation of forces is particularly important for invasive tool-tissue interaction tasks such as grasping, cutting, dissection, and percutaneous (GCDP) therapy. Lack of haptic information overloads the visual sensing required of the surgeon, requiring a demanding level attention. For information technology to truly enhance the practice of surgery, multiple sensory channels must be utilized. We propose to develop instrumentation and algorithms for modeling the haptic aspect of tool-tissue interaction in four common surgical tasks, namely: grasping, cutting, dissection, and percutaneous therapy. This system will significantly enhance information display in three ways. First, data acquired in real time will be used to provide feedback to the surgeon during model-based teleoperated procedures, increasing the transparency. of the robot-assisted surgical system. Second, real-time modeling techniques will enable model-based teleoperation, removing the strict constraints imposed by time delays in traditional, direct teleoperation. Third, realistic surgical simulations will improve training, increasing surgeon competence and patient safety. The instrumentation and modeling algorithms will be used to determine parameter values for different tissue types, particularly liver, prostate, spleen, and kidney. Ex-vivo tissues and phantom hydrogel tissues will be used in these experiments. Once the instrumentation and modeling techniques are developed, we will proceed to extensive validation of the three applications of enhanced information display. Performance experiments will verify improvements in accuracy and precision for direct and model-based teleoperation, and a computer vision/force sensing system will determine the realism of force and deformation models developed for surgical simulation. This proposal addresses the ITR challenge for developing an information-enhanced display with computational, simulation, and data analysis methods for modeling common surgical GCDP tasks for which currently there is no systematic model. Intellectual merit: The specific goal of this project is to significantly improve the information-enhanced operating room through the sensing and acquisition of models representing haptic information during tool-tissue interactions in minimally invasive surgery. This research will significantly impact: (a) tool-tissue interaction models that reflect the actual forces and deformation occurring during surgery, (b) haptic feedback to the surgeon during direct or modelbased teleoperation, thereby improving surgical outcomes, c) the development of realistic simulations for training current and future health care professionals, and d) the development of instrumented smart tools for both traditional minimally invasive and robot-assisted surgeries. Broader Impacts: The PIs from both institutions have an excellent history of involving undergraduate students (both men and women) in research projects through Research Experience for Undergraduates (REU). Additionally, the Drexel Research Experience for Teachers (RET) site proposal recommended for funding by NSF, along with an established RET program at Johns Hopkins, will involve high-school math and science teachers from high schools to participate in summer research projects related to this proposal. These activities will enhance participation of underrepresented groups from inner-city schools and lead to broader dissemination of scientific and technological education of the students and teachers. For graduate students, the interdisciplinary nature of this research offers new opportunities in education, broadening the interaction of mechanical engineers and medical professionals. Finally, the proposed research will lead to improvements in surgical outcomes, benefiting the patient and the society at large. Due to the diverse areas of research required by this project, the proposed research will benefit from the collaboration of PIs from Drexel and JHU. Drexel has expertise in haptics, FEM modeling, grasping/dissection tasks, and phantom tissues while JHU has expertise in reality-based modeling for cutting/percutaneous therapies, haptics, and validation experiments for human-machine systems. The unique facilities and collaborations at both institutions, along with strong ties to medical professionals, make this work appropriate as a collaborative research project.