ITR: Collaborative Research: Modeling and Display of Haptic Information for Enhanced Performance of Computer-Integrated Surgery

信息技术在手术室中取得了巨大的进步。在微创手术的实践中，三维建模技术使术前规划、新的图像采集和显示技术以及远程操作机器人系统的卓越灵活性成为可能。然而，在这些技术进步中，有一个感官通道目前被忽视了：触摸。在手工手术中，触觉反馈对于触诊、缝合操作和穿刺事件的检测至关重要。力的感觉对于诸如抓取、切割、解剖和经皮（GCDP）治疗等侵袭性工具-组织相互作用任务尤为重要。触觉信息的缺乏使外科医生所需的视觉感知超负荷，需要高度的关注。信息技术要真正提升手术实践，必须利用多种感觉通道。我们建议开发仪器和算法来模拟四种常见手术任务中工具-组织相互作用的触觉方面，即：抓取，切割，解剖和经皮治疗。该系统将从三个方面显著增强信息显示。首先，在基于模型的远程手术过程中，实时获取的数据将用于向外科医生提供反馈，从而增加透明度。机器人辅助手术系统。其次，实时建模技术将使基于模型的遥操作成为可能，消除了传统的直接遥操作中时间延迟所带来的严格限制。第三，真实的手术模拟将改善培训，提高外科医生的能力和患者的安全。仪器和建模算法将用于确定不同组织类型的参数值，特别是肝脏、前列腺、脾脏和肾脏。这些实验将使用离体组织和幻影水凝胶组织。一旦仪器和建模技术被开发出来，我们将继续对增强信息显示的三种应用进行广泛的验证。性能实验将验证直接和基于模型的远程操作的准确性和精度的提高，计算机视觉/力传感系统将确定为外科模拟开发的力和变形模型的真实感。该提案解决了ITR的挑战，即开发一种具有计算、仿真和数据分析方法的信息增强显示，用于对目前没有系统模型的常见外科GCDP任务进行建模。智力优势：该项目的具体目标是通过感知和获取微创手术中工具-组织相互作用过程中代表触觉信息的模型，显著改善信息化手术室。这项研究将显著影响：（a)反映手术过程中实际作用力和变形的工具-组织相互作用模型，(b)在直接或基于模型的远程手术期间向外科医生反馈触觉，从而改善手术结果，c）为培训当前和未来的医疗保健专业人员开发逼真的模拟，以及d)为传统微创手术和机器人辅助手术开发仪器化智能工具。更广泛的影响：来自这两个机构的pi都有通过本科生研究经验（REU）让本科生（包括男性和女性）参与研究项目的优秀历史。此外，由NSF推荐资助的Drexel教师研究经验（RET）网站提案，以及约翰霍普金斯大学建立的RET项目，将涉及高中数学和科学教师参与与该提案相关的暑期研究项目。这些活动将加强来自市中心学校的代表性不足的群体的参与，并导致更广泛地传播学生和教师的科学和技术教育。对于研究生来说，本研究的跨学科性质为教育提供了新的机会，扩大了机械工程师和医学专业人员的互动。最后，所提出的研究将导致手术结果的改善，使患者和整个社会受益。由于该项目需要研究的不同领域，拟议的研究将受益于德雷克塞尔和JHU的pi的合作。Drexel在触觉、FEM建模、抓取/解剖任务和幻影组织方面拥有专业知识，而JHU在基于现实的切割/经皮治疗建模、触觉和人机系统验证实验方面拥有专业知识。两所机构独特的设施和合作，以及与医疗专业人员的紧密联系，使这项工作适合作为合作研究项目。

Information Technology is making great progress in the operating room. In the practice of minimally invasive surgery, three-dimensional modeling techniques are enabling preoperative planning, new image acquisition and display techniques, and superior dexterity through teleoperated robotic systems. However, one sensory channel is presently ignored in these technological improvements: touch. In manual surgeries, haptic feedback is crucial for palpation, suture manipulation, and detection of puncture events. Sensation of forces is particularly important for invasive tool-tissue interaction tasks such as grasping, cutting, dissection, and percutaneous (GCDP) therapy. Lack of haptic information overloads the visual sensing required of the surgeon, requiring a demanding level attention. For information technology to truly enhance the practice of surgery, multiple sensory channels must be utilized. We propose to develop instrumentation and algorithms for modeling the haptic aspect of tool-tissue interaction in four common surgical tasks, namely: grasping, cutting, dissection, and percutaneous therapy. This system will significantly enhance information display in three ways. First, data acquired in real time will be used to provide feedback to the surgeon during model-based teleoperated procedures, increasing the transparency. of the robot-assisted surgical system. Second, real-time modeling techniques will enable model-based teleoperation, removing the strict constraints imposed by time delays in traditional, direct teleoperation. Third, realistic surgical simulations will improve training, increasing surgeon competence and patient safety. The instrumentation and modeling algorithms will be used to determine parameter values for different tissue types, particularly liver, prostate, spleen, and kidney. Ex-vivo tissues and phantom hydrogel tissues will be used in these experiments. Once the instrumentation and modeling techniques are developed, we will proceed to extensive validation of the three applications of enhanced information display. Performance experiments will verify improvements in accuracy and precision for direct and model-based teleoperation, and a computer vision/force sensing system will determine the realism of force and deformation models developed for surgical simulation. This proposal addresses the ITR challenge for developing an information-enhanced display with computational, simulation, and data analysis methods for modeling common surgical GCDP tasks for which currently there is no systematic model. Intellectual merit: The specific goal of this project is to significantly improve the information-enhanced operating room through the sensing and acquisition of models representing haptic information during tool-tissue interactions in minimally invasive surgery. This research will significantly impact: (a) tool-tissue interaction models that reflect the actual forces and deformation occurring during surgery, (b) haptic feedback to the surgeon during direct or modelbased teleoperation, thereby improving surgical outcomes, c) the development of realistic simulations for training current and future health care professionals, and d) the development of instrumented smart tools for both traditional minimally invasive and robot-assisted surgeries. Broader Impacts: The PIs from both institutions have an excellent history of involving undergraduate students (both men and women) in research projects through Research Experience for Undergraduates (REU). Additionally, the Drexel Research Experience for Teachers (RET) site proposal recommended for funding by NSF, along with an established RET program at Johns Hopkins, will involve high-school math and science teachers from high schools to participate in summer research projects related to this proposal. These activities will enhance participation of underrepresented groups from inner-city schools and lead to broader dissemination of scientific and technological education of the students and teachers. For graduate students, the interdisciplinary nature of this research offers new opportunities in education, broadening the interaction of mechanical engineers and medical professionals. Finally, the proposed research will lead to improvements in surgical outcomes, benefiting the patient and the society at large. Due to the diverse areas of research required by this project, the proposed research will benefit from the collaboration of PIs from Drexel and JHU. Drexel has expertise in haptics, FEM modeling, grasping/dissection tasks, and phantom tissues while JHU has expertise in reality-based modeling for cutting/percutaneous therapies, haptics, and validation experiments for human-machine systems. The unique facilities and collaborations at both institutions, along with strong ties to medical professionals, make this work appropriate as a collaborative research project.