Cooperative Control Robotics and Computer Vision: Development of Semi-Autonomous Temporal Bone and Skull Base Surgery

协作控制机器人和计算机视觉：半自主颞骨和颅底手术的发展

• 批准号: 10631972
• 负责人: Francis Xavier Creighton
• 金额: $ 19.18万
• 依托单位: JOHNS HOPKINS UNIVERSITY
• 依托单位国家: 美国
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-06-01 至 2026-05-31
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10631972
• 关键词: 3-Dimensional; Address; Adoption; Algorithms; Anatomy; Area; Atlases; Automobile Driving; Automobiles; Biomedical Engineering; Cadaver; Cochlea; Communication; Complex; Computer Vision Systems; Computers; Consumption; Data; Development; Drug Delivery Systems; Dura Mater; Educational process of instructing; Electromagnetics; Engineering; Equilibrium; Ethics; Facial nerve structure; Feedback; Future; Goals; Hand; Head and Neck Surgery; Hearing; Human; Image; Image-Guided Surgery; Intervention; Judgment; K-Series Research Career Programs; Knowledge; Labyrinth; Lateral; Left; Manuals; Mentors; Methods; Microscope; Modeling; Modernization; Monitor; Motion; Movement; Operative Surgical Procedures; Otolaryngology; Outcome; Patients; Positioning Attribute; Postoperative Period; Procedures; Process; Research; Research Personnel; Robot; Robotics; Safety; Scientist; Shapes; Sigmoid colon; Site; Speed; Structure; Surface; Surgeon; Surgical Instruments; Surgical complication; System; Tactile; Techniques; Technology; Temporal bone structure; Testing; Time; Training; Tremor; Update; Work; X-Ray Computed Tomography; automated segmentation; career; computer science; deep learning; digital; digital imaging; image guided; image registration; improved; instrument; learning network; medical schools; microscopic imaging; millimeter; multidisciplinary; neurovascular; novel; professor; responsible research conduct; robot control; simulation; skills; skull base; stereoscopic; success; surgery outcome; three-dimensional modeling; virtual

Project Summary I am an assistant professor in the department of Otolaryngology-Head and Neck Surgery at the Johns Hopkins School of Medicine, where my practice is focused on neurotology and lateral skull base surgery. I am applying for a mentored surgeon-scientist career development award (CDA) to obtain further training in robotics, deep learning and computer vision. This will further my long-term career goals of improving neurotologic surgical outcomes through novel applications of engineering methods and ultimately to investigate semi-autonomous, robotic interventions in the inner ear and skull base which are beyond the limits of the human hand alone. Operating in the temporal bone and lateral skull base is technically demanding due to complex three- dimensional anatomy, small working spaces and delicate neurovascular structures. Many of these challenges are ideally suited to semi-autonomous surgical platforms to augment a surgeon’s skills with robotic and image- guided assistance. Despite the widespread implementation of robotic surgery and image guidance in other areas of the body, the field of neurotology has had relatively little adoption of this technology. We believe one reason for this is the precise registration needed in this field, where millimeter differences differentiate a successful from a catastrophic result. This CDA proposes expanding on my prior work investigating cooperative control robotics and virtual safety barriers by using computer vision and deep learning networks to develop highly accurate surgical image registration. This CDA aims to provide me with multi-disciplinary training in the departments of Otolaryngology, Biomedical Engineering and Computer Science. Specific training goals include: (1) Training in robotics, statistical shape modeling and computer tomography landmark segmentation, (2) Training in deep learning networks and computer vision video image registration, (3) Integrating this training, with my knowledge of temporal bone and skull base surgery to develop into an independent investigator (4) Pursue additional training in the ethical and responsible conduct of research. The research plan addresses the hypothesis that virtual safety barriers can be accurately enforced by a cooperative control robot, and computer vision methods can be used to automate accurate placement and registration of these safety barriers. I believe that the integration of these techniques will allow for semi- autonomous surgical methods resulting in improved surgical safety and efficiency. The specific aims of the proposal are to: (1) Develop and Validate Cooperative Control Robot Enforced Virtual Safety Barriers for Cortical Mastoidectomy (2) Develop and Test Autonomous Segmentation of Lateral Skull Base Anatomy (3) Develop Video-Based, Fiducial-less, Surgical Image Registration to Detect and Update the 3-D Position of Temporal Bone Anatomy from Intraoperative Stereoscopic Microscope Video.

项目摘要 我是约翰霍普金斯大学耳鼻咽喉头颈外科系的助理教授 医学院，在那里我的做法是专注于神经耳科学和侧颅底外科。我正在申请 获得指导外科医生科学家职业发展奖（CDA），以获得机器人技术的进一步培训，深入 学习和计算机视觉。这将进一步提高我的长期职业目标， 通过工程方法的新应用，最终研究半自主， 内耳和颅底的机器人干预超出了人手的限制。 在颞骨和侧颅底手术技术要求很高，因为复杂的三个- 三维解剖结构、小的工作空间和精细的神经血管结构。其中许多挑战 非常适合半自主手术平台，以增强外科医生的机器人和图像技能， 指导援助。尽管机器人手术和图像引导在其他领域得到广泛应用 在身体的各个方面，神经耳科学领域很少采用这种技术。我们相信一个原因 因为这是该领域所需的精确配准，其中毫米差异区分成功的 灾难性的结果这CDA建议扩大我以前的工作调查合作控制 利用计算机视觉和深度学习网络高度发展机器人技术和虚拟安全屏障 精确的手术图像配准。这个CDA旨在为我提供多学科的培训， 耳鼻喉科、生物医学工程和计算机科学系。具体的培训目标包括： (1)机器人、统计形状建模和计算机断层扫描标志分割培训，（2） 在深度学习网络和计算机视觉视频图像配准方面的训练，（3）整合这种训练， 凭借我对颞骨和颅底手术的了解，我将发展成为一名独立的调查员（4） 在道德和负责任的研究行为方面进行额外的培训。 该研究计划提出了一个假设，即虚拟安全屏障可以通过一个 合作控制机器人，计算机视觉方法可以用来自动精确放置， 这些安全屏障的登记。我相信，这些技术的整合将允许半- 自主手术方法，从而提高手术安全性和效率。的具体目标 提出了以下几点建议：（1）开发并实现了基于协作控制的机器人增强虚拟安全屏障系统 乳突切除术（2）开发和测试侧颅底解剖结构的自主分割（3）开发 基于视频的无基准点手术图像配准检测和更新颞叶三维位置 骨解剖从术中立体显微镜视频。

项目成果

A force-sensing surgical drill for real-time force feedback in robotic mastoidectomy.

用于机器人乳突切除术中实时力反馈的力感应手术钻。

• DOI: 10.1007/s11548-023-02873-7
• 发表时间: 2023
• 期刊: International journal of computer assisted radiology and surgery
• 影响因子: 3
• 作者: Chen,Yuxin;Goodridge,Anna;Sahu,Manish;Kishore,Aditi;Vafaee,Seena;Mohan,Harsha;Sapozhnikov,Katherina;Creighton,FrancisX;Taylor,RussellH;Galaiya,Deepa
• 通讯作者: Galaiya,Deepa

Tremor Assessment in Robot-Assisted Microlaryngeal Surgery Using Computer Vision-Based Tool Tracking.

使用基于计算机视觉的工具跟踪进行机器人辅助显微喉手术的震颤评估。

• DOI: 10.1002/ohn.714
• 发表时间: 2024
• 期刊: Otolaryngology--head and neck surgery : official journal of American Academy of Otolaryngology-Head and Neck Surgery
• 作者: Cho,SueM;Joo,HenryH;Golla,Pranathi;Sahu,Manish;Shankar,Ahjeetha;Trakimas,DanielleR;Creighton,Francis;Akst,Lee;Taylor,RussellH;Galaiya,Deepa
• 通讯作者: Galaiya,Deepa

Understanding Public Perceptions Regarding Cochlear Implant Surgery in Adults.

了解公众对成人人工耳蜗植入手术的看法。

• DOI: 10.1097/mao.0000000000003439
• 发表时间: 2022
• 期刊: Otology & neurotology : official publication of the American Otological Society, American Neurotology Society [and] European Academy of Otology and Neurotology
• 作者: Zhang,Lisa;Ding,AndyS;Xie,DeborahX;Creighton,FrancisX
• 通讯作者: Creighton,FrancisX