Task-aware and Autonomous Robotic C-arm Servoing for Flouroscopy-guided Interventions

项目摘要 全美有1700多万例手术使用C臂X射线系统进行透视引导 并构成各种经皮手术的护理状态，包括骨盆环内fi固定术 受伤。为了从2D X线片推断手术过程，必须获得对解剖的良好的fi内观 并在手术过程中多次恢复。这一过程，也就是人们熟知的fluoro狩猎，与4.7%的S联系在一起 每个C臂位置的fl输尿管检查时间过长(例如，每个fiXation总共有120个S)，产生的X光片永远不会 经临床解释，但大大增加了手术时间和患者和外科工作人员的辐射剂量。 我们的长期项目目标是使用机器学习和主动视觉的概念来开发任务意识 自主解译术中X线片的自主机器人C臂伺服算法 调整C臂姿势以获取最适合推断的fl透视图像。我们有三个特定的fic目标： 1)从单平面fl输尿管检查图像中检测不利的K线轨迹：我们将扩展基于物理的SIM- 用于CT的fl输尿管内窥镜检查的模拟框架，支持快速生成结构化和逼真的X射线照片 记录程序进展。基于这些数据，我们将训练一种最先进的卷积神经网络- 解释fl透视图像以推断程序进展的工作。2)开发和验证任务感知 Silico成像系统：使用自主解释工具和可通过 目的1，训练一种基于强化学习和主动视觉的人工智能。这位特工将会是 能够分析术中fl超声图像以自主调整C臂姿势以产生任务- 解剖学上的最佳视角。3)证明我们的任务感知成像概念在体外的可行性：我们的第三个目标 将在受控的临床环境中建立任务感知的C臂成像。我们将尝试内部fixation 我们的有任务意识的fi代理将解释术中获得性骨折的原因。 用于推断程序进度并建议最佳C臂姿势的航海图，这些姿势将通过手动实现 光学跟踪移动C形臂系统。 这项工作结合了计算机科学家、外科机器人专家和整形外科专家的专业知识 创伤外科医生将探索尚未开发、研究不足的自主成像领域，这是由于 fl输卵管镜术中的机器学习指导程序。这种发展直到最近才变得可行。 通过CT在快速fl输尿管镜检模拟方面的创新，为培训提供结构化数据，这是最新的fi 对临床数据进行推广是现实的。在NIH开拓者奖的支持下，我们的团队 将是fi第一个研究自主和任务感知的C臂成像系统，为新的 医学图像采集的范式，它将通过面向任务的图像直接造福于数以百万计的患者fi 根据患者的具体情况购买fic。后续的R01资助将把这一概念定制为其他大批量 手术，如椎体成形术。

Project Summary Fluoroscopy guidance using C-arm X-ray systems is used in more than 17 million procedures across the US and constitutes the state-of-care for various percutaneous procedures, including internal fixation of pelvic ring injuries. To infer procedural progress from 2D radiographs, well-defined views onto anatomy must be achieved and restored multiple times during surgery. This process, known as ”fluoro hunting”, is associated with 4.7 s of excessive fluoroscopy time per C-arm position (c. f. 120 s total per fixation), yielding radiographs that are never interpreted clinically, but drastically increasing procedure time and radiation dose to patient and surgical staff. Our long-term project goal is to use concepts from machine learning and active vision to develop task-aware algorithms for autonomous robotic C-arm servoing that interpret intra-operative radiographs and autonomously adjust the C-arm pose to acquire fluoroscopic images that are optimal for inference. We have three specific aims: 1) Detecting unfavorable K-wire trajectories from monoplane fluoroscopy images: We will extend a physics-based sim- ulation framework for fluoroscopy from CT that enables fast generation of structured and realistic radiographs documenting procedural progress. Based on this data, we will train a state-of-the-art convolutional neural net- work that interprets fluoroscopic images to infer procedural progress. 2) Developing and validating a task-aware imaging system in silico: Using the autonomous interpretation tools and simulation pipeline available through Aim 1, we will train an artificial agent based on reinforcement learning and active vision. This agent will be capable of analyzing intra-operative fluoroscopic images to autonomously adjust the C-arm pose to yield task- optimal views onto anatomy. 3) Demonstrating feasibility of our task-aware imaging concept ex vivo: Our third aim will establish task-aware C-arm imaging in controlled clinical environments. We will attempt internal fixation of anterior pelvic ring fractures and our task-aware artificial agent will interpret intra-operatively acquired ra- diographs to infer procedural progress and suggest optimal C-arm poses that will be realized manually with an optically-tracked mobile C-arm system. This work combines the expertise of a computer scientist, a surgical robotics expert, and an orthopedic trauma surgeon to explore the untapped, understudied area of autonomous imaging enabled by advances in machine learning in fluoroscopy-guided procedures. This development has only recently been made feasible by innovations in fast fluoroscopy simulation from CT to provide structured data for training that is sufficiently realistic to warrant generalization to clinical data. With support from the NIH Trailblazer Award, our team will be the first to investigate autonomous and task-aware C-arm imaging systems, paving the way for a new paradigm in medical image acquisition, which will directly benefit millions of patients by task-oriented image acquisition on a patient-specific basis. Subsequent R01 funding will customize this concept to other high-volume procedures, such as vertebroplasty.