CAREER: Digital Twins of Surgical Environments for Situational Awareness and Immersive Simulation

职业：用于态势感知和沉浸式模拟的手术环境数字孪生

• 批准号: 2239077
• 负责人: Mathias Unberath
• 金额: $ 70万
• 依托单位: Johns Hopkins University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-07-01 至 2028-06-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2239077&HistoricalAwards=false
• 关键词: CAREER; Digital; Twins; Surgical; Environments

Developing new tools and procedures for robot-assisted surgery is difficult and expensive. This is because of the need for special-purpose tools and environments for building and testing prototype surgical robots, even before clinical testing can begin. This project seeks to address these problems through developing “digital twins” of image-guided surgery. The high level idea is to develop virtual reality simulations and computational models that closely mimic real patients, surgical tools, and operating environments. These simulations will help designers correct possible problems with robot and procedure designs early on; this in turn will allow faster, cheaper design cycles that will make robot-assisted surgery design more practical and widely available. The simulations can also be used to support real operations; room-scale models of the operating environment will support algorithms to monitor the robotic systems for safer surgeries, as well as provide new opportunities for educating both surgeons and the general public about ways technology is used to advance surgery.The technical aims of this project are divided into two thrusts: In the first thrust, the team will develop innovative methods that enable the creation of virtual immersive and interactive simulation environments that precisely recreate scenarios of the real world. To build these so-called digital twins, the team will create computer vision and machine learning techniques to create, update, and leverage digital twins of the surgical environment from multi-modal measurements. The second thrust will use digital twin models of unstructured, human-in-the-loop environments to explore how these immersive simulation models may be used to accelerate the design of intelligent surgical systems and provide complementary situational awareness to enhance decision making.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

开发机器人辅助手术的新工具和程序是困难和昂贵的。这是因为需要特殊用途的工具和环境来构建和测试原型手术机器人，甚至在临床测试开始之前。该项目旨在通过开发图像引导手术的“数字孪生”来解决这些问题。高层次的想法是开发虚拟现实模拟和计算模型，密切模仿真实的患者，手术工具和操作环境。这些模拟将帮助设计人员在早期纠正机器人和手术设计中可能存在的问题;这反过来将允许更快，更便宜的设计周期，使机器人辅助手术设计更加实用和广泛。模拟也可用于支持真实的手术;手术环境的房间规模模型将支持监控机器人系统的算法，以实现更安全的手术，并为外科医生和公众提供新的机会，让他们了解如何使用技术来推进手术。该项目的技术目标分为两个方面：在第一个推力中，该团队将开发创新方法，使虚拟沉浸式和交互式仿真环境的创建，精确地再现真实的世界的场景。为了构建这些所谓的数字双胞胎，该团队将创建计算机视觉和机器学习技术，以通过多模态测量来创建、更新和利用手术环境的数字双胞胎。第二个重点将使用非结构化的数字孪生模型，人在回路环境中，探索这些沉浸式仿真模型如何用于加速智能手术系统的设计，并提供互补的态势感知，以提高决策能力。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。