Visual Modelling and Rendering for Virtual Reality Based Surgical Simulation

缺乏有效的外科培训方法和工具一直是医学教育中存在的问题。目前的方法表现出各种局限性和缺点：动物具有与人类不同的解剖结构;尸体无法提供生理反应;外科住院医师对患者进行实践是有风险的。CBC最近的一份报告显示，很大一部分加拿大外科住院医生没有表现出基本的动手技能和手工灵巧。 基于虚拟现实的手术模拟器由于其许多独特的优点，越来越多地被认为是一个有前途的解决方案，目前的困境，代表下一代范式的外科手术培训。手术模拟器能够生成具有人体解剖结构和生理反应的虚拟患者，提供无风险的培训环境。除了这些明显的好处外，外科住院医生可以在各种病理状态下无限次练习;在对患者进行手术之前，可以在手术模拟器上安全地尝试新的复杂手术程序。 然而，目前的手术模拟器仍处于起步阶段，存在许多技术挑战和困难。具体地，没有模拟器能够提供与真实手术足够高的视觉相似度。针对这一问题，本文将研究手术仿真视觉反馈的几个关键问题：物理建模（即，软组织的生物力学特性的建模）、几何建模（软组织的变形的建模）和视觉渲染。预期的结果将在神经外科模拟器原型的开发中实施和验证。 长期目标是开发原则，算法和工具，这些对于实现高度逼真和自然的手术模拟视觉反馈至关重要，使教育机构和医院能够使用基于虚拟现实的模拟器培训外科医生（特别是神经外科医生）。为实现这一目标，具体研究目标如下： - 开发适当的模型，以纳入各向异性、粘弹性和生理特征，提高人体软组织物理（生物力学）特性建模的准确性; - 探索利用CT和/或MRI图像提取人体软组织生物力学参数的方法，解决现有方法存在的各种困难; - 建立人体软组织变形几何模型的无网格方法; - 开发渲染方法，以真实的手术视频为背景，无缝混合/集成模拟手术场景，实现高度的视觉沉浸感; - 应用预期结果与工业合作伙伴和临床医生合作开发新的神经外科模拟器。

The lack of effective methods and tools for surgical training has been a long-standing problem in medical education. Current approaches exhibit various limitations and drawbacks: animals have different anatomies from humans; cadavers are unable to provide physiological responses; it is risky for surgical residents to practice on patients. A recent CBC report shows that a great percentage of Canadian surgical residents do not demonstrate basic hands-on skills and manual dexterity.     Because of its many unique advantages, virtual-reality based surgical simulators are increasingly regarded as a promising solution to the current predicament, representing next-generation paradigm for surgical training. A surgical simulator is able to generate virtual patients with human anatomy and physiological responses, providing a risk-free training environment. In addition to these obvious benefits, surgery residents can practice on a wide range of pathological states with unlimited times; new complex surgical procedures can be tried safely on a surgical simulator before proceeding to patients.     Current surgical simulators, however, are still in their infancy and there exist numerous technical challenges and difficulties. In particular, no simulator is able to provide a sufficiently high degree of visual resemblance to real-life surgery. Aiming at solving this problem, this proposal will deal with some key issues about the visual feedback of surgical simulation: physical modelling (i.e., the modelling of soft tissue's biomechanical properties), geometric modelling (the modelling of soft tissue's deformation) and visual rendering. The anticipated results will be implemented and validated in the development of a neurosurgery simulator prototype.     The long-term goal is to develop principles, algorithms and tools that are essential to realize highly realistic and naturalistic visual feedback for surgical simulation, enabling educational institutions and hospitals to train surgeons (particularly neurosurgeons) using virtual-reality based simulators. To achieve this goal, specific research objectives are as follows:     - To develop suitable models to incorporate anisotropic, viscoelastic and physiological features, improving the modelling accuracy of the physical (biomechanical) properties of human soft tissue;     - To explore to extract the biomechanical parameters of human soft tissue using CT and/or MRI images, addressing the dilemma due to various difficulties associated with current methods;     - To develop a meshless approach to geometric models for the deformation of human soft tissue;     - To develop rendering methods to mix/integrate seamlessly simulated surgical scenes with real surgical video as the background, achieving high degree of visual immersion;      - To apply anticipated results to develop a new neurosurgery simulator in collaboration with the industrial partner and clinical practitioner.