Visual Modelling and Rendering for Virtual Reality Based Surgical Simulation

缺乏有效的外科培训方法和工具是医学教育中长期存在的问题。目前的方法显示出各种局限性和缺点：动物的解剖结构与人类不同；尸体无法提供生理反应；外科住院医师在病人身上实习是有风险的。加拿大广播公司最近的一份报告显示，很大比例的加拿大外科住院医生不具备基本的动手技能和手工灵巧性。由于其许多独特的优势，基于虚拟现实的手术模拟器越来越被认为是解决当前困境的有希望的解决方案，代表了下一代手术培训的范式。手术模拟器能够生成具有人体解剖和生理反应的虚拟患者，提供无风险的训练环境。除了这些明显的好处，外科住院医师可以在各种病理状态下无限制地练习；新的复杂的外科手术程序可以安全地在手术模拟器上进行，然后再进行患者。然而，目前的手术模拟器仍处于起步阶段，存在许多技术上的挑战和困难。特别是，没有任何模拟器能够提供与现实手术足够高的视觉相似性。针对这一问题，本课题将涉及手术模拟视觉反馈的几个关键问题：物理建模（即软组织生物力学特性的建模）、几何建模（软组织变形的建模）和视觉渲染。预期的结果将在神经外科模拟器原型的开发中得到实施和验证。长期目标是开发原则、算法和工具，这些原则、算法和工具对于实现手术模拟的高度逼真和自然的视觉反馈至关重要，使教育机构和医院能够使用基于虚拟现实的模拟器培训外科医生（特别是神经外科医生）。为实现这一目标，具体的研究目标如下：-开发合适的模型，将各向异性、粘弹性和生理特征结合起来，提高人体软组织物理（生物力学）特性的建模精度；-探索利用CT和/或MRI图像提取人体软组织的生物力学参数，解决当前方法所带来的各种困难；-开发人体软组织变形几何模型的无网格方法；-开发渲染方法，将模拟手术场景与真实手术视频作为背景进行无缝混合/融合，实现高度的视觉沉浸；-与工业伙伴和临床医生合作，应用预期结果开发新的神经外科模拟器。

The lack of effective methods and tools for surgical training has been a long-standing problem in medical education. Current approaches exhibit various limitations and drawbacks: animals have different anatomies from humans; cadavers are unable to provide physiological responses; it is risky for surgical residents to practice on patients. A recent CBC report shows that a great percentage of Canadian surgical residents do not demonstrate basic hands-on skills and manual dexterity.     Because of its many unique advantages, virtual-reality based surgical simulators are increasingly regarded as a promising solution to the current predicament, representing next-generation paradigm for surgical training. A surgical simulator is able to generate virtual patients with human anatomy and physiological responses, providing a risk-free training environment. In addition to these obvious benefits, surgery residents can practice on a wide range of pathological states with unlimited times; new complex surgical procedures can be tried safely on a surgical simulator before proceeding to patients.     Current surgical simulators, however, are still in their infancy and there exist numerous technical challenges and difficulties. In particular, no simulator is able to provide a sufficiently high degree of visual resemblance to real-life surgery. Aiming at solving this problem, this proposal will deal with some key issues about the visual feedback of surgical simulation: physical modelling (i.e., the modelling of soft tissue's biomechanical properties), geometric modelling (the modelling of soft tissue's deformation) and visual rendering. The anticipated results will be implemented and validated in the development of a neurosurgery simulator prototype.     The long-term goal is to develop principles, algorithms and tools that are essential to realize highly realistic and naturalistic visual feedback for surgical simulation, enabling educational institutions and hospitals to train surgeons (particularly neurosurgeons) using virtual-reality based simulators. To achieve this goal, specific research objectives are as follows:     - To develop suitable models to incorporate anisotropic, viscoelastic and physiological features, improving the modelling accuracy of the physical (biomechanical) properties of human soft tissue;     - To explore to extract the biomechanical parameters of human soft tissue using CT and/or MRI images, addressing the dilemma due to various difficulties associated with current methods;     - To develop a meshless approach to geometric models for the deformation of human soft tissue;     - To develop rendering methods to mix/integrate seamlessly simulated surgical scenes with real surgical video as the background, achieving high degree of visual immersion;      - To apply anticipated results to develop a new neurosurgery simulator in collaboration with the industrial partner and clinical practitioner.