Free Viewpoint Video System for MIS Training

拟议的研究项目旨在开发一个触觉视听环境（HAVE），以优化医疗培训经验，特别是因为它涉及微创手术（MIS）程序。目前，MIS培训是一个漫长而昂贵的过程，特别是在其早期阶段，因为必须尽量减少由于受训者的精细运动协调性差而导致的错误给患者带来的风险。这些问题由于手术室的时间非常宝贵而变得更加复杂，初级和高级住院医生相互谈判，以获得学习和完善这些困难的外科手术的机会。该项目的主要目标是创建一个协作培训环境，在该环境中，住院医生可以从培训外科医生执行的真实的手术中远程和实时学习手术。 通过记录每一个微妙的动作以及训练有素的手术团队在手术过程中的语音语调和视觉效果，初级住院医生可以通过高速网络实时接受远程专家的培训，或者事后通过多模式接口以回放模式进行培训，该多模式接口能够通过环境回放站和自由视点立体显示系统复制这些动作。为了实现这一目标，我们计划建立一个新的OR兼容的多摄像机系统，这将使我们能够模拟一个场景，然后从任意视点使用基于几何的视图插值技术回放过程。

The proposed research project aims at developing a Hapto-Audio-Visual Environment (HAVE) to optimize medical training experience, especially as it pertains to Minimally Invasive Surgical (MIS) procedures. Presently, MIS training is a long and expensive process, especially at its earliest stage because one must minimize the risks to the patient which arise from mistakes due to the poor fine-motor coordination of trainees. These issues are compounded by the fact that time in the operating room is at a premium, and junior and senior residents negotiate with each other for the opportunity to learn and perfect these difficult surgical procedures. The main goal of this project is to create a collaborative training environment in which residents can learn surgery, remotely and in real-time, from real operations performed by training surgeons. By recording every delicate move as well as voice intonations and visuals of a well-trained surgical team during a procedure, it is possible for junior residents to be trained by remote experts in real-time over a high-speed network, or after-the-fact in playback mode through a multi-modal interface capable of replicating those moves through an ambidextrous playback station and free viewpoint stereo display system. In order to reach this goal, we are planning to build a new OR-compatible multi-camera system that will allow us to digitize a scene and then play back the procedure from an arbitrary viewpoint using a disparity-based view interpolation technique.