ITR/SY: Electronic Books for the Tele-immersion Age: A New Paradigm for Teaching Surgical Procedures

远程沉浸将为相距遥远的人群提供一种戏剧性的新媒介，让他们在身临其境的3D虚拟环境中共同工作和分享经验，就像他们共同位于一个共享的物理空间一样。沉浸式电子书实际上将时间机器与3D超媒体融合在一起，它将增加一个额外的重要维度，即能够记录体验，沉浸在3D重建中的观众可以真正地在场景中漫步或在时间上来回移动。虽然这种新技术有许多潜在的应用领域(例如，设计和虚拟原型、维护和修复、古生物和考古重建)，但这里的重点将是一个具有社会重要性和技术挑战性的驾驶应用，教授疑难的、可能致命的伤害的外科处理。今天，外科创新的步伐已经大大加快，但培训和再培训的机制存在时间不灵活、时间承诺延长和内容有限的问题。传统的录像指导早已可以帮助外科医生学习新的手术程序，但这种方法只是略微有效，因为叙述中不可或缺的固定视角，缺乏深度感知和互动性，以及信息缺失；简而言之，观看视频的体验不够接近于身临其境和观看手术。在这个项目中，PI将开发一种新的手术程序教学范例，允许外科医生见证和探索(在时间和空间上)过去的手术程序，就像他们在那里一样，并增加了原始外科医生或另一位讲师的指导，以及集成的3D插图、注释和相关的医学元数据。受训者应该能够自由自然地在真人大小的高保真3D图形重建原始时变事件的周围走动，在时间上暂停或前进和后退，以满足好奇心或消除困惑。为了实现这一点，PI和他的团队聚集了几个学科的专家，并将能够共同利用他们在远程沉浸、时变3D场景捕获、交互隐喻和“电影”技术方面的先前工作。和创作工具。

Tele-immersion will provide a dramatic new medium for groups of people remote from each other to work and share experiences together in an immersive 3D virtual environment, much as if they were co-located in a shared physical space. Immersive electronic books that in effect blend a "time machine" with 3D hypermedia, will add an additional important dimension, that of being able to record experiences in which a viewer, immersed in the 3D reconstruction, can literally walk through the scene or move backward and forward in time. While there are many potential application areas for such novel technologies (e.g., design and virtual prototyping, maintenance and repair, paleontological and archaeological reconstruction), the focus here will be on a societally important and technologically challenging driving application, teaching surgical management of difficult, potentially lethal, injuries. Today, the pace of surgical innovations has increased dramatically, yet the mechanisms for training and re-training suffer from inflexible timing, extended time commitments, and limited content. Traditional videotaped instruction has long been available to help surgeons learn new procedures, but this approach is only marginally effective due to the fixed point of view that is integral to the narration, lack of depth perception and interactivity, and missing information; in short, the experience of watching a video is not sufficiently close to being there and seeing the procedure. In this project the PI will develop a new paradigm for teaching surgical procedures that allows surgeons to witness and explore (in time and space) a past surgical procedure as if they were there, with the added benefit of instruction from the original surgeon or another instructor, as well as integrated 3D illustrations, annotations, and relevant medical metadata. The trainees should be able to freely and naturally walk around a life-sized, high-fidelity, 3D graphical reconstruction of the original time-varying events, pausing or stepping forward and backward in time to satisfy curiosity or allay confusion. To make this reality, the PI and his team bring together experts in several disciplines, and will be able to collectively leverage their prior work in tele-immersion, time-varying 3D scene capture, interaction metaphors, "cinematic" techniques. and authoring tools.