Novel human-computer interaction paradigms for surgical augmented reality

点击翻译按钮获取中文摘要

The technology of augmented reality (AR) enables the superimposition of visual information onto real objects in our environment in order to facilitate the execution of certain tasks. By doing so, it is possible to radically improve the workflow in several work environments. Despite recent progress, AR is still largely absent from fields where precision, accuracy and safety are critical. The proposed research aims to remove the technological barriers to the adoption of AR for the specific application of computer-assisted surgery. Every AR application can be characterized by its approach to three key processing steps: the rendering of 3D virtual models, the alignment of virtual models and real images and the blending of rendered models with reality. To spark adoption of AR for a specific application domain, the perceptual and physical accuracy of each processing step must be analysed and often, improved. In this research program, we will formulate novel human-computer interaction paradigms that take advantage of the user's expertise to address the accuracy of every processing step in surgical AR, an application domain where nearly all sources of inaccuracies are exemplified. More specifically, we will 1) use inexpensive virtual reality head-mounted displays and hand tracking devices as a basis to design and implement new interactive methods that facilitate the exploration of 3D medical images (rendering component), 2) propose interactive semi-automatic techniques to correct for the misalignment of real and virtual components of an AR image (aligning component) and, 3) design and implement rendering methods that enable the perceptually sound fusion of real and virtual components of an AR image (blending component). At the end of the research program we will obtain a series of solutions to improve the physical and perceptual accuracy of each of the processing steps involved in the creation of AR images to guide surgical operations. More importantly, we will have defined a framework to analyse and correct the most important usability issues when addressing a new application domain for AR.