Augmented Visual Displays for Teleoperation

点击翻译按钮获取中文摘要

The proposed research will focus on challenges posed by remote manipulation. In particular, we are considering situations in which a human operator is in some way `removed' from a worksite and thus, in order to visualise the remote site and interact with it, must rely upon some kind of sensor - such as cameras, endoscopes, microscopes, etc. Application domains that fall within this general class of teleoperation tasks include minimally invasive surgery, robotic surgery, remote training, military, space and underwater telerobotics and mining. Three related issues will be addressed, each with a defined objective.***Objective A: To develop and test a stereoscopic augmented reality display capability that will enable one to present computer generated / virtual objects as if they lie underneath a real object surface, thereby creating the illusion of transparency. To achieve this, we shall exploit the conflict that occurs in visual perception between stereoscopic depth cues and interposition cues. The research will investigate the properties of a real object surface that facilitate this phenomenon, as well as how to artificially manipulate the apparent texture of such surfaces to achieve this. Our ultimate aim is to develop a flexible tool that will optimise this transition to transparency, ideally for any arbitrary real object surface. ***The other two issues relate to the "keyhole effect", which results from the limited field of view (FOV) typically provided by video sensors and which restricts the viewer's ability to maintain awareness about information outside of the FOV. In earlier research we have demonstrated the usefulness of bifocal filtering for helping operators such as surgeons manipulate instruments outside the central magnified visual field, by providing a compressed image of that information. Objective B involves extending that concept to develop and test a polyfocal filtering capability, whose multiple foci will also be able to translate in real time in response to tracked instrument movements. Experimental testing of the concept will be based on surgically relevant three dimensional manipulation tasks.***Objective C is motivated by the emergence of multiple camera endoscopes, which address the keyhole problem by providing multiple points of view. In our earlier work we demonstrated that real-time interactive view synthesis can provide valuable depth cues to an operator. In the present research we shall both improve the robustness of our viewpoint synthesis algorithms and carry out a more extensive evaluation of the concept using more complex surgically relevant tasks.