Augmented Visual Displays for Teleoperation

用于远程操作的增强视觉显示

• 批准号: RGPIN-2015-05304
• 负责人: Milgram, Paul
• 金额: $ 1.6万
• 依托单位: University of Toronto
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2019
• 资助国家: 加拿大
• 起止时间: 2019-01-01 至 2020-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=678599
• 关键词: Augmented; Visual; Displays; Teleoperation

The proposed research will focus on challenges posed by remote manipulation. In particular, we are considering situations in which a human operator is in some way `removed' from a worksite and thus, in order to visualise the remote site and interact with it, must rely upon some kind of sensor - such as cameras, endoscopes, microscopes, etc. Application domains that fall within this general class of teleoperation tasks include minimally invasive surgery, robotic surgery, remote training, military, space and underwater telerobotics and mining. Three related issues will be addressed, each with a defined objective.***Objective A: To develop and test a stereoscopic augmented reality display capability that will enable one to present computer generated / virtual objects as if they lie underneath a real object surface, thereby creating the illusion of transparency. To achieve this, we shall exploit the conflict that occurs in visual perception between stereoscopic depth cues and interposition cues. The research will investigate the properties of a real object surface that facilitate this phenomenon, as well as how to artificially manipulate the apparent texture of such surfaces to achieve this. Our ultimate aim is to develop a flexible tool that will optimise this transition to transparency, ideally for any arbitrary real object surface. ***The other two issues relate to the "keyhole effect", which results from the limited field of view (FOV) typically provided by video sensors and which restricts the viewer's ability to maintain awareness about information outside of the FOV. In earlier research we have demonstrated the usefulness of bifocal filtering for helping operators such as surgeons manipulate instruments outside the central magnified visual field, by providing a compressed image of that information. Objective B involves extending that concept to develop and test a polyfocal filtering capability, whose multiple foci will also be able to translate in real time in response to tracked instrument movements. Experimental testing of the concept will be based on surgically relevant three dimensional manipulation tasks.***Objective C is motivated by the emergence of multiple camera endoscopes, which address the keyhole problem by providing multiple points of view. In our earlier work we demonstrated that real-time interactive view synthesis can provide valuable depth cues to an operator. In the present research we shall both improve the robustness of our viewpoint synthesis algorithms and carry out a more extensive evaluation of the concept using more complex surgically relevant tasks.

拟议的研究将集中在远程操纵带来的挑战。特别是，我们正在考虑人类操作员以某种方式从工作现场“移除”的情况，因此，为了可视化远程站点并与之交互，必须依赖于某种传感器-例如相机，内窥镜，显微镜等。属于这种一般类别的远程操作任务的应用领域包括微创手术，机器人手术，远程培训，军事，太空和水下遥控机器人和采矿。 将讨论三个相关问题，每个问题都有一个明确的目标。目标A：开发和测试立体增强现实显示能力，使人们能够呈现计算机生成/虚拟对象，就好像它们位于真实的对象表面之下，从而创建透明的错觉。为了实现这一点，我们将利用立体深度线索和插入线索之间的视觉感知中发生的冲突。这项研究将调查一个真实的物体表面的属性，促进这种现象，以及如何人为地操纵这种表面的表观纹理，以实现这一点。我们的最终目标是开发一个灵活的工具，将优化这种过渡到透明，理想的任何任意真实的物体表面。* 另外两个问题与“锁眼效应”有关，这是由视频传感器通常提供的有限视场（FOV）引起的，并且限制了观看者保持对FOV之外的信息的感知的能力。在早期的研究中，我们已经证明了双焦点滤波的有用性，通过提供该信息的压缩图像，可以帮助手术者（如外科医生）在中央放大视野之外操作器械。 目标B涉及扩展该概念以开发和测试多焦点滤波能力，其多个焦点也将能够响应于跟踪的仪器运动而在真实的时间内平移。该概念的实验测试将基于手术相关的三维操作任务。*目标C的动机是多摄像头内窥镜的出现，它通过提供多个视角来解决锁孔问题。在我们早期的工作中，我们证明了实时交互式视图合成可以为操作员提供有价值的深度线索。在目前的研究中，我们将提高我们的观点合成算法的鲁棒性，并进行更广泛的评估的概念，使用更复杂的手术相关的任务。