Augmented Visual Displays for Teleoperation

用于远程操作的增强视觉显示

• 批准号: RGPIN-2015-05304
• 负责人: Milgram, Paul
• 金额: $ 1.6万
• 依托单位: University of Toronto
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2018
• 资助国家: 加拿大
• 起止时间: 2018-01-01 至 2019-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=659785
• 关键词: Augmented; Visual; Displays; Teleoperation

The proposed research will focus on challenges posed by remote manipulation. In particular, we are considering situations in which a human operator is in some way `removed' from a worksite and thus, in order to visualise the remote site and interact with it, must rely upon some kind of sensor - such as cameras, endoscopes, microscopes, etc. Application domains that fall within this general class of teleoperation tasks include minimally invasive surgery, robotic surgery, remote training, military, space and underwater telerobotics and mining. Three related issues will be addressed, each with a defined objective.***Objective A: To develop and test a stereoscopic augmented reality display capability that will enable one to present computer generated / virtual objects as if they lie underneath a real object surface, thereby creating the illusion of transparency. To achieve this, we shall exploit the conflict that occurs in visual perception between stereoscopic depth cues and interposition cues. The research will investigate the properties of a real object surface that facilitate this phenomenon, as well as how to artificially manipulate the apparent texture of such surfaces to achieve this. Our ultimate aim is to develop a flexible tool that will optimise this transition to transparency, ideally for any arbitrary real object surface. ***The other two issues relate to the "keyhole effect", which results from the limited field of view (FOV) typically provided by video sensors and which restricts the viewer's ability to maintain awareness about information outside of the FOV. In earlier research we have demonstrated the usefulness of bifocal filtering for helping operators such as surgeons manipulate instruments outside the central magnified visual field, by providing a compressed image of that information. Objective B involves extending that concept to develop and test a polyfocal filtering capability, whose multiple foci will also be able to translate in real time in response to tracked instrument movements. Experimental testing of the concept will be based on surgically relevant three dimensional manipulation tasks.***Objective C is motivated by the emergence of multiple camera endoscopes, which address the keyhole problem by providing multiple points of view. In our earlier work we demonstrated that real-time interactive view synthesis can provide valuable depth cues to an operator. In the present research we shall both improve the robustness of our viewpoint synthesis algorithms and carry out a more extensive evaluation of the concept using more complex surgically relevant tasks.

拟议的研究将侧重于远程操作带来的挑战。特别是，我们正在考虑这样一种情况，即操作员被以某种方式从工作地点“移走”，因此，为了使远程地点可视化并与之互动，必须依赖某种传感器--例如照相机、内窥镜、显微镜等。属于这类一般远程操作任务的应用领域包括微创手术、机器人手术、远程训练、军事、空间和水下远程机器人和采矿。将讨论三个相关问题，每个问题都有一个明确的目标。*目标A：开发和测试立体增强现实显示能力，使人们能够呈现计算机生成的/虚拟对象，就像它们位于真实对象表面下一样，从而创造出透明的错觉。为了实现这一点，我们将利用视觉感知中立体深度线索和插入线索之间的冲突。这项研究将调查促成这一现象的真实物体表面的特性，以及如何人为地操纵此类表面的外观纹理来实现这一点。我们的最终目标是开发一种灵活的工具，以优化这种向透明度的过渡，理想情况下适用于任何任意的真实对象表面。*另外两个问题与“钥匙孔效应”有关，这是由于视频传感器通常提供的有限视野(FOV)造成的，这限制了观众对FOV之外的信息的感知能力。在早期的研究中，我们已经证明了双焦滤波通过提供信息的压缩图像来帮助操作员(如外科医生)在中央放大视野之外操纵器械的用处。目标B涉及扩展这一概念，以开发和测试多焦点过滤能力，其多焦点也将能够对跟踪的仪器移动做出实时平移。这一概念的实验测试将基于与手术相关的三维操作任务。*目标C的动机是多摄像机内窥镜的出现，它通过提供多个视角来解决锁孔问题。在我们之前的工作中，我们演示了实时交互视图合成可以为操作员提供有价值的深度线索。在目前的研究中，我们将提高我们的视点合成算法的健壮性，并使用更复杂的手术相关任务对概念进行更广泛的评估。