Telerobotics Simulation in Mixed Reality

空间计算、机器学习和远程操作领域最新研究的融合，将改变现场、工业和医疗机器人技术。潜在的积极影响在对人类来说太危险的场景中最为明显，在这些场景中，机器人化身是不可替代的。深海就是这样一个场景。在本报告中，我们回顾了围绕无人水下航行器（uuv），远程操作和计算机视觉的几个主题的文献，以确定研究空白。在考虑多种差距的基础上，提出了一种混合现实模拟遥操作系统。在描述用于在水下环境中进行深度估计、目标检测和分割以及混合现实实验的同步立体视觉系统之前，研究了初步的计算机视觉和立体视觉研究。本文还讨论了未来在水箱中进行可变照明和浊度测试的工作，以及混合现实视听辅助设备的上下文感知功能。研究AIMS1。探索无人[水下]航行器遥控技术的现状和挑战。开发一种计算机视觉和/或图像处理技术，以提高水下航行器及其操纵器在操作过程中的效率和感知。以数据为中心的水下航行器在其环境中的仿真建模，实现机器人和场景数据的有效混合现实反馈。OBJECTIVES1.a。b.与行业利益相关者合作，收集水下航行器操作员的反馈。进行学术文献调查，以了解移动机器人远程操作的艺术状态和当前挑战。探索水下航行器市场，了解控制和传感能力。针对不同的应用案例/场景，对水下航行器的计算机视觉和水下图像处理进行学术文献调查。探索现有的计算机视觉技术和框架。实现水下航行器场景的管道。在给定场景下测试所开发技术的性能。开发可视化技术以支持人工操作。为远程机器人和环境的数据中心建模和仿真进行学术文献调查。研究目标3的进一步目标将在稍后阶段确定，并将取决于研究目标1和2的结果。

The convergence of state-of-the-art research in spatial computing, machine learning, and teleoperation proposes to transform field, industrial, and medical robotics. The potential positive impact is most observed in scenarios too dangerous to humans, where robotic avatars are irreplaceable. One such scenario is the deep sea. In this report we review literature in several topics surrounding unmanned underwater vehicles (UUVs), teleoperation, and computer vision to identify research gaps. Several gaps are considered, and a mixed reality simulated teleoperation system is proposed. Preliminary computer vision and stereo vision studies are investigated before a synchronised stereo vision system is described for performing depth estimation, object detection and segmentation, and mixed reality experiments in an underwater environment. Future work on variable lighting and turbidity tests in the tank, and context-awareness features of the mixed reality audiovisual aids are also discussed.RESEARCH AIMS1. Explore state of the art and challenges in remote-control of unmanned [underwater] vehicles.2. Develop a computer vision and/or image processing technique for enhancing efficiency and perception of an underwater vehicle and its manipulators during operation.3. Data-centric simulation modelling of an underwater vehicle in its environment for efficient mixed reality feedback of robot and scene data. OBJECTIVES1.a. Engage with industry stakeholders to collect feedback from human operators of underwater vehicles.b. Perform an academic literature survey to understand the state of the art and current challenges in mobile robot teleoperation.c. Explore underwater vehicles market to understand control and sensing capabilities.2.a. Perform academic literature survey for computer vision and underwater image processing for underwater vehicles for different applied use cases/scenarios.b. Explore existing computer vision techniques and frameworks.c. Implement pipeline for underwater vehicle scenario(s).d. Test performance of the developed technique for given scenario(s).e. Develop visualisation technique to support human operator.3.a. Perform academic literature survey for data-centric modelling and simulation of remote robots and environments.b. Further objectives of research aim 3 will be identified at a later stage and will depend on the outcomes of research aims 1 and 2.